Overview
Route AI requests through a pluggable gateway abstraction
ChatJS uses a gateway to route all AI requests (model listing, chat completions, and image generation) through a single backend. Install a gateway with the CLI to select its source, model defaults and credentials. ai.gateway must match the installed provider.
Available Gateways
| Gateway | Models | Auth | Image Generation | Best For |
|---|---|---|---|---|
| Vercel AI Gateway (default) | 120+ | AI_GATEWAY_API_KEY or auto OIDC |
Dedicated image models | Vercel deployments |
| OpenRouter | Hundreds | OPENROUTER_API_KEY |
Not declared by this adapter | Broadest model access |
| OpenAI | OpenAI only | OPENAI_API_KEY |
gpt-image-1 and others |
Direct OpenAI access |
| LiteLLM | 100+ providers | LITELLM_BASE_URL + optional key |
Provider-dependent | Unified proxy for all major providers |
| OpenAI Compatible | Varies | OPENAI_COMPATIBLE_API_KEY (optional) |
Provider-dependent | Ollama, LM Studio, vLLM, Azure |
Need a provider not listed here? See Custom Gateway.
Choosing a Gateway
- Vercel AI Gateway is the default. It aggregates 120+ models from multiple providers behind a single key. Zero-config on Vercel deployments.
- OpenRouter gives access to hundreds of models with per-token pricing. Good when you want the widest selection or models not on Vercel’s gateway.
- OpenAI connects directly to the OpenAI API. Use this when you only need OpenAI models and want type-safe model IDs.
- LiteLLM proxies requests to 100+ LLM providers (OpenAI, Anthropic, Google, Azure, Bedrock, and more) through a single unified endpoint. Ideal when you want provider-agnostic access with centralized key management and spend tracking.
- OpenAI Compatible works with any endpoint that follows the OpenAI API format (local servers, self-hosted inference, or cloud services).
Configuration
Select a gateway during creation or explicitly replace it in an existing app:
bunx chat-js@alpha create my-chat --gateway openrouter
bunx chat-js@alpha add --gateway openrouter --replace
Replacement preserves runtime overrides. Review model IDs for the new provider and its supported media capabilities. Editing only ai.gateway cannot install a new implementation. See Configuration.
How Model Fetching Works
Every gateway implements fetchModels() which returns the list of available models. The app calls this at runtime and caches the result for 1 hour.
When no API key is available, the app falls back to a static snapshot in models.generated.ts. Refresh this snapshot periodically:
bun fetch:models
Snapshot Gateway Validation
The snapshot file records which gateway generated it via a generatedForGateway export. When you run bun check-env, it compares this value against config.ai.gateway and warns if they don’t match:
⚠️ models.generated.ts was built for "vercel" but config uses "openrouter".
Run `bun fetch:models` to update the fallback snapshot.
This is caught at two levels:
- Build time:
bun check-envprints the warning shown above. - Runtime: each gateway checks
generatedForGatewaybefore using the fallback. If it doesn’t match, the fallback is skipped and an empty model list is returned instead of stale model IDs.
See Models for how the catalog is cached and refreshed.
Related
- Models for model configuration and visibility
- Custom Gateway for implementing your own gateway