Skip to content
ChatJS
Esc
↑↓navigate↵open⌘Jpreview
On this page

Overview

Route AI requests through a pluggable gateway abstraction

ChatJS uses a gateway to route all AI requests (model listing, chat completions, and image generation) through a single backend. Install a gateway with the CLI to select its source, model defaults and credentials. ai.gateway must match the installed provider.

Available Gateways

Gateway Models Auth Image Generation Best For
Vercel AI Gateway (default) 120+ AI_GATEWAY_API_KEY or auto OIDC Dedicated image models Vercel deployments
OpenRouter Hundreds OPENROUTER_API_KEY Not declared by this adapter Broadest model access
OpenAI OpenAI only OPENAI_API_KEY gpt-image-1 and others Direct OpenAI access
LiteLLM 100+ providers LITELLM_BASE_URL + optional key Provider-dependent Unified proxy for all major providers
OpenAI Compatible Varies OPENAI_COMPATIBLE_API_KEY (optional) Provider-dependent Ollama, LM Studio, vLLM, Azure

Need a provider not listed here? See Custom Gateway.

Choosing a Gateway

  • Vercel AI Gateway is the default. It aggregates 120+ models from multiple providers behind a single key. Zero-config on Vercel deployments.
  • OpenRouter gives access to hundreds of models with per-token pricing. Good when you want the widest selection or models not on Vercel’s gateway.
  • OpenAI connects directly to the OpenAI API. Use this when you only need OpenAI models and want type-safe model IDs.
  • LiteLLM proxies requests to 100+ LLM providers (OpenAI, Anthropic, Google, Azure, Bedrock, and more) through a single unified endpoint. Ideal when you want provider-agnostic access with centralized key management and spend tracking.
  • OpenAI Compatible works with any endpoint that follows the OpenAI API format (local servers, self-hosted inference, or cloud services).

Configuration

Select a gateway during creation or explicitly replace it in an existing app:

bunx chat-js@alpha create my-chat --gateway openrouter
bunx chat-js@alpha add --gateway openrouter --replace

Replacement preserves runtime overrides. Review model IDs for the new provider and its supported media capabilities. Editing only ai.gateway cannot install a new implementation. See Configuration.

How Model Fetching Works

Every gateway implements fetchModels() which returns the list of available models. The app calls this at runtime and caches the result for 1 hour.

When no API key is available, the app falls back to a static snapshot in models.generated.ts. Refresh this snapshot periodically:

bun fetch:models

Snapshot Gateway Validation

The snapshot file records which gateway generated it via a generatedForGateway export. When you run bun check-env, it compares this value against config.ai.gateway and warns if they don’t match:

⚠️  models.generated.ts was built for "vercel" but config uses "openrouter".
    Run `bun fetch:models` to update the fallback snapshot.

This is caught at two levels:

  • Build time: bun check-env prints the warning shown above.
  • Runtime: each gateway checks generatedForGateway before using the fallback. If it doesn’t match, the fallback is skipped and an empty model list is returned instead of stale model IDs.

See Models for how the catalog is cached and refreshed.

Last updated on Nov 8, 2026

Was this page helpful?