OpenAI Compatible Gateway

Connect to any OpenAI-compatible API endpoint

The OpenAI Compatible gateway connects ChatJS to any endpoint that follows the OpenAI API format. This is the fastest way to connect to local inference servers and alternative cloud providers.

Compatible Providers

This gateway works with:

Ollama — local models on your machine
LM Studio — local model management with a GUI
vLLM — high-throughput self-hosted inference
Azure OpenAI — OpenAI models on Azure
Any other endpoint that implements /v1/chat/completions and /v1/models

Setup

Set the base URL and (optionally) an API key in .env.local:

OPENAI_COMPATIBLE_BASE_URL=http://localhost:11434/v1  # Ollama example
OPENAI_COMPATIBLE_API_KEY=                             # Optional, depends on provider

Set the gateway in your config:

const config: ConfigInput = {
  ai: {
    gateway: "openai-compatible",
    workflows: {
      chat: "llama3.2",
      title: "llama3.2",
      // ...
    },
    // ...
  },
};

Authentication

Variable	Description
`OPENAI_COMPATIBLE_BASE_URL`	Required. The base URL of the API (e.g., `http://localhost:11434/v1`)
`OPENAI_COMPATIBLE_API_KEY`	Optional. API key if the provider requires authentication

Available Models

Depends entirely on your provider. Models are fetched at runtime from {OPENAI_COMPATIBLE_BASE_URL}/models and cached for 1 hour.

Ollama

After installing Ollama, pull models and they appear automatically:

ollama pull llama3.2
ollama pull codellama

LM Studio

Download models through the LM Studio UI, then start the local server. Models appear at http://localhost:1234/v1/models.

OPENAI_COMPATIBLE_BASE_URL=http://localhost:1234/v1

Image generation support depends on the provider. If the endpoint supports the OpenAI image generation API format, createImageModel() will work. Otherwise, multimodal language models with image output capabilities can still generate images inline.

Gateways Overview for the full gateway comparison
Custom Gateway if you need more control than this generic gateway provides