---
title: "OpenAI Compatible Gateway"
description: "Connect to any OpenAI-compatible API endpoint"
---

The OpenAI Compatible gateway connects ChatJS to any endpoint that follows the [OpenAI API format](https://platform.openai.com/docs/api-reference). This is the fastest way to connect to local inference servers and alternative cloud providers.

## Compatible Providers

This gateway works with:

- [Ollama](https://ollama.com) — local models on your machine
- [LM Studio](https://lmstudio.ai) — local model management with a GUI
- [vLLM](https://github.com/vllm-project/vllm): high-throughput self-hosted inference
- [Azure OpenAI](https://learn.microsoft.com/azure/ai-foundry/openai/overview) — OpenAI models on Azure
- Any other endpoint that implements `/v1/chat/completions` and `/v1/models`

## Setup

1. Set the base URL and (optionally) an API key in `.env.local`:

```bash
OPENAI_COMPATIBLE_BASE_URL=http://localhost:11434/v1  # Ollama example
OPENAI_COMPATIBLE_API_KEY=                             # Optional, depends on provider
```

2. Set the gateway in your config:

```typescript title="chat.config.ts"
const config: ConfigInput = {
  ai: {
    gateway: "openai-compatible",
    workflows: {
      chat: "llama3.2",
      title: "llama3.2",
      // ...
    },
    // ...
  },
};
```

## Authentication

| Variable | Description |
| --- | --- |
| `OPENAI_COMPATIBLE_BASE_URL` | Required. The base URL of the API (e.g., `http://localhost:11434/v1`) |
| `OPENAI_COMPATIBLE_API_KEY` | Optional. API key if the provider requires authentication |

## Available Models

Depends entirely on your provider. Models are fetched at runtime from `{OPENAI_COMPATIBLE_BASE_URL}/models` and cached for 1 hour.

### Ollama

After [installing Ollama](https://ollama.com/download), pull models and they appear automatically:

```bash
ollama pull llama3.2
ollama pull codellama
```

### LM Studio

Download models through the LM Studio UI, then start the local server. Models appear at `http://localhost:1234/v1/models`.

```bash
OPENAI_COMPATIBLE_BASE_URL=http://localhost:1234/v1
```

## Image Generation

Image generation support depends on the provider. If the endpoint supports the OpenAI image generation API format, `createImageModel()` will work. Otherwise, multimodal language models with image output capabilities can still generate images inline.

## Related

- [Gateways Overview](/gateways/overview) for the full gateway comparison
- [Custom Gateway](/gateways/custom) if you need more control than this generic gateway provides