Skip to main content

OpenAI-Compatible

Overview​

Open WebUI connects to any server or provider that implements the OpenAI-compatible API. This guide covers how to set up connections for popular cloud providers, gateways, and local servers.

For OpenAI itself (or Azure OpenAI), see the dedicated OpenAI guide.

Protocol-Oriented Design​

Open WebUI is built around Standard Protocols. Instead of building specific modules for every individual AI provider (which leads to inconsistent behavior and configuration bloat), Open WebUI focuses on protocols like the OpenAI Chat Completions Protocol.

This means that while Open WebUI handles the interface and tools, it expects your backend to follow the universal Chat Completions standard.

  • We Support Protocols: Any provider that follows widely adopted API standards is natively supported. We also have experimental support for Open Responses.
  • We Avoid Proprietary APIs: We do not implement provider-specific, non-standard APIs in the core to maintain a universal, maintainable codebase. For unsupported providers, use a pipe or a middleware proxy like LiteLLM or OpenRouter to bridge them.

For a detailed explanation of this architectural decision, see our FAQ on protocol support.

Important: Connection Verification May Fail for Some Providers

When you add a connection, Open WebUI verifies it by calling the provider's /models endpoint using a standard Bearer token. Some providers do not implement the /models endpoint at all or use non-standard authentication for it. In these cases:

  • The connection verification will fail with an error (e.g., 400, 401 or 403).
  • This does not mean the provider is incompatible: chat completions will still work.
  • You just need to manually add model names to the Model IDs (Filter) allowlist in the connection settings.

Providers with known /models issues:

Provider/models works?Action Needed
AnthropicYes, built-in compatibility layerAuto-detection works
GitHub ModelsNo, uses non-standard pathAdd model IDs manually to the whitelist
PerplexityNo, endpoint doesn't existAdd model IDs manually to the whitelist
MiniMaxNo, endpoint doesn't existAdd model IDs manually to the whitelist
OpenRouterYes, but returns thousands of modelsStrongly recommend adding a filtered allowlist
Vercel AI GatewayYesAuto-detection works; filtering is recommended if you only want specific gateway models
Google GeminiYesAuto-detection works
DeepSeekYesAuto-detection works
MistralYesAuto-detection works
GroqYesAuto-detection works

How to add models manually: In the connection settings, find Model IDs (Filter), type the model ID, and click the + icon, then save. The models will then appear in your model selector even though the connection verification showed an error. Each ID goes on the list once; typing one that is already there is refused with Model ID is already added, and any spaces around what you type are stripped off first.

Step 1: Add Your Provider Connection​

  1. Open Open WebUI in your browser.
  2. Go to Settings > Admin > Connections and find the Manage OpenAI API Connections list.
  3. Click ➕ Add Connection.
  4. Fill in the URL and API Key for your provider (see tabs below). The URL field will suggest common provider endpoints as you type.
  5. If your provider doesn't support /models auto-detection, add your model IDs to the Model IDs (Filter) allowlist.
  6. Click Save.
tip

If running Open WebUI in Docker and your model server is on the host machine, replace localhost with host.docker.internal in the URL.

Enable/Disable Connections

Each connection has a toggle switch that lets you enable or disable it without deleting the connection. This is useful for temporarily deactivating a provider while preserving its configuration.

Cloud Providers​

tip

See the dedicated Anthropic (Claude) guide for a full step-by-step walkthrough.

Anthropic (Claude) offers an OpenAI-compatible endpoint. Open WebUI includes a built-in compatibility layer that automatically detects Anthropic URLs and handles model discovery: just plug in your API key and models are auto-detected. Note that this is intended for testing and comparison. For production use with full Claude features (PDF processing, citations, extended thinking, prompt caching), Anthropic recommends their native API.

SettingValue
URLhttps://api.anthropic.com/v1
API KeyYour Anthropic API key from console.anthropic.com
Model IDsAuto-detected, leave empty or filter to specific models

Local Servers​

Llama.cpp runs efficient, quantized GGUF models locally with an OpenAI-compatible API server. See the dedicated Llama.cpp guide for full setup instructions (installation, model download, server startup).

SettingValue
URLhttp://localhost:10000/v1 (or your configured port)
API KeyLeave blank

Quick start:

./llama-server --model /path/to/model.gguf --port 10000 --ctx-size 1024 --n-gpu-layers 40
Connection Timeout Configuration

If your server is slow to start or you're connecting over a high-latency network, you can adjust the model list fetch timeout:

# Adjust timeout for slower connections (default is 10 seconds)
AIOHTTP_CLIENT_TIMEOUT_MODEL_LIST=5

If you've saved an unreachable URL and the UI becomes unresponsive, see the Model List Loading Issues troubleshooting guide for recovery options.

The Provider Setting​

Every OpenAI connection has a Provider dropdown, hidden behind Advanced in the connection dialog. It is a hint about what the server on the other end actually is. Open WebUI cannot tell a llama.cpp server from a LiteLLM proxy by looking at /v1/chat/completions, since both answer it the same way, so the dropdown is how you tell it, and each option switches on the extras that server supports.

Leaving it at Default is always safe. Chatting works the same with every option; the setting only adds capabilities around the chat.

OptionWhat it changes
DefaultNothing. The connection is treated as a plain OpenAI-compatible endpoint.
Azure OpenAIRequests are rewritten into Azure's deployment-based URL form with an api-version parameter, so the connection needs an API Version and the deployment names listed under Model IDs. Also switched on automatically for Azure-looking URLs, see the Azure OpenAI tab under Cloud Providers.
llama.cppUnlocks the model management panel, the green Loaded marker and the admin Eject button in the model selector, and the extra model details on the model rows. Stored reasoning is also replayed to the server as a reasoning_content field, which llama.cpp routes through the chat template. See the llama.cpp guide and reasoning models.
LM StudioUnlocks the same model management panel, loaded markers and model details, minus deleting models and minus the Eject button.
LiteLLMRequests arriving at Open WebUI's own Anthropic Messages endpoint are forwarded to LiteLLM as-is instead of being converted to Chat Completions and back. Nothing changes for chats started inside Open WebUI.
Where the dropdown is not shown

The Provider dropdown belongs to OpenAI connections only. Ollama connections are recognised on their own, and the personal Direct Connections users add in their own settings do not offer it.

Managing Local Models from Open WebUI​

Two provider types go beyond plain chat completions: setting Provider on the connection tells Open WebUI which management API the server speaks, which unlocks a model management panel for admins.

Open it from Settings > Admin > Models, the ⋯ menu, then Manage. When you have both Ollama and a managed provider connection, a picker at the top of the dialog switches between them; a second picker lists every managed connection by URL.

llama.cppLM Studio
List the server's modelsYesYes
Download a model by refYesYes
Load / unload a modelYesYes
Delete a cached modelYesNo

Everything in the panel is admin-only, and the actions are proxied to the local server rather than run in Open WebUI, so what happens to the files on disk is the server's business.

The connection URL you already use for chat is the right one here: Open WebUI strips a trailing /v1, /api/v1 or /api/v0 before calling the management API, so http://localhost:1234/v1 and http://127.0.0.1:10000/v1 both work unchanged.

From the model selector​

Admins do not have to open the panel to pull a model. Type a model ref into the model selector's search box and Open WebUI offers one Download row per target: Ollama if it is connected, plus every llama.cpp and LM Studio connection, each labelled with the provider it would download from. Arrow keys reach the rows and Enter starts the download. A row in progress shows a percentage and an ✕ to cancel it, and up to three downloads can run at once across all targets.

Model rows for these two providers also carry their parameter size, quantization and file size next to the name, taken from whatever the server reports.

Loaded models and eject​

Loaded models are marked in the model selector for both provider types. The Eject button that admins see on a loaded row only works for llama.cpp; on an LM Studio model it returns Provider "lmstudio" does not support model unloading, so unload those from the management panel instead.

Required API Endpoints​

To ensure full compatibility with Open WebUI, your server should implement the following OpenAI-standard endpoints:

EndpointMethodRequired?Purpose
/v1/modelsGETRecommendedUsed for model discovery and selecting models in the UI. If not available, add models to the allowlist manually.
/v1/chat/completionsPOSTYesThe core endpoint for chat, supporting streaming and parameters like temperature.
/v1/embeddingsPOSTNoRequired if you want to use this provider for RAG (Retrieval Augmented Generation).
/v1/audio/speechPOSTNoRequired for Text-to-Speech (TTS) functionality.
/v1/audio/transcriptionsPOSTNoRequired for Speech-to-Text (STT/Whisper) functionality.
/v1/images/generationsPOSTNoRequired for Image Generation (DALL-E) functionality.

Supported Parameters​

Open WebUI passes standard OpenAI parameters such as temperature, top_p, max_tokens (or max_completion_tokens), stop, seed, and logit_bias. It also supports Tool Use (Function Calling) if your model and server support the tools and tool_choice parameters.

Step 2: Start Chatting!​

Select your connected provider's model in the chat menu and get started!

That's it! Whether you choose a cloud provider or a local server, you can manage multiple connections, all from within Open WebUI.

🚀 Enjoy building your perfect AI setup!

This content is for informational purposes only and does not constitute a warranty, guarantee, or contractual commitment. Open WebUI is provided "as is." See your license for applicable terms.