๐ Starting with OpenAI-Compatible Servers
Overviewโ
Open WebUI isn't just for OpenAI/Ollama/Llama.cppโyou can connect any server that implements the OpenAI-compatible API, running locally or remotely. This is perfect if you want to run different language models, or if you already have a favorite backend or ecosystem. This guide will show you how to:
- Set up an OpenAI-compatible server (with a few popular options)
- Connect it to Open WebUI
- Start chatting right away
Step 1: Choose an OpenAI-Compatible Serverโ
There are many servers and tools that expose an OpenAI-compatible API. Here are some of the most popular:
- Llama.cpp: Extremely efficient, runs on CPU and GPU
- Ollama: Super user-friendly and cross-platform
- LM Studio: Rich desktop app for Windows/Mac/Linux
- Lemonade (ONNX TurnkeyML): Fast ONNX-based backend with NPU/iGPU acceleration
Pick whichever suits your workflow!
๐ Get Started with Lemonade (ONNX TurnkeyML)โ
Lemonade is a plug-and-play ONNX-based OpenAI-compatible server. Hereโs how to try it on Windows:
-
Run
Lemonade_Server_Installer.exe
-
Install and download a model using Lemonadeโs installer
-
Once running, your API endpoint will be:
http://localhost:8000/api/v0
See their docs for details.
Step 2: Connect Your Server to Open WebUIโ
-
Open Open WebUI in your browser.
-
Go to โ๏ธ Admin Settings โ Connections โ OpenAI Connections.
-
Click โ Add Connection.
- URL: Use your serverโs API endpoint (for example,
http://localhost:11434/v1
for Ollama, or your own Llama.cpp serverโs address). - API Key: Leave blank unless required.
- URL: Use your serverโs API endpoint (for example,
-
Click Save.
Tip: If running Open WebUI in Docker and your model server on your host machine, use http://host.docker.internal:<your-port>/v1
.
For Lemonade: When adding Lemonade, use http://localhost:8000/api/v0
as the URL.โ
Step 3: Start Chatting!โ
Select your connected serverโs model in the chat menu and get started!
Thatโs it! Whether you choose Llama.cpp, Ollama, LM Studio, or Lemonade, you can easily experiment and manage multiple model serversโall in Open WebUI.
๐ Enjoy building your perfect local AI setup!