Harbor
CLI that spins up full local LLM stacks with Docker in one command.
Overview
Harbor orchestrates local LLM backends such as Ollama, llama.cpp and vLLM together with front ends like Open WebUI and tools like SearXNG through Docker Compose. One command starts a pre-wired stack.
- Many backends and front ends
- Pre-wired services
- Config profiles
- Companion desktop app
Our verdict
What we like
- Great for experimenting
- Saves Docker wiring
Watch out for
- Docker knowledge helps
Works with these providers
Keys from these providers work in this app. Usage is billed by the provider at its list price.
BYOK Key Safety Score
Key handling in Harbor
Grade A
Your key is stored on your device and requests go directly to the provider.
- Key storage40/40
Key stays on your device; requests go direct to the provider.
- Auditable code25/25
Open source: anyone can verify how keys are handled.
- Self-hosting15/15
Can run entirely on your own infrastructure.
- Offline option10/10
Supports local models, so no key is needed at all for private work.
- Provider choice4/10
2 provider options, which lets you pick the provider whose data terms you trust.
Whatever the score, give each app its own key, set a monthly spend limit with your provider, and revoke keys you no longer use. Read the key-safety checklist.
Harbor FAQ
- Does Harbor support bring your own key?
- Yes. Harbor accepts your own API key from Ollama and OpenAI-compatible. Usage is billed by the provider at its API rates.
- Where does Harbor store my API key?
- On your device. Requests go directly from Harbor to the provider. Its BYOK Tools Key Safety Score is 94/100.
- Is Harbor free?
- Open source: Free & open source. API usage is billed separately by your model provider.
- Is Harbor open source?
- Yes. The source code is at https://github.com/av/harbor. It can also be self-hosted.
- Can Harbor run local models?
- Yes. It can connect to local runtimes such as Ollama or LM Studio, so no API key is needed for private or offline use.
- Which platforms does Harbor support?
- CLI, Self-hosted.
Community reviews
No reviews yet.
Alternatives to Harbor
All local ai →llama.cpp
Local AI
C and C++ LLM inference engine with a built-in server and web UI.
Ollama
Local AI
The simplest way to run open LLMs locally with one command.
Hollama
Local AI
Minimal web UI for Ollama and OpenAI-compatible servers.
LocalAI
Local AI
Self-hosted, drop-in OpenAI API replacement for local models.
Jan
Local AI
Open-source ChatGPT alternative that runs 100% offline.
Text Generation WebUI
Local AI
Gradio web UI for running local LLMs with many backends.