Local AI
Running local models with Ollama alongside cloud API keys
Run open-weight models on your own machine through Ollama's local API, connect them to BYOK apps through the OpenAI-compatible endpoint, and use cloud keys only for harder tasks.
By the BYOK Tools editorial team · 6 November 2026 · 7 min read
BYOK has a counterpart: bring your own model. Tools such as Ollama and LM Studio run open-weight models on your computer and expose them through a local HTTP API. Many BYOK apps connect to them the same way they connect to OpenAI.
Reasons to run models locally
- No per-token cost. After the download, a local model costs nothing per request beyond electricity.
- Data stays on the machine. Prompts and responses never leave your device.
- Offline use. Useful when travelling or in restricted environments.
- No rate limits other than your hardware.
Hardware
A recent laptop with 16 GB of memory runs small and mid-sized models well. Apple Silicon Macs perform strongly because the GPU shares unified memory. Larger models need 32 GB or more, or a dedicated GPU with enough VRAM. Quantised builds trade a small amount of quality for much lower memory use.
Setting up Ollama
Install Ollama from ollama.com.
Pull a model from the Ollama library, starting with a small general-purpose model. For example:
ollama pull <model-name> ollama run <model-name>The server listens on http://localhost:11434. Its native API lives under /api (for example /api/chat), and it also serves an OpenAI-compatible endpoint at http://localhost:11434/v1.
To connect a BYOK app, add an OpenAI-compatible provider with:
- Base URL: http://localhost:11434/v1
- API key: any non-empty string (Ollama ignores it, but many clients require a value)
- Model: the exact name shown by ollama list
To reach Ollama from another machine or a container, set the OLLAMA_HOST environment variable so the server binds to a network interface rather than localhost only, and restrict access with a firewall.
Context length
Local servers often default to a shorter context window than cloud models. If long documents or chat histories are being cut off, raise the context length setting for the model. Larger contexts use more memory.
LM Studio
LM Studio provides a desktop interface for downloading models and runs a local OpenAI-compatible server, by default at http://localhost:1234/v1. Configuration in BYOK apps is the same as for Ollama.
A hybrid setup
A common arrangement: a local model for drafts, quick questions and private data, and a cloud key for complex reasoning and coding. Filter the directory by Local models to find apps that support both.
Editorial independence: rankings are based on our own testing and research. Some outbound links may in future be affiliate links; that never affects scores. See our editorial policy.
Get the BYOK Brief
New listings, price changes and model releases. Fortnightly.