Skip to content
BYOK Tools

Local AI

Running local models with Ollama alongside cloud API keys

Run open-weight models on your own machine through Ollama's local API, connect them to BYOK apps through the OpenAI-compatible endpoint, and use cloud keys only for harder tasks.

By the BYOK Tools editorial team · 6 November 2026 · 7 min read

BYOK has a counterpart: bring your own model. Tools such as Ollama and LM Studio run open-weight models on your computer and expose them through a local HTTP API. Many BYOK apps connect to them the same way they connect to OpenAI.

Reasons to run models locally

  • No per-token cost. After the download, a local model costs nothing per request beyond electricity.
  • Data stays on the machine. Prompts and responses never leave your device.
  • Offline use. Useful when travelling or in restricted environments.
  • No rate limits other than your hardware.

Hardware

A recent laptop with 16 GB of memory runs small and mid-sized models well. Apple Silicon Macs perform strongly because the GPU shares unified memory. Larger models need 32 GB or more, or a dedicated GPU with enough VRAM. Quantised builds trade a small amount of quality for much lower memory use.

Setting up Ollama

  1. Install Ollama from ollama.com.

  2. Pull a model from the Ollama library, starting with a small general-purpose model. For example:

     ollama pull <model-name>
     ollama run <model-name>
    
  3. The server listens on http://localhost:11434. Its native API lives under /api (for example /api/chat), and it also serves an OpenAI-compatible endpoint at http://localhost:11434/v1.

To connect a BYOK app, add an OpenAI-compatible provider with:

  • Base URL: http://localhost:11434/v1
  • API key: any non-empty string (Ollama ignores it, but many clients require a value)
  • Model: the exact name shown by ollama list

To reach Ollama from another machine or a container, set the OLLAMA_HOST environment variable so the server binds to a network interface rather than localhost only, and restrict access with a firewall.

Context length

Local servers often default to a shorter context window than cloud models. If long documents or chat histories are being cut off, raise the context length setting for the model. Larger contexts use more memory.

LM Studio

LM Studio provides a desktop interface for downloading models and runs a local OpenAI-compatible server, by default at http://localhost:1234/v1. Configuration in BYOK apps is the same as for Ollama.

A hybrid setup

A common arrangement: a local model for drafts, quick questions and private data, and a cloud key for complex reasoning and coding. Filter the directory by Local models to find apps that support both.

Editorial independence: rankings are based on our own testing and research. Some outbound links may in future be affiliate links; that never affects scores. See our editorial policy.

Get the BYOK Brief

New listings, price changes and model releases. Fortnightly.

Fortnightly: new listings, price changes and model releases. Unsubscribe at any time.