AnythingLLM
All-in-one desktop and Docker app for private document chat and agents.
Overview
AnythingLLM turns documents into workspaces you can chat with, combining RAG, agents and a built-in vector database. It runs on desktop or Docker and connects to local models or any major provider via your own keys.
- Document workspaces
- Built-in vector DB
- AI agents
- Multi-user (Docker)
- Embeddable widget
Our verdict
What we like
- Easy private RAG
- Huge provider support
Watch out for
- Large documents need tuning
Works with these providers
Keys from these providers work in this app. Usage is billed by the provider at its list price.
BYOK Key Safety Score
Key handling in AnythingLLM
Grade A
Your key is stored on your device and requests go directly to the provider.
- Key storage40/40
Key stays on your device; requests go direct to the provider.
- Auditable code25/25
Open source: anyone can verify how keys are handled.
- Self-hosting15/15
Can run entirely on your own infrastructure.
- Offline option10/10
Supports local models, so no key is needed at all for private work.
- Provider choice10/10
16 provider options, which lets you pick the provider whose data terms you trust.
Whatever the score, give each app its own key, set a monthly spend limit with your provider, and revoke keys you no longer use. Read the key-safety checklist.
AnythingLLM FAQ
- Does AnythingLLM support bring your own key?
- Yes. AnythingLLM accepts your own API key from OpenAI, Anthropic, Google Gemini, OpenRouter, Mistral, Groq, DeepSeek, xAI, Ollama, LM Studio, Azure OpenAI, AWS Bedrock, Perplexity, Together AI, Cohere and OpenAI-compatible. Usage is billed by the provider at its API rates.
- Where does AnythingLLM store my API key?
- On your device. Requests go directly from AnythingLLM to the provider. Its BYOK Tools Key Safety Score is 100/100.
- Is AnythingLLM free?
- Open source: Free & open source. API usage is billed separately by your model provider.
- Is AnythingLLM open source?
- Yes. The source code is at https://github.com/Mintplex-Labs/anything-llm. It can also be self-hosted.
- Can AnythingLLM run local models?
- Yes. It can connect to local runtimes such as Ollama or LM Studio, so no API key is needed for private or offline use.
- Which platforms does AnythingLLM support?
- macOS, Windows, Linux, Self-hosted, Web.
Community reviews
No reviews yet.
Alternatives to AnythingLLM
All notes & knowledge →RAGFlow
Notes & Knowledge
Open-source RAG engine built on deep document understanding.
Onyx
Notes & Knowledge
Open-source enterprise search and AI assistant over your company tools.
Copilot for Obsidian
Notes & Knowledge
AI chat and vault Q&A inside Obsidian, using your keys.
LobeChat (LobeHub)
Chat Clients
Modern open-source AI chat with agents, plugins and dozens of providers.
Kotaemon
Notes & Knowledge
Clean open-source RAG UI for chatting with your documents.
AIChat
Command Line
All-in-one LLM CLI with chat REPL, RAG, agents and a local server.