Ollama vs llama.cpp — the wrapper, and the engine it wraps.
This is not two competing runtimes. Ollama's own README lists llama.cpp under "Supported backends", so the question is what the wrapper adds. Less than it used to: llama.cpp now installs from a script, pulls models straight from Hugging Face, ships a web UI, speaks both the OpenAI and Anthropic APIs, and can hot-swap models in router mode. Ollama still wins on packaging, model naming and ecosystem defaults — llama.cpp wins on release speed and control.
| Category | Winner | Margin |
|---|---|---|
| Install · getting to a running model | AOllama | Ollama installs an OS service that starts on boot; llama.cpp offers an installer at llama.app, a Docker image or prebuilt binaries, all of which you start yourself |
| Getting a model · the download step | ·Tie | This used to be the gap. Ollama has human tags — ollama run gemma4 — while llama.cpp now takes -hf ggml-org/Qwen3.5-0.8B-GGUF and pulls from Hugging Face directly |
| Release speed · how fast new models arrive | Bllama.cpp | llama.cpp published 18 releases on Aug 27, 2026 alone and 94 in the preceding ten days; Ollama's latest was v0.33.1 on Aug 26 — support for a brand-new architecture lands upstream first |
| Hardware reach · what it will run on | Bllama.cpp | llama.cpp lists 17 backends including Vulkan, SYCL, OpenCL, Ascend NPU, Moore Threads and IBM Z; Ollama exposes the CUDA, Metal and ROCm paths |
| Models bigger than VRAM · when the card is too small | Bllama.cpp | llama.cpp documents CPU+GPU hybrid inference to partially accelerate models larger than total VRAM; Ollama requires a model to fit in VRAM to load it concurrently |
| Serving several requests · concurrency | Bllama.cpp | llama-server has slots with a unified KV buffer and a /slots metrics endpoint; Ollama's OLLAMA_NUM_PARALLEL scales memory by parallel requests times context length |
| Multiple models at once · load, swap, unload | ·Tie | Ollama keeps three per GPU with keep_alive timers; llama-server's router mode loads up to four by default and forwards each request to the right instance |
| API surface · what a client can call | ·Tie | Both expose OpenAI chat/responses/embeddings plus the Anthropic Messages API; llama-server adds API-key auth and GBNF grammars, Ollama adds its own /api with pull and Modelfile |
| Quantization · making a model fit | Bllama.cpp | llama.cpp is where GGUF quants are made: llama-quantize, imatrix and perplexity ship in the repo, from 1.5-bit up to 8-bit; Ollama consumes the output |
| Measuring speed · knowing what you got | Bllama.cpp | llama-bench and batched-bench are first-class tools in the repo; Ollama has no equivalent benchmark harness |
| Ecosystem defaults · what third-party tools expect | AOllama | Ollama's docs carry integration pages for Claude Code, Codex, opencode, Cline, Zed, VS Code, JetBrains, n8n and more; most 'local model' toggles mean Ollama |
| Hosted fallback · when local is not enough | AOllama | Ollama sells cloud models behind the same CLI from $20/mo; llama.cpp has no hosted offering and never will — it is a library and a binary |
| Best overall · for most people, most of the time | ·Depends | Ollama if you want a service that other software finds; llama.cpp if you want the flags, the newest architectures and the quantization toolchain |
CHOOSE A · OLLAMA
If you want a local model to behave like infrastructure.
- It runs itself — installs as a service, starts with the machine, unloads idle models on a timer
- Human model names —
ollama run gemma4resolves a tag; no repo path, no quant filename, no flags - Everything integrates with it — around 25 integration pages in its own docs, from Claude Code to JetBrains to n8n
- Modelfile — pin a system prompt, parameters and a template into a named model you can share
- Cloud escape hatch — a
:cloudtag runs a model too big for your box through the same commands
CHOOSE B · LLAMA.CPP
If you want the knobs, and them first.
- New architectures land here — 18 releases in one day is not a typo; Ollama inherits this work later
- Seventeen backends — Vulkan, SYCL, OpenCL, HIP, CANN, MUSA, zDNN and more, well past the CUDA/Metal/ROCm mainstream
- Runs what does not fit — CPU+GPU hybrid inference for models larger than your total VRAM
- The quantization toolchain —
llama-quantize,imatrixandperplexitylet you make and check your own quants - Real serving controls — parallel slots, a unified KV buffer, prompt caching, API keys and a
/slotsmetrics endpoint - GBNF grammars — constrain output to a grammar you write, not just a JSON schema
| Aspect | Ollama | llama.cpp |
|---|---|---|
| The softwarewhat it costs to run locally | $0 Installers for macOS, Windows and Linux plus an official Docker image | $0 Install script at llama.app, prebuilt release binaries, a Docker image, or build from source |
| Licencethe terms on the code | MIT Go codebase, 179K stars, MIT throughout | MIT C/C++ on top of the ggml library, 126K stars, MIT throughout |
| Commercial useshipping it inside something | Unrestricted Same MIT terms at home and at work; no separate commercial licence exists | Unrestricted Same MIT terms; the library is designed to be embedded, and XCFramework builds are documented for iOS and macOS apps |
| Hosted modelsrenting someone else's GPU | $20–$100/mo Pro $20/mo or $200/year, Max $100/mo with new signups paused; the allowance is described as '50x more usage' rather than in tokens A wins | Not offered No hosted service and no account — the project ships code, and the cloud question is left to whoever deploys it |
| Team planseats and shared billing | $25/seat/mo Five-seat minimum, marked introductory; US/Europe hosting, zero data retention, priority support A wins | Not applicable Nothing to buy a seat on; teams standardise on a build and a set of flags instead |
| Supportwhen it breaks | Community + paid tiers GitHub issues, plus priority support attached to the Team and Enterprise plans | Community GitHub issues and discussions, with a named maintainer group and public dev stats |
| What you actually paythe invoice neither project sends | Hardware + power VRAM sets which models you can run and how much context they get | Hardware + power Same machine, lower floor: hybrid CPU+GPU inference will run a model that does not fit in VRAM, slowly B wins |
| Model weightsthe file being executed | $0 Ollama's own library of tags, or Hugging Face GGUF repos | $0 Any GGUF on Hugging Face via -hf, or one you quantized yourself |
| Capability | Ollama | llama.cpp |
|---|---|---|
| Relationship | Wraps llama.cpp | Is the engine |
| Licence | ✓ MIT | ✓ MIT |
| Runs as an OS service | ✓ installed by default | ~ you start llama-server |
| Named model tags | ✓ own registry | ~ Hugging Face repo paths |
| Pull from Hugging Face | ✓ hf.co tags | ✓ -hf owner/repo |
| Built-in web UI | ✓ desktop app | ✓ served by llama-server |
| OpenAI chat completions | ✓ | ✓ |
| OpenAI Responses API | ✓ /v1/responses | ✓ responses route |
| Anthropic Messages API | ✓ /v1/messages | ✓ compatible route |
| API-key authentication | ~ keys for cloud, not local | ✓ --api-key, multiple keys |
| Multiple models loaded | ✓ 3 × GPU count | ✓ router mode, 4 by default |
| Automatic unload of idle models | ✓ keep_alive | ~ router autoload, manual unload |
| Parallel request slots | ✓ default 1, RAM scales linearly | ✓ auto slots, unified KV buffer |
| Per-slot metrics endpoint | ✗ | ✓ /slots |
| Prompt caching to disk | ~ in-memory only | ✓ idle slots saved to prompt cache |
| K/V cache quantization | ✓ q8_0 / q4_0 | ✓ cache type flags |
| Quantize a model yourself | ✗ consumes GGUF | ✓ llama-quantize + imatrix |
| Perplexity / quality check | ✗ | ✓ llama-perplexity |
| Benchmark harness | ✗ | ✓ llama-bench, batched-bench |
| Grammar-constrained output | ✓ JSON schema | ✓ GBNF grammars + JSON schema |
| Vision / multimodal | ✓ | ✓ mtmd subsystem |
| Text-to-speech tooling | ✗ | ✓ tools/tts |
| Distributed inference over network | ✗ | ✓ RPC backend |
| CPU+GPU hybrid for oversized models | ~ CPU fallback, not partial offload control | ✓ documented |
| Backends | CUDA, Metal, ROCm | 17 listed, incl. Vulkan, SYCL, CANN, MUSA, zDNN |
| Mobile / embedded builds | ✗ | ✓ Android build + XCFramework |
| Documented third-party integrations | ✓ ~25 in its own docs | ~ via the OpenAI endpoint |
| Hosted models | ✓ :cloud tags | ✗ |
| Release cadence | Versioned, every few days | Nightly builds, often many per day |
| Latest release seen | v0.33.1 · Aug 26, 2026 | b10664 · Aug 27, 2026 |
§ 04 / BENCHMARKS
Why there's no chart here.
No neutral benchmark covers this pair. Neutral leaderboards score models, not the code that loads them — and here the two sides are not even independent: Ollama's README lists llama.cpp under "Supported backends", so a chart would largely be llama.cpp against itself with a wrapper on one side. Throughput gaps that do appear come from build flags, quantization, context length and slot count, which makes any number a fact about one test box rather than about either project. We compare the things that hold across machines: what each ships, what it costs, what hardware it reaches and how fast new model support arrives.
A · OLLAMA
Ollama
A packaging and lifecycle layer that turns the engine into something the rest of your machine can rely on.
Strengths
- Lifecycle is the product — install once and a service is running on :11434, loading models on demand and unloading them on a
keep_alivetimer - Naming that humans use — a registry of tags means
ollama run gemma4instead of a repo path plus a quant suffix - The default local backend — its docs carry integration pages for Claude Code, Codex, opencode, Cline, Zed, VS Code, JetBrains, Xcode, Goose, n8n and more
- Modelfile — bake a system prompt, template and parameters into a named model and hand it to a colleague
- Cloud when local runs out —
:cloudtags execute on Ollama's hardware through the same CLI, API and tools - Speaks Anthropic too — two environment variables point Claude Code at a model running on your own machine
Weaknesses
- The 4K default — under 24 GiB of VRAM a model gets a 4K window unless you set
OLLAMA_CONTEXT_LENGTH, which its own docs say should be at least 64K for agents - Parallelism is expensive — required RAM scales by parallel requests times context length, because each slot gets a separate window
- A version behind — new architectures land in llama.cpp first and arrive here on Ollama's own release cadence
- No tooling to make quants — you consume what someone else uploaded, and you cannot check it with a perplexity run
Best for
- Serving models to editors, agents and scripts without babysitting a process
- Teams that want one command in the onboarding doc
- Anyone whose next step is pointing Claude Code or opencode at a local model
- Machines where the model should be present but invisible
B · LLAMA.CPP
llama.cpp
The C/C++ engine most local inference runs on, and a full toolchain around it that people keep forgetting is there.
Strengths
- Where the work happens — 18 releases on a single day in August 2026, and the place a new architecture is supported first
- Hardware nobody else reaches — 17 backends, from Vulkan and SYCL to Ascend NPUs, Moore Threads GPUs and IBM Z
- Quantization is yours —
llama-quantizewith importance matrices, from 1.5-bit to 8-bit, andllama-perplexityto see what the squeeze cost - A real server — parallel slots, unified KV buffer, disk prompt cache, API keys, a
/slotsmetrics endpoint and router mode for several models - It got easy — an installer at llama.app,
-hfto pull from Hugging Face and a built-in web UI, none of which the folklore about this project mentions - Embeddable — plain C/C++ with no dependencies, an Android build and an XCFramework for Apple platforms
Weaknesses
- You are the operator — no service is installed, no model is unloaded for you, and the flag list is long enough to have its own table
- Nightly cadence cuts both ways — pinning a build is your job, and 'latest' can change under you several times a day
- No registry — models are repo paths and file names, not tags, which is fine until you are onboarding somebody
- No hosted option — when a model does not fit, the answer is a bigger machine or another vendor
Best for
- Running a model on hardware without a mainstream CUDA path
- Squeezing a large model onto a small card with a custom quant
- Anyone who wants to measure throughput rather than guess at it
- Embedding local inference in a desktop or mobile application
01
You just want a local model behind your editor
A working machine, an editor with a local-model setting, and no wish to learn a new binary this week.
Reasoning: Ollama installs a service, the editor's local option almost certainly names it, and the model is a tag rather than a file path. Nothing about llama.cpp prevents this, but you would be writing the launch command and the systemd unit yourself for an outcome that is identical.
Picked
Ollama
Runner-up: llama.cpp with llama-server, if you already run it
02
A model released this morning
A new architecture just dropped, the GGUF conversion is already on Hugging Face, and you want to try it today.
Reasoning: Support for new architectures lands in llama.cpp first — that is where the kernel work is done, and the release stream carries it within hours. Ollama picks it up on its own cadence. If the tag does not exist yet on Ollama, this is not a preference, it is the only path.
Picked
llama.cpp
Runner-up: Ollama, once the tag appears
03
An AMD card and a Vulkan driver
A workstation with a GPU outside the CUDA mainstream, where the vendor's compute stack is a source of pain.
Reasoning: llama.cpp lists 17 backends and Vulkan is one of them, which routinely turns an unsupported card into a working one. Ollama's documented GPU support is narrower, so the wrapper can be the thing standing between you and a GPU that would otherwise work.
Picked
llama.cpp
Runner-up: Ollama, if the card is on its supported list
04
Standardising a team of eight on one setup
Mixed hardware, an onboarding document, and a strong preference that nobody debugs build flags on their first day.
Reasoning: Ollama's install-and-tag model fits in two lines of a README, and its Modelfile lets you ship a system prompt and parameters as a named artefact. The team also gets a hosted fallback at $25 a seat when a model will not fit somebody's laptop.
Picked
Ollama
Runner-up: llama.cpp with a pinned build and a shared launch script
05
A 70B model and a 24 GB card
The weights are far larger than the VRAM, and buying a second GPU is not on the table this quarter.
Reasoning: llama.cpp documents CPU+GPU hybrid inference specifically to partially accelerate models larger than total VRAM, and its quantization tooling lets you make a smaller quant and measure what the smaller quant cost you in perplexity. Both of those levers are absent from the wrapper.
Picked
llama.cpp
Runner-up: Ollama with a smaller model someone else quantized
06
Serving a small internal tool to a handful of colleagues
One box, maybe five concurrent users, an internal chat tool that should not fall over at lunchtime.
Reasoning: llama-server's slots share a unified KV buffer and the /slots endpoint tells you what is actually happening; Ollama's parallelism multiplies memory by request count. At five users on one machine that difference is the whole question. Past a few dozen, neither is the right answer — vLLM is.
Picked
llama.cpp
Runner-up: Ollama, if the load is genuinely one user at a time
Frequently asked.
Common questions about this comparison, with sources where they matter.
Q · 01 Does Ollama use llama.cpp? +
Yes. Ollama's README has a "Supported backends" section, and the single entry in it is llama.cpp, credited to Georgi Gerganov. That is why comparing them on raw speed is mostly incoherent: on the same machine with the same GGUF file and the same settings, you are timing the same kernels. The differences that matter are what each project ships around those kernels, and how quickly upstream work reaches you.
Q · 02 Is llama.cpp faster than Ollama? +
It can be, and when it is, the reason is usually settings rather than architecture. llama.cpp exposes the full flag surface — offload layers, batch sizes, slot count, KV cache type, load mode, flash attention — while Ollama picks defaults for you. The other reason is time: kernel improvements land in llama.cpp first, so during the days between an upstream optimisation and the next Ollama release, upstream is genuinely quicker. Once Ollama picks it up, the gap closes again.
Q · 03 Is llama.cpp still hard to set up? +
Much less than its reputation suggests. The README's quick start is an installer at
llama.app, a Docker image or a prebuilt binary from the releases page — building from source is the fourth option, not the first. Getting a model is llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF, which downloads from Hugging Face directly, and llama serve starts an OpenAI-compatible API with a web UI attached. Most articles comparing these two still describe the 2024 experience.Q · 04 Can llama.cpp serve more than one model? +
Yes, since router mode. Start
llama-server without naming a model and it becomes a router that loads models on demand and forwards each request to the right instance, with --models-max capping how many stay resident (default 4) and --models-dir or a preset file telling it where to look. Ollama does the same thing with a different shape: up to three models per GPU, unloaded on a keep_alive timer.Q · 05 Which one should I use to serve several users? +
Between these two, llama.cpp — its slots share a unified KV buffer, it can save idle slots to a prompt cache, and
/slots reports per-slot metrics. Ollama's own documentation warns that memory scales by OLLAMA_NUM_PARALLEL multiplied by context length, which turns concurrency into a VRAM problem quickly. Above a few dozen concurrent users, neither is the tool — see vLLM vs Ollama.Q · 06 Do I have to choose? +
No, and many people run both: Ollama as the always-on endpoint that editors and agents talk to, llama.cpp when they want to try a model that has no tag yet, quantize something themselves, or measure throughput with
llama-bench. Both are MIT, both are free, and both read the same GGUF files — the only real cost of having both is disk space for two model directories.Q · 07 Which is better licensed for a commercial product? +
Both are MIT, so neither imposes conditions beyond attribution. The practical difference is shape: llama.cpp is a library with no dependencies, an Android build and an XCFramework for Apple platforms, so it is what you embed inside an application. Ollama is a service you install alongside an application. If you are shipping software to end users, that distinction usually decides it before the licence does.