Ollama vs llama.cpp: the wrapper or the engine?

Ollama hides the machinery: one command pulls a model and the server is already running. llama.cpp IS the machinery: you manage the model file and the server flags yourself, and in exchange you get every knob the engine has, the day it ships.

The part most comparisons bury: under the hood, this is the same engine. Ollama builds on llama.cpp's inference technology, so the same model on the same GPU generates at roughly the same speed. Both are free. Both serve an OpenAI-compatible API on localhost. Both connect to mi·do·na the same way: pick the runner in the connection panel, point it at the /v1 endpoint, done.

Everything below is about the EXPERIENCE around the engine. You cannot pick wrong.

Ollama is the two-word version. ollama run pulls a model from its library, quantized and configured, server already listening. You never think about GGUF files or quantization formats unless you choose to.

llama.cpp makes the file your problem... which is the point. You download a GGUF (or pass -hf and let it fetch one from Hugging Face) and start llama-server with the flags you want. For the roleplay crowd this matters more than it sounds: the community finetunes people actually play with ship as GGUF files, and llama.cpp takes any of them without ceremony. Ollama can too, via a Modelfile, but that's an extra step past the tidy path.

Ollama exposes the essentials: temperature, top-p, top-k, repeat penalty. For a lot of stories, genuinely fine.

llama.cpp is where samplers are BORN. It's the reference implementation. Min-p, dynamic temperature, the newest experimental samplers... they land here first, and every flag is yours from day one. If you're the kind of player who tunes prose the way some people tune synthesizers, this is the runner that never tells you no.

If you'd rather not learn any of that, mi·do·na's tuning panel speaks one dial, Imagination, to either runner. The ceiling is just higher on llama.cpp.

Ollama's gotcha is CORS. A browser app like mi·do·na can't reach it until you set OLLAMA_ORIGINS. One environment variable, and our connection panel walks you through it... but it IS the step people miss.

llama.cpp's gotcha is everything else. Install varies by platform, flags are on you, and nobody holds your hand. In return, permissive CORS is on by default, so once the server is up, mi·do·na reaches it with no extra configuration at all.

Pick Ollama if you want the engine without the engine room. Pull a model, set one variable, play. It's the tidy default for a reason.

Pick llama.cpp if you want to be closest to the metal: every sampler, every flag, every update first. It asks more of you on day one and repays it every day after.

Either way, mi·do·na does the story work on top. Ports and defaults reflect each project's documentation as of July 2026. Check the official docs if a release has moved since.