AINews wire, Yesterday

Switching a home server from Ollama to llama.cpp can speed up local LLMs

Reported by XDA Developers

Image: XDA Developers

A home server owner reports that moving a local language model from Ollama to llama.cpp made it run faster, with the same hardware, according to an XDA Developers article published on 20 November 2026.

The author says Ollama already uses llama.cpp in the background to run GGUF models. Running llama.cpp directly gave better results in their tests. The author did not publish a full benchmark, so the size of the gain is not known.

Ollama handles model loading, unloading, request scheduling and resource allocation. Users can adjust context length, parallel requests and GPU offloading, but Ollama still makes several choices about how a model runs. That suits people who want little setup. It may not get the most out of their hardware.

On a home server, memory and processing power are often limited, especially when other services run alongside the model. Larger context windows use more memory. Handling several requests at once adds to that load. The number of CPU threads also affects generation speed, and more threads do not always help.

llama.cpp offers more direct control. Its llama-server lets users load a GGUF model and set CPU threads, batch sizes and context length with launch arguments. It also provides an OpenAI-compatible API, so existing applications can keep working with the model. The author notes that llama.cpp can allocate CPU resources separately for reading a prompt and for generating a response. This can be useful because the two steps have different needs.

The trade-off is convenience. Ollama lets users download a model with one command and switch between models easily. With llama.cpp, users manage model files and server settings themselves. Docker can be used to run llama.cpp, which may suit a home server.

The article also lists other options. LM Studio offers a graphical interface for downloading, managing and testing models, and supports GGUF and Apple's MLX models. Its headless service, llmster, can run without the desktop app. BaseRT is an inference engine built for Apple Silicon. Its developers claim faster text generation than llama.cpp on an M4 Pro, but these figures come from the developers, not from independent tests. Docker Model Runner uses llama.cpp by default and can also use vLLM on compatible NVIDIA GPUs.

The author's conclusion is that Ollama remains the most popular option, but users who want more from their own hardware should compare alternatives. The article does not give full test data, so readers should run their own checks on their hardware before changing setup.

Enjoying this? Get the AI brief

Every Thursday: AI that matters for your home.

✓ Free · seven editions · one click to leave

Topics
Story continues on XDA DevelopersAdd us as a preferred source on GoogleSee our stories first in Google’s Top Stories.
Get alerts on big newsOnly the big news, at most once a day. No email, no account.

Original reporting by the sources above. Summary by smart-home.space.