A model can load on your GPU and still run out of memory when you give it a long document. llama.cpp 1.0, released on November 13, 2026, adds a way to reduce part of that memory cost. It also changes how the project signals breaking updates to developers who embed the engine in their own applications.
llama.cpp is an open-source inference engine for running AI models on your own hardware. It can load compatible models packaged in GGUF files and use different CPU and GPU backends. The impact of this release depends on how you use it: through a desktop application, as a local server, or as a C library embedded in your software.
An FP8 cache can make room for longer prompts
During a conversation, the engine stores intermediate attention data from earlier tokens in a key-value cache, often called the KV cache. Reusing that data avoids recalculating it for every new token. The cache can grow as you feed the model more text, so the same model can have very different memory needs for a short question and a long coding session.
Version 1.0 adds an FP8 option for that cache. FP8 uses eight bits per value, compared with sixteen for an FP16 cache. That can reduce the cache footprint in a compatible setup. The model weights, temporary buffers and other allocations still account for part of the total. A model whose weights already exceed the available memory can therefore remain too large to run entirely on the GPU.
This matters when a setup is close to its limit. Suppose a small coding model works well with a single source file, but a prompt with several files and a long error log pushes it over the edge. A smaller cache may provide enough headroom to keep the longer prompt in memory. Check speed and answer quality on your actual workload, since the result depends on the model, hardware and backend. Our guide to RAM and VRAM for local AI covers the other parts of the memory budget.
The 1.0 version number matters most to integrators
The project is applying semantic versioning to the public C interfaces of libllama and libggml. The maintainers say future breaking changes to the public interface or binary compatibility will be signaled by a major version bump. Compatible additions can arrive in minor versions, while bug fixes without API changes can arrive in patches.
The jump to 1.0 itself removes deprecated functions and moves the NextN speculative decoding interface into the main llama.h header. If you maintain a native integration, rebuild it with the new headers and test the binaries together. If you use a desktop app that bundles llama.cpp, its own release notes tell you when the new engine reaches your installation.
For teams that talk to llama-server over HTTP, the tests are different. Start the new server with the arguments you actually deploy and rerun the requests your client depends on. The release also adds an OpenAI-compatible /v1/audio/speech endpoint for text-to-speech. Teams interested in that feature should test it with a compatible speech model and their existing client.
How to decide whether to upgrade
Keep the same GGUF file, hardware and GPU offload settings. Use one short prompt and one long prompt that represents your real work. First compare the old and new engines with the same cache settings. Then compare your usual cache format with FP8 on 1.0. Changing one setting at a time will show whether the version upgrade or the cache accounts for any difference.
Record peak RAM and VRAM usage, time to first token and generation speed. For a coding workload, check whether the model can still use a relevant detail from early in the long prompt and whether the resulting patch passes its tests. If FP8 lets the full prompt fit with acceptable speed and quality, it has solved a practical problem for your setup.
