llamafile in 2026: Run a Local LLM From a Single File

llamafile turns a local LLM into a single file you download and run. There’s no installer, no Python environment and no separate model folder. The same file runs on Linux, macOS, Windows and the BSDs. It combines the llama.cpp inference engine with Cosmopolitan Libc, the portable C library that makes one executable work across operating systems. The project is now maintained by Mozilla.ai. Version 0.10, released in March 2026, rebuilt it on a new build system, and the latest release at the time of writing is 0.10.5 (August 2026). This guide covers how to run a llamafile, how to use your GPU and your own models, and where it fits next to Ollama and llama.cpp.

Quick answer

• What it is: one executable file containing both the inference engine and, for pre-built llamafiles, the model weights.
• How to run it: download, make it executable, run it. A chat opens in your terminal, and a web UI is served at http://localhost:8080/.
• GPU: Metal is on by default on Apple Silicon. On NVIDIA and AMD, pass -ngl 999.
• Windows catch: Windows won’t run an executable larger than 4GB. For bigger models, run the llamafile binary with separate GGUF weights instead.
• Licence: Apache 2.0. The project’s own changes to llama.cpp and whisper.cpp are MIT. The bundled whisper.cpp covers speech-to-text; for the other direction see the best local text-to-speech models, where the licences differ sharply between projects.

What llamafile actually is

Most local AI tools are an application plus a model you download separately. llamafile packs them together. A pre-built llamafile is a single file with the llama.cpp engine inside and the model weights appended. You run it the way you’d run any program. The same idea covers speech: the project also ships whisperfile, a single-file speech-to-text tool built on whisper.cpp.

Because the engine is llama.cpp, llamafile runs the same GGUF models as the rest of the llama.cpp ecosystem. What it adds is packaging and portability, not a different way of running models. Speed and memory needs are those of llama.cpp. Use our VRAM calculator to check what fits before you download anything large.

Run your first llamafile

The project’s quickstart uses a small Qwen3.5 model from Mozilla.ai’s Hugging Face repository. On macOS, Linux or BSD:

curl -LO https://huggingface.co/mozilla-ai/llamafile_0.10/resolve/main/Qwen3.5-0.8B-Q8_0.llamafile
chmod +x Qwen3.5-0.8B-Q8_0.llamafile
./Qwen3.5-0.8B-Q8_0.llamafile

Three things happen when it starts:

• A chat opens in the terminal. Type /help for commands, or /upload to add an image.
• A web UI is available at http://localhost:8080/. It’s llama.cpp’s own web interface.
• An API is listening at http://127.0.0.1:8080/v1/chat/completions. The docs describe it as compatible with the OpenAI API and with Anthropic’s Messages API.

Press Ctrl-C to stop it. After the download, nothing needs an internet connection.

On Windows

Rename the file so it ends in .exe, then run it. The limit to know about is that Windows won’t run an executable larger than 4GB, so any llamafile above that size fails. For larger models, the project’s docs point to the fix: download the plain llamafile binary from the GitHub releases page and pass a separate GGUF file with -m.

.\llamafile.exe -m your-model.gguf

Use your GPU

The supported-systems page lists these GPU backends:

-ngl sets how many model layers go on the GPU; 999 simply means all of them. If a model doesn’t fit in your VRAM, lower the number and the rest runs on the CPU, much more slowly. The docs also note that the CUDA build bundled with llamafile doesn’t include kernels for the IQ family of quantizations. Those parts run on the CPU automatically, so choose a Q4_K_M or Q8_0 file if you want everything on an NVIDIA GPU.

Bring your own model

You aren’t limited to pre-built llamafiles. Any GGUF file works with -m, which is also the way around the Windows size limit:

./llamafile -m Qwen3-8B-Q4_K_M.gguf -ngl 999

Choose the size from the memory you have. At Q4_K_M, an 8B model’s weights take roughly 5GB and a 32B model’s roughly 20GB, with the context’s KV cache on top. Our models by VRAM tier page lists what fits at each common capacity. The RAM and VRAM guide explains the maths.

Supported systems

On the CPU side, x86-64 processors must support AVX, and ARM64 processors must be ARMv8a or newer, which includes Apple Silicon and 64-bit Raspberry Pi boards.

llamafile vs Ollama vs llama.cpp

All three run the same kind of model, and llamafile and Ollama both build on llama.cpp. The real difference is how you want to work:

• llamafile is for portability. One file, no install, easy to share or to keep on a USB drive or an offline machine.
• Ollama is for convenience. It has a model library, ollama pull, a background service and a large ecosystem of tools that expect it.
• llama.cpp is for control. It’s the engine itself, with every flag exposed.

llama.cpp vs Ollama covers that trade-off in depth, and our runner comparison adds LM Studio, Jan and vLLM. If you want the model available to other machines on your network, the security advice in our Ollama server guide applies equally to llamafile’s port 8080.

FAQ

Is llamafile still maintained?

Yes. It’s maintained by Mozilla.ai, and the GitHub repository had releases through 2026: 0.10.0 in March, 0.10.5 on 3 August and 0.10.6 on 15 September. Version 0.10 moved to a new build system. The project labels which server version pre-built llamafiles were bundled with, so older 0.9 files are still identifiable.

Does llamafile need an internet connection?

Only to download it. The model runs entirely on your machine afterwards.

Why won’t my llamafile run on Windows?

Most likely because the file is over 4GB, which Windows won’t execute. Run the separate llamafile binary with your model passed via -m. Also check that the file ends in .exe.

Is llamafile faster than Ollama?

Both run models through llama.cpp, so on the same hardware, model and settings you shouldn’t expect a large difference. Speed is set mainly by your GPU’s memory bandwidth and the model’s size. The hardware matrix gives estimates per device.

Sources

• Mozilla.ai: llamafile GitHub repository and releases
• llamafile docs: Quickstart, Supported systems
• Hugging Face: mozilla-ai/llamafile_0.10

Updated September 15, 2026. Release list re-checked 10 October 2026.