Is GGUF worse quality than safetensors?

Not because of the format. A GGUF is usually quantized and the safetensors original usually is not, and that is where any quality difference comes from. A Q8_0 GGUF is near-lossless against the FP16 original; a Q2_K one is visibly degraded. The container is not the variable.

Can I use a GGUF with vLLM?

vLLM has added partial GGUF support, but it is not the intended path and the performance characteristics vLLM is built for — continuous batching against fully resident GPU weights — assume a GPU-native format such as safetensors, AWQ or FP8. For serving, use those.

Why do GGUF repositories have so many files?

Each file is the same model at a different quantization level, so you download one and ignore the rest. Very large models are also split into shards of a few gigabytes each; in that case you need every part of the split you chose, and the runner reassembles them.

Does GGUF work on Apple Silicon?

Yes. llama.cpp has a Metal backend, so GGUF runs on the GPU of any Apple Silicon Mac. MLX is the Apple-native alternative with its own format, and that is a separate trade-off between portability and platform fit.

Model formats

GGUF vs Safetensors

Safetensors is a tensor container. GGUF is a container plus a quantization ladder plus a runtime contract. Which one you need is decided by the tool you run, not by preference.

The short answer

These are not competing formats so much as formats aimed at different runtimes. Safetensors is a safe, memory-mappable container for raw tensors, and it is what Hugging Face Transformers, diffusers and vLLM load. GGUF is a single self-contained file that also carries the tokenizer, the chat template and the quantization scheme, and it is what llama.cpp, Ollama and LM Studio load. If you run models on a desktop or laptop you want GGUF; if you serve them on a GPU with vLLM or fine-tune them, you want safetensors.

GGUF

One file, quantized, runs anywhere including CPU. The desktop and laptop format.

Safetensors

Raw tensors in full or half precision. The training, serving and Hugging Face format.

GGUF vs Safetensors at a glance

Factor	GGUF	Safetensors
What it holds	Weights, tokenizer, chat template and metadata in one file	Tensors only — config, tokenizer and code live in separate files
Typical precision	Quantized, roughly 3.15 to 8.51 bits per weight	BF16 or FP16 as published; FP8 and 4-bit variants exist
Runs on CPU	Yes, and it can split layers between CPU and GPU	Possible but slow and rarely the intent
Loaded by	llama.cpp, Ollama, LM Studio, Jan, KoboldCpp	Transformers, diffusers, vLLM, TGI, SGLang, MLX conversions
Can you fine-tune it	No — convert back or train the original	Yes, this is the training format
Safety	No code execution; plain binary	Designed specifically to replace pickle, no code execution

Safetensors solved a security problem, not a size problem

Before safetensors, model weights shipped as PyTorch pickle files. Pickle can execute arbitrary Python on load, which meant downloading a model from a stranger was equivalent to running their script. Safetensors replaced that with a dumb, strictly-parsed binary layout: a JSON header describing tensor names, dtypes and byte offsets, then the raw bytes. Nothing in it can execute.

The second benefit is that the layout is memory-mappable. A loader can map the file and pull individual tensors without reading the whole thing into RAM first, which is why large models load faster from safetensors than they did from a monolithic pickle.

What safetensors deliberately does not do is describe the model. It is a container for named tensors and nothing else. The architecture lives in config.json, the tokenizer in its own files, and anything unusual in custom modelling code in the repository. That is fine inside the Hugging Face ecosystem, where those files travel together, and awkward everywhere else.

GGUF is a runtime contract in a single file

GGUF came out of llama.cpp, and it was designed around a different constraint: run this model on a machine that might have no GPU at all, with no Python, no Hugging Face cache and no separate tokenizer download. So a GGUF file carries everything — the quantized tensors, the vocabulary, the chat template, the rope parameters and arbitrary key-value metadata.

That self-containment is why the desktop tools standardised on it. When you type a model name into Ollama or click download in LM Studio, one file arrives and it works. There is no dependency resolution and no config mismatch, because the config is inside the file.

The other half of GGUF is the quantization scheme. The k-quant formats are mixed precision: the quantizer keeps attention and embedding tensors at higher bit depth and pushes the bulk of the feed-forward weights lower. That is why a file labelled Q2_K measures nearer 3.15 bits per weight in practice, and why Q4_K_M at roughly 4.9 bits per weight has become the default recommendation rather than a plain 4-bit format.

Conversion goes one way in practice

Converting safetensors to GGUF is routine — llama.cpp ships scripts that read a Hugging Face repository and write a GGUF, after which you quantize to whichever k-quant you want. This is how nearly every GGUF on Hugging Face was made.

Going the other direction is possible but pointless. Dequantizing a Q4 GGUF back to FP16 gives you a file the size of the original with the quality of the quantized one. If you need safetensors, download the original repository.

The practical consequence is for fine-tuning. You cannot train a GGUF. If you want to adapt a model you already run locally, fetch the base model in safetensors, train there, then convert the result to GGUF for daily use.

Choose GGUF when

You run models on a laptop, desktop or Mac through Ollama, LM Studio or llama.cpp
You need to split a model between GPU and system RAM because it does not fit in VRAM
You want one file that works without a Python environment
You are choosing a quantization level to fit a specific memory budget

Choose Safetensors when

You serve a model to multiple concurrent users with vLLM, TGI or SGLang
You are fine-tuning, training adapters or merging models
You use the model through Transformers or diffusers in your own code
You need the unquantized reference weights for evaluation

Work it out for your own setup

GGUF size calculatorWork out the file size of any parameter count at every quantization level.VRAM calculatorCheck whether the quantization you picked actually fits your card.Model directoryParameter counts, licences and memory requirements for local models.

Common questions

Is GGUF worse quality than safetensors?: Not because of the format. A GGUF is usually quantized and the safetensors original usually is not, and that is where any quality difference comes from. A Q8_0 GGUF is near-lossless against the FP16 original; a Q2_K one is visibly degraded. The container is not the variable.
Can I use a GGUF with vLLM?: vLLM has added partial GGUF support, but it is not the intended path and the performance characteristics vLLM is built for — continuous batching against fully resident GPU weights — assume a GPU-native format such as safetensors, AWQ or FP8. For serving, use those.
Why do GGUF repositories have so many files?: Each file is the same model at a different quantization level, so you download one and ignore the rest. Very large models are also split into shards of a few gigabytes each; in that case you need every part of the split you chose, and the runner reassembles them.
Does GGUF work on Apple Silicon?: Yes. llama.cpp has a Metal backend, so GGUF runs on the GPU of any Apple Silicon Mac. MLX is the Apple-native alternative with its own format, and that is a separate trade-off between portability and platform fit.

Guides that go deeper

Longer reads on the concepts behind this comparison.

Related comparisons

GGUF vs AWQModel formats GGUF vs MLXModel formats Q4_K_M vs Q8_0Quantization