Not formally — it is still supported in vLLM and the tooling is maintained. But new quantized releases increasingly ship as AWQ, FP8 or GGUF, and for desktop use GPTQ has effectively been replaced.

On a GPU with the whole model resident, the GPU-native formats have the advantage, because llama.cpp's CUDA kernels are not the primary optimisation target that vLLM's are. Once any part of the model spills to system RAM the comparison inverts, because GPTQ simply cannot run in that configuration.

Can I convert GPTQ to GGUF?

Not directly in any reliable way. Both are quantized, and going from one quantization to another compounds the error. Start from the original safetensors weights and quantize once.

What replaced GPTQ for serving?

Mostly AWQ, which uses activation statistics to decide where quantization error is tolerable, and FP8 on hardware that supports it natively. Both target the same deployment shape as GPTQ.

Model formats

GGUF vs GPTQ

GPTQ was the first widely used 4-bit format for local models and is now mostly a serving format. GGUF replaced it on the desktop. Here is what separates them and when GPTQ still makes sense.

The short answer

GPTQ arrived first and for a while it was how you ran a quantized model on a consumer GPU. GGUF took over the desktop because it runs on CPU as well as GPU, offers many more quantization levels, and ships as one self-contained file. GPTQ survives as a GPU serving format alongside AWQ. Unless you are specifically running vLLM or an ExLlama-based stack, GGUF is the current answer.

GGUF

The current desktop default. Any hardware, many bit depths, one file.

GPTQ

An older GPU-only 4-bit format, still used for serving.

GGUF vs GPTQ at a glance

Factor	GGUF	GPTQ
Released	2023, successor to GGML	2022, from the GPTQ paper
Quantization approach	Mixed-precision k-quants, no calibration data needed	One-shot layer-wise quantization using approximate second-order information
Bit depths	Roughly 3.15 to 8.51 bits per weight, eleven levels	Typically 4-bit, sometimes 3-bit or 8-bit
Hardware	CPU, CUDA, Metal, ROCm, Vulkan	CUDA, with partial ROCm support
CPU offload	Yes, layer by layer	No
Current momentum	The default for desktop tooling	Maintained for serving; superseded by AWQ and FP8 for new work

GPTQ's method, and why it was a big deal

GPTQ quantizes a model one layer at a time, and while it does so it uses an approximation of the layer's Hessian — second-order curvature information — to decide how to round each weight so that the layer's output changes as little as possible. Earlier approaches rounded to the nearest representable value and accepted whatever error that caused.

The result was 4-bit models that stayed usable, which is what first made running a 30B-class model on a single 24GB consumer card realistic. That mattered enormously in 2023, and a lot of the local-model ecosystem was built on top of it.

The cost is that GPTQ requires a calibration dataset and a quantization run per model, and the output is CUDA-oriented. Neither is a problem in a server context. Both are a problem if your target is a MacBook.

Why the desktop moved to GGUF

Three things decided it. GGUF runs on the CPU, so a model that exceeds your VRAM still runs — slower, but it runs. GGUF is one file with the tokenizer and chat template inside, so nothing has to be assembled correctly. And llama.cpp's Metal and Vulkan backends meant the same file worked on a Mac, an AMD card and an Intel GPU.

The k-quant ladder also gives you somewhere to go. A 32B model at Q4_K_M needs about 19.6GB of weights; at Q3_K_M it needs about 16GB, and at Q2_K about 12.6GB. Each rung is a real decision you can make against the card you own. GPTQ's 4-bit is a single point on that line.

The knock-on effect is availability. Because converting to GGUF needs no calibration data, quantized builds of new models appear almost immediately, and usually at every level. GPTQ builds are rarer now, and for models released this year they frequently do not exist at all.

Where GPTQ still has a role

vLLM supports GPTQ, and if you already have a GPTQ build of a model that suits your workload there is no reason to redo it. On older serving deployments GPTQ is often what is already deployed and working.

For new server-side work, AWQ or an FP8 build is generally the more current choice, and on newer NVIDIA hardware FP8 has the advantage of native tensor-core support rather than needing dequantization kernels.

None of this changes the desktop answer. If the question is what to download to run a model on your own machine, it is GGUF, and the only real decision left is which rung of the ladder your memory budget allows.

Choose GGUF when

You run models locally on any hardware, including Macs and CPUs
You need a quantization level between the standard 4-bit and 8-bit points
The model is recent and you want a quantized build that actually exists
You may need to offload part of the model to system RAM

Choose GPTQ when

You already run a vLLM deployment configured for GPTQ
You have an existing GPTQ build that is working and validated
You use an ExLlama-based stack on NVIDIA hardware

Work it out for your own setup

GGUF size calculatorCompare every k-quant level against your memory budget.Model directoryWhich models exist at which sizes, with licences.Models by VRAM tierStart from the memory you have and see what fits.

Common questions

Is GPTQ deprecated?: Not formally — it is still supported in vLLM and the tooling is maintained. But new quantized releases increasingly ship as AWQ, FP8 or GGUF, and for desktop use GPTQ has effectively been replaced.
Which is faster?: On a GPU with the whole model resident, the GPU-native formats have the advantage, because llama.cpp's CUDA kernels are not the primary optimisation target that vLLM's are. Once any part of the model spills to system RAM the comparison inverts, because GPTQ simply cannot run in that configuration.
Can I convert GPTQ to GGUF?: Not directly in any reliable way. Both are quantized, and going from one quantization to another compounds the error. Start from the original safetensors weights and quantize once.
What replaced GPTQ for serving?: Mostly AWQ, which uses activation statistics to decide where quantization error is tolerable, and FP8 on hardware that supports it natively. Both target the same deployment shape as GPTQ.

Guides that go deeper

Longer reads on the concepts behind this comparison.

Related comparisons

GGUF vs AWQModel formats GGUF vs SafetensorsModel formats Q4_K_M vs Q8_0Quantization