LoRA vs QLoRA

Both methods freeze the base model and train the same low-rank adapter, with the same gradients and the same optimizer state. The only difference is how the frozen base is stored. LoRA keeps it at BF16, 2 bytes per parameter. QLoRA stores it as 4-bit NormalFloat with double quantization, about 0.52 bytes per parameter, and dequantizes each layer as it is used. Because the base weights are most of the memory, that one change is most of the saving. You pay for it with slower training steps. On quality, the QLoRA paper reports matching 16-bit fine-tuning on its benchmarks. If LoRA fits your card at the sequence length you need, use it. If it does not, QLoRA is the method that makes the job possible.

LoRA vs QLoRA at a glance

Training VRAM, LoRA vs QLoRA, for four model sizes

Rank 16 on all linear layers, 8-bit AdamW, 1,024-token sequences, batch size 1, gradient checkpointing on — the fine-tuning calculator's defaults, computed by the same function. Includes about 1GB of framework overhead. Activation memory is an approximation; plan with headroom.

Where the saving comes from

Training memory has four parts: the frozen base weights, the gradients, the optimizer state and the activations. LoRA already shrank the middle two. Only the adapter is trained, and it is usually well under 1% of the model, so its gradients and optimizer state are small.

That leaves the base weights as the biggest term by far, and QLoRA goes after exactly that. Storing them in 4-bit instead of BF16 cuts them to roughly a quarter. Everything else stays the same, so the saving you get is almost exactly the size of the base-weight reduction.

The table above is computed with the same function as the fine-tuning VRAM calculator. It uses the calculator's default settings: rank 16 on all linear layers, 8-bit AdamW, a 1,024-token sequence, batch size 1 and gradient checkpointing on. Change any of those in the calculator and both columns move together.

What QLoRA costs you

Speed. The base weights are stored in 4-bit, but the computation runs in higher precision. Each layer is dequantized as the forward and backward passes reach it, so every step does extra work that LoRA skips. If you train for a long time, that difference adds up.

Quality, at least on paper. Quantizing a model you are going to train would hurt it. QLoRA only quantizes the frozen part, and the adapter is computed in higher precision on top. The QLoRA paper (Dettmers et al., 2023) reports matching 16-bit fine-tuning on its benchmarks. Treat that as the published result, and check against your own evaluation set.

One step at deployment. You usually merge an adapter into a BF16 copy of the base, not into the 4-bit weights, and then quantize the merged model for inference. Merging into an already-quantized base is where precision gets lost.

Sequence length can matter more than the method

Both methods leave activation memory unchanged, and activations grow with sequence length and batch size. On long sequences without gradient checkpointing, activations can outgrow the base weights, and at that point switching between LoRA and QLoRA matters much less than the sequence length you choose.

So when a run does not fit, check these three settings before choosing a method: turn on gradient checkpointing, lower the sequence length, and use gradient accumulation instead of a bigger batch. If it still does not fit after that, QLoRA is the next step.

Work it out for your own setup

Common questions

Is QLoRA worse than LoRA?

The QLoRA paper reports matching 16-bit fine-tuning on its benchmarks. That makes sense, because only the frozen base is quantized and the adapter is computed in higher precision. The clear cost is speed. Any quality difference is small enough that you should measure it on your own task rather than assume it.

How much VRAM does QLoRA save?

Most of the base-weight memory. BF16 stores 2 bytes per parameter and QLoRA's NF4 with double quantization stores about 0.52, so the frozen weights shrink to roughly a quarter. The table on this page shows the resulting totals per model. The fine-tuning VRAM calculator gives the figure for your own settings.

Is the adapter from QLoRA different from a LoRA adapter?

No. It is the same kind of low-rank adapter, and you load it the same way. The difference was only in how the base model was stored while you trained.

Should I use QLoRA if LoRA already fits?

Usually not. If the BF16 base fits with room left for your sequence length, LoRA trains faster and removes one variable. QLoRA exists for the cases where LoRA does not fit.