How much VRAM do you need to fine-tune an LLM?

Where the memory goes

Base weights are fixed by the model and the precision you hold it at: 2 bytes per parameter at BF16, or about 0.52 with QLoRA's 4-bit NormalFloat and double quantization. This term does not care whether you are training all of the model or none of it.

Gradients and optimizer state scale with the parameters you are actually training. With 32-bit AdamW that is 2 bytes of gradient plus 12 bytes of optimizer state — an FP32 master copy and two FP32 moment estimates — for a total of 16 bytes per parameter when you include the BF16 weight itself. That figure is why full fine-tuning of an 8B model needs roughly 130GB, and why freezing the base changes everything.

Activations are the one approximation here. They scale with sequence length, batch size and model width, and the real figure depends on the attention implementation and the trainer. We assume flash attention or SDPA, which removes the quadratic term that dominated older implementations, and follow the standard per-layer accounting of roughly 34 bytes per token per hidden unit. Treat it as a planning figure with headroom, not a guarantee.

The LoRA trap: adapters cut the gradient and optimizer terms to almost nothing, and do nothing at all about activations. The backward pass still runs through the whole frozen base to reach the adapters. That is why gradient checkpointing matters as much for LoRA as for full fine-tuning.

Where these numbers come from

Layer counts, hidden size, attention heads, key/value heads and intermediate size are read from each model's config.json on Hugging Face. Nothing here is estimated.

Predicting each dense model's parameter count from its config (attention + SwiGLU MLP + embeddings) reproduced the published count within 1% for 15 of 20 dense models and within 8% for all but Qwen3.8 27B, whose hybrid attention the uniform formula does not describe. That check confirms the dimensions are real; the fine-tuning maths below uses them only for activation memory.

Seven repositories (Meta Llama, Gemma 2, Llama 4) require authentication to read. Their configs were taken from well-known ungated mirrors of the same weights, marked in the source field. Command-R 35B is gated everywhere and is therefore not listed.

Adapter parameter counts are exact, not estimated: a LoRA on a weight matrix of shape in x out adds r x (in + out) parameters, summed over the targeted modules in every layer. Two independent checks: this model reproduces the published QLoRA result that a 65B-class model fine-tunes on a single 48GB card, and it returns exactly 16 bytes per parameter for full fine-tuning with 32-bit AdamW, which is the textbook figure.

Sizes are decimal GB, the same convention as the rest of our calculators. Dimensions checked 2026-09-12.

Common questions

How much VRAM do I need to fine-tune a 7B or 8B model?

With QLoRA at rank 16 on all linear layers, a 1,024-token sequence and gradient checkpointing, an 8B model comes to roughly 6GB — comfortable on a 12GB card and workable on 8GB. Plain LoRA on a BF16 base roughly quadruples the weight term, to about 18GB. Full fine-tuning is a different order of magnitude: about 82GB with the same 8-bit optimizer, or roughly 130GB with 32-bit AdamW, where the weight, gradient and optimizer terms together come to 16 bytes for every parameter in the model.

Why does fine-tuning need so much more memory than inference?

Inference holds the weights and a KV cache. Training holds the weights, a gradient for every trainable parameter, the optimizer's state for every trainable parameter, and the activations needed for the backward pass. With 32-bit AdamW the optimizer alone is 12 bytes per trainable parameter — six times the weights themselves at BF16.

Does LoRA reduce activation memory?

No, and this surprises people. Activations still flow through the entire frozen base model because the backward pass has to reach the adapters. LoRA removes the gradient and optimizer terms, not the activation term. Gradient checkpointing is what reduces activations, and it is worth leaving on.

What actually causes out-of-memory errors during QLoRA?

Usually sequence length or batch size rather than the model. Both scale activation memory linearly, and the base weights are already fixed by the time you start. Reduce sequence length first, then batch size — gradient accumulation lets you keep the effective batch size while lowering the real one.

Is 8-bit AdamW worth using?

For QLoRA, almost always. It quantizes both moment estimates to 8 bits, taking the optimizer from 12 bytes per trainable parameter to 6. On a large adapter that is several gigabytes, and the quality cost is generally reported as negligible.

Do these numbers include the operating system and display?

They include 1GB for the CUDA context and framework allocator, which is the same reservation our other calculators use. They do not include whatever your desktop is already using. On a machine driving a monitor from the same GPU, leave more headroom.

Before you fine-tune anything

Check that fine-tuning is the right tool. It teaches behaviour — format, tone, domain conventions, one narrow task — and it is unreliable at teaching facts. If the problem is that the model does not know something, retrieval is the answer and it is far cheaper. We compare the two in fine-tuning vs RAG, and the memory trade-off between methods in LoRA vs full fine-tuning. Also confirm the base model's licence permits it — the licence checker covers what each one allows for derivatives.

How memory requirements are built up, and which hardware gives you headroom.

• How Much RAM and VRAM Do You Need to Run Local LLMs?Model weights, KV cache and OS overhead, tier by tier.Read the guide
• Quantization Guide: Q4_K_M vs Q8_0 vs FP8What each quant level costs you in memory and quality.Read the guide
• Best Budget GPUs for Local LLMs16GB cards, used RTX 3090s and current street prices.Read the guide