How do I know how much VRAM I have?

On Windows, open Task Manager, go to Performance and select your GPU — dedicated GPU memory is your VRAM. On a Mac there is no separate VRAM: the GPU shares unified memory with the CPU and can typically address around 70% of total RAM, so a 32GB Mac behaves roughly like a 22GB card.

Why can I not use all of my VRAM for the model?

The display output, the inference framework and the CUDA or Metal context all reserve memory before your model loads. About 1GB is a realistic reservation. Planning to your full VRAM figure is the most common reason a model that should fit fails to load.

Does more VRAM make models run faster?

No. VRAM capacity decides which models fit; memory bandwidth decides how fast they run. Two cards with the same 24GB can differ by an order of magnitude in tokens per second. Each tier page lists the bandwidth range of real machines at that capacity.

What if my model does not fit in VRAM?

You can offload some layers to system RAM, which works but is much slower because those layers run at system memory bandwidth rather than GPU bandwidth. Dropping to a smaller quantization, or choosing a mixture-of-experts model where only a fraction of the weights are active per token, is usually the better trade.

What LLM can I run?

Start with the memory you have. Each tier below scores every model in our directory against the memory a card that size realistically gives you, and shows the best quantization each model can use — not just whether it loads.

8GB6 of 30 models

Largest that fits: Meta Llama 3.2 3B Instruct at Q8_0. 1 run comfortably.

~7GB usable · 19 machines profiled at this size

12GB11 of 30 models

Largest that fits: Gemma 4 12B at Q4_K_M. 7 run comfortably.

~11GB usable · 4 machines profiled at this size

16GB12 of 30 models

Largest that fits: DeepSeek R1 Distill Qwen 14B at Q4_K_M. 11 run comfortably.

~15GB usable · 26 machines profiled at this size

24GB22 of 30 models

Largest that fits: Gemma 4 31B at Q4_K_M. 19 run comfortably.

~23GB usable · 4 machines profiled at this size

32GB23 of 30 models

Largest that fits: Qwen3.6 35B-A3B (MoE) at Q4_K_M. 23 run comfortably.

~31GB usable · 5 machines profiled at this size

48GB26 of 30 models

Largest that fits: Qwen3.6 35B-A3B (MoE) at Q8_0. 23 run comfortably.

~47GB usable · 4 machines profiled at this size

64GB26 of 30 models

Largest that fits: Qwen 2.5 72B Instruct at Q4_K_M. 26 run comfortably.

~63GB usable · 5 machines profiled at this size

128GB28 of 30 models

Largest that fits: OpenAI gpt-oss-120b at Q4_K_M. 28 run comfortably.

~127GB usable · 4 machines profiled at this size

How much VRAM do I need?

Enough for three things: the model weights, the KV cache for your context, and roughly 1GB the display and framework reserve before anything loads. The weights are the part you can size up front — parameters × bits per weight ÷ 8. At Q4_K_M, which is 4.9 bits per weight measured from real GGUF files, that gives:

Model size	Weights at Q4_K_M	Smallest tier that holds them
8B	4.9 GB	8GB
14B	8.6 GB	12GB
32B	19.6 GB	24GB
70B	42.9 GB	48GB

The weights fitting is the floor, not the plan. The KV cache grows with context length, so a model that only just fits will load and then fail on a long conversation — if you want long context, go one tier up. For an exact figure for one model at your context length, use the VRAM calculator.

Capacity is not speed

These pages answer one question: what fits. That is decided by VRAM capacity. How fast a model runs is decided by memory bandwidth, which is a property of the specific card or chip, not of its capacity — an RTX 3090 and a 24GB unified-memory Mac hold the same models and generate at very different speeds.

That is why you will not find tokens-per-second figures on a tier page. Each one lists the bandwidth range of real machines at that capacity and links through to the hardware matrix, where every device carries its own estimate.

Common questions

How do I know how much VRAM I have?: On Windows, open Task Manager, go to Performance and select your GPU — dedicated GPU memory is your VRAM. On a Mac there is no separate VRAM: the GPU shares unified memory with the CPU and can typically address around 70% of total RAM, so a 32GB Mac behaves roughly like a 22GB card.
Why can I not use all of my VRAM for the model?: The display output, the inference framework and the CUDA or Metal context all reserve memory before your model loads. About 1GB is a realistic reservation. Planning to your full VRAM figure is the most common reason a model that should fit fails to load.
Does more VRAM make models run faster?: No. VRAM capacity decides which models fit; memory bandwidth decides how fast they run. Two cards with the same 24GB can differ by an order of magnitude in tokens per second. Each tier page lists the bandwidth range of real machines at that capacity.
What if my model does not fit in VRAM?: You can offload some layers to system RAM, which works but is much slower because those layers run at system memory bandwidth rather than GPU bandwidth. Dropping to a smaller quantization, or choosing a mixture-of-experts model where only a fraction of the weights are active per token, is usually the better trade.

Guides behind these numbers

How memory requirements are calculated, and which hardware gives you headroom.