What is the best LLM for 12GB of VRAM?

Gemma 4 12B (12B) is the largest model with comfortable headroom, at Q4_K_M, needing about 10.5GB against the ~11GB usable on a 12GB card. DeepSeek R1 Distill Qwen 14B (14.8B) is larger and will still load, but only as a tight fit with a short context. 11 of the 30 models in our directory fit at 12GB in some form.

Which small LLMs run well on 12GB?

The best small LLMs for 12GB are the ones that fit with room left for context: Gemma 4 12B (12B, Q4_K_M), Google Gemma 2 9B (9.24B, Q4_K_M), Qwen3 8B (8.2B, Q4_K_M), Llama 3.1 8B Instruct (8.03B, Q4_K_M), DeepSeek R1 Distill Llama 8B (8.0B, Q4_K_M). DeepSeek R1 Distill Qwen 14B (14.8B, Q4_K_M), Qwen 2.5 Coder 14B Instruct (14.7B, Q4_K_M), Microsoft Phi-4 14B (14.7B, Q4_K_M), Mistral NeMo 12B Instruct (12.2B, Q4_K_M) will also load, but as tight fits that need a short context. A small model with headroom answers faster and holds a longer conversation than a larger one squeezed in.

Why is only about 11GB of a 12GB card usable?

The display output, the inference framework and the CUDA or Metal context all reserve memory before your model loads. Roughly 1GB is a realistic reservation, so a 12GB card gives you about 11GB to work with. Planning to the full 12GB is the most common reason a model that "should" fit fails to load.

Should I run a bigger model at Q4 or a smaller one at Q8?

Usually the bigger model at Q4_K_M. Above about 4 bits per weight, parameter count matters more than precision, so a larger model quantized harder generally beats a smaller model kept at full precision. Below Q4 that stops being true and quality falls away quickly.

How fast will these run on 12GB?

VRAM capacity decides what fits; memory bandwidth decides how fast it runs, and that varies by device. Among the 4 machines we profile at this capacity, bandwidth ranges from 360 to 504 GB/s — which can mean an order-of-magnitude difference in tokens per second for the same model. The hardware matrix gives an estimate per device.

Can I generate images with 12GB?

Yes — 9 open image models run at 1024×1024 on a card this size, the best being Stable Diffusion XL 1.0 at BF16 / FP16. Diffusion models load a large text encoder alongside the image model, so offloading that encoder to CPU matters more than it does for LLMs.

Best LLMs for 12GB VRAM

11 of 30 models fit on a 12GB card. The biggest with real headroom is Gemma 4 12B at Q4_K_M, though DeepSeek R1 Distill Qwen 14B (14.8B) will also load if you keep the context short. Below is every model scored against the ~11GB a 12GB card realistically gives you, with the best quantization each one can use.

8GB 12GB 16GB 24GB 32GB 48GB 64GB 128GB

Every model, scored for 12GB

Showing the best quantization each model can use at this capacity — higher precision first, dropping to Q4_K_M only where it has to.

Model	Best quant	Needs	Headroom	Fit
Gemma 4 12B12B · Google DeepMind	Q4_K_M	10.5 GB	+0.5 GB	Comfortable
Google Gemma 2 9B9.24B · Google DeepMind	Q4_K_M	8 GB	+3 GB	Comfortable
Qwen3 8B8.2B · Alibaba Qwen	Q4_K_M	8.2 GB	+2.8 GB	Comfortable
Llama 3.1 8B Instruct8.03B · Meta AI	Q4_K_M	8 GB	+3 GB	Comfortable
DeepSeek R1 Distill Llama 8B8.0B · DeepSeek AI	Q4_K_M	8 GB	+3 GB	Comfortable
Qwen 2.5 Coder 7B Instruct7.6B · Alibaba Cloud	Q4_K_M	8 GB	+3 GB	Comfortable
Meta Llama 3.2 3B Instruct3.21B · Meta AI	BF16 / FP16	8 GB	+3 GB	Comfortable
DeepSeek R1 Distill Qwen 14B14.8B · DeepSeek AI	Q4_K_M	12 GB	—	Tight fit
Qwen 2.5 Coder 14B Instruct14.7B · Alibaba Cloud	Q4_K_M	12 GB	—	Tight fit
Microsoft Phi-4 14B14.7B · Microsoft	Q4_K_M	12 GB	—	Tight fit
Mistral NeMo 12B Instruct12.2B · Mistral AI & NVIDIA	Q4_K_M	12 GB	—	Tight fit
DeepSeek V3 671B MoE671B (37B active) · DeepSeek AI	—	—	—	Will not fit
DeepSeek V4-Flash284B (13B active) · DeepSeek AI	—	—	—	Will not fit
OpenAI gpt-oss-120b116.8B (5.1B active) · OpenAI	—	—	—	Will not fit
Llama 4 Scout (17B active, 16 experts)109B (17B active) · Meta	—	—	—	Will not fit
Qwen 2.5 72B Instruct72.7B · Alibaba Cloud	—	—	—	Will not fit
DeepSeek R1 Distill Llama 70B70.6B · DeepSeek AI	—	—	—	Will not fit
Llama 3.3 70B Instruct70.6B · Meta AI	—	—	—	Will not fit
Qwen3.6 35B-A3B (MoE)36B (3B active) · Alibaba Qwen	—	—	—	Will not fit
Cohere Command R 35B35.0B · Cohere	—	—	—	Will not fit
DeepSeek R1 Distill Qwen 32B32.5B · DeepSeek AI	—	—	—	Will not fit
Qwen 2.5 Coder 32B Instruct32.5B · Alibaba Cloud	—	—	—	Will not fit
Gemma 4 31B31.3B · Google DeepMind	—	—	—	Will not fit
Qwen3-Coder 30B-A3B Instruct30.5B (3.3B active) · Alibaba Qwen	—	—	—	Will not fit
Qwen3.8 27B27.8B · Alibaba Qwen	—	—	—	Will not fit
Google Gemma 2 27B27.2B · Google DeepMind	—	—	—	Will not fit
Gemma 4 26B-A4B (MoE)26.5B (3.8B active) · Google DeepMind	—	—	—	Will not fit
Mistral Small 3 24B24.0B · Mistral AI	—	—	—	Will not fit
Mistral Codestral 22B22.2B · Mistral AI	—	—	—	Will not fit
OpenAI gpt-oss-20b20.9B (3.6B active) · OpenAI	—	—	—	Will not fit

“Needs” is the recommended memory for that quantization, which leaves room for a working context. A 12GB card is treated as ~11GB usable, because the display, framework and CUDA or Metal context reserve roughly 1GB before your model loads.

Machines with about 12GB

We profile 4 machines at roughly this capacity. Their memory bandwidth ranges from 360 to 504 GB/s — and since generation speed is bandwidth-bound, two machines with the same 12GB can differ by an order of magnitude in tokens per second. Capacity decides what fits; bandwidth decides how fast.

Desktop PC (RTX 4070 Super 12GB)504 GB/s · ~66.9 t/s est. (8B)Desktop with Intel Arc B580 (12GB)456 GB/s · ~60.5 t/s est. (8B)Lenovo Legion Pro 7i (RTX 4080 12GB)432 GB/s · ~57.3 t/s est. (8B)Budget Workhorse PC (RTX 3060 12GB)360 GB/s · ~47.8 t/s est. (8B)

Image generation on 12GB

9 open image models run at 1024×1024 on a card this size — the best being Stable Diffusion XL 1.0 at BF16 / FP16, peaking near 9.2GB. Diffusion pipelines load a large text encoder alongside the image model, so offloading it matters more here than it does for language models.

Size a diffusion model for 12GB →

12GB VRAM: common questions

What is the best LLM for 12GB of VRAM?: Gemma 4 12B (12B) is the largest model with comfortable headroom, at Q4_K_M, needing about 10.5GB against the ~11GB usable on a 12GB card. DeepSeek R1 Distill Qwen 14B (14.8B) is larger and will still load, but only as a tight fit with a short context. 11 of the 30 models in our directory fit at 12GB in some form.
Which small LLMs run well on 12GB?: The best small LLMs for 12GB are the ones that fit with room left for context: Gemma 4 12B (12B, Q4_K_M), Google Gemma 2 9B (9.24B, Q4_K_M), Qwen3 8B (8.2B, Q4_K_M), Llama 3.1 8B Instruct (8.03B, Q4_K_M), DeepSeek R1 Distill Llama 8B (8.0B, Q4_K_M). DeepSeek R1 Distill Qwen 14B (14.8B, Q4_K_M), Qwen 2.5 Coder 14B Instruct (14.7B, Q4_K_M), Microsoft Phi-4 14B (14.7B, Q4_K_M), Mistral NeMo 12B Instruct (12.2B, Q4_K_M) will also load, but as tight fits that need a short context. A small model with headroom answers faster and holds a longer conversation than a larger one squeezed in.
Why is only about 11GB of a 12GB card usable?: The display output, the inference framework and the CUDA or Metal context all reserve memory before your model loads. Roughly 1GB is a realistic reservation, so a 12GB card gives you about 11GB to work with. Planning to the full 12GB is the most common reason a model that "should" fit fails to load.
Should I run a bigger model at Q4 or a smaller one at Q8?: Usually the bigger model at Q4_K_M. Above about 4 bits per weight, parameter count matters more than precision, so a larger model quantized harder generally beats a smaller model kept at full precision. Below Q4 that stops being true and quality falls away quickly.
How fast will these run on 12GB?: VRAM capacity decides what fits; memory bandwidth decides how fast it runs, and that varies by device. Among the 4 machines we profile at this capacity, bandwidth ranges from 360 to 504 GB/s — which can mean an order-of-magnitude difference in tokens per second for the same model. The hardware matrix gives an estimate per device.
Can I generate images with 12GB?: Yes — 9 open image models run at 1024×1024 on a card this size, the best being Stable Diffusion XL 1.0 at BF16 / FP16. Diffusion models load a large text encoder alongside the image model, so offloading that encoder to CPU matters more than it does for LLMs.

Less memoryBest LLMs for 8GB VRAM →More memoryBest LLMs for 16GB VRAM →

Guides behind these numbers

How memory requirements are calculated, and which cards give you this much.

All VRAM tiers