What is the best LLM for 8GB of VRAM?

Meta Llama 3.2 3B Instruct (3.21B) is the largest model with comfortable headroom, at Q8_0, needing about 6GB against the ~7GB usable on a 8GB card. Google Gemma 2 9B (9.24B) is larger and will still load, but only as a tight fit with a short context. 6 of the 30 models in our directory fit at 8GB in some form.

Which small LLMs run well on 8GB?

At 8GB, one model in our directory fits with real room left for context: Meta Llama 3.2 3B Instruct (3.21B, Q8_0). Google Gemma 2 9B (9.24B, Q4_K_M), Qwen3 8B (8.2B, Q4_K_M), Llama 3.1 8B Instruct (8.03B, Q4_K_M), DeepSeek R1 Distill Llama 8B (8.0B, Q4_K_M) will also load, but as tight fits that need a short context. A small model with headroom answers faster and holds a longer conversation than a larger one squeezed in.

Why is only about 7GB of a 8GB card usable?

The display output, the inference framework and the CUDA or Metal context all reserve memory before your model loads. Roughly 1GB is a realistic reservation, so a 8GB card gives you about 7GB to work with. Planning to the full 8GB is the most common reason a model that "should" fit fails to load.

Should I run a bigger model at Q4 or a smaller one at Q8?

Usually the bigger model at Q4_K_M. Above about 4 bits per weight, parameter count matters more than precision, so a larger model quantized harder generally beats a smaller model kept at full precision. Below Q4 that stops being true and quality falls away quickly.

How fast will these run on 8GB?

VRAM capacity decides what fits; memory bandwidth decides how fast it runs, and that varies by device. Among the 19 machines we profile at this capacity, bandwidth ranges from 68.25 to 272 GB/s — which can mean an order-of-magnitude difference in tokens per second for the same model. The hardware matrix gives an estimate per device.

Can I generate images with 8GB?

Yes — 6 open image models run at 1024×1024 on a card this size, the best being Stable Diffusion 1.5 at BF16 / FP16. Diffusion models load a large text encoder alongside the image model, so offloading that encoder to CPU matters more than it does for LLMs.

Best LLMs for 8GB VRAM

6 of 30 models fit on a 8GB card. The biggest with real headroom is Meta Llama 3.2 3B Instruct at Q8_0, though Google Gemma 2 9B (9.24B) will also load if you keep the context short. Below is every model scored against the ~7GB a 8GB card realistically gives you, with the best quantization each one can use.

8GB 12GB 16GB 24GB 32GB 48GB 64GB 128GB

Every model, scored for 8GB

Showing the best quantization each model can use at this capacity — higher precision first, dropping to Q4_K_M only where it has to.

Model	Best quant	Needs	Headroom	Fit
Meta Llama 3.2 3B Instruct3.21B · Meta AI	Q8_0	6 GB	+1 GB	Comfortable
Google Gemma 2 9B9.24B · Google DeepMind	Q4_K_M	8 GB	—	Tight fit
Qwen3 8B8.2B · Alibaba Qwen	Q4_K_M	8.2 GB	—	Tight fit
Llama 3.1 8B Instruct8.03B · Meta AI	Q4_K_M	8 GB	—	Tight fit
DeepSeek R1 Distill Llama 8B8.0B · DeepSeek AI	Q4_K_M	8 GB	—	Tight fit
Qwen 2.5 Coder 7B Instruct7.6B · Alibaba Cloud	Q4_K_M	8 GB	—	Tight fit
DeepSeek V3 671B MoE671B (37B active) · DeepSeek AI	—	—	—	Will not fit
DeepSeek V4-Flash284B (13B active) · DeepSeek AI	—	—	—	Will not fit
OpenAI gpt-oss-120b116.8B (5.1B active) · OpenAI	—	—	—	Will not fit
Llama 4 Scout (17B active, 16 experts)109B (17B active) · Meta	—	—	—	Will not fit
Qwen 2.5 72B Instruct72.7B · Alibaba Cloud	—	—	—	Will not fit
DeepSeek R1 Distill Llama 70B70.6B · DeepSeek AI	—	—	—	Will not fit
Llama 3.3 70B Instruct70.6B · Meta AI	—	—	—	Will not fit
Qwen3.6 35B-A3B (MoE)36B (3B active) · Alibaba Qwen	—	—	—	Will not fit
Cohere Command R 35B35.0B · Cohere	—	—	—	Will not fit
DeepSeek R1 Distill Qwen 32B32.5B · DeepSeek AI	—	—	—	Will not fit
Qwen 2.5 Coder 32B Instruct32.5B · Alibaba Cloud	—	—	—	Will not fit
Gemma 4 31B31.3B · Google DeepMind	—	—	—	Will not fit
Qwen3-Coder 30B-A3B Instruct30.5B (3.3B active) · Alibaba Qwen	—	—	—	Will not fit
Qwen3.8 27B27.8B · Alibaba Qwen	—	—	—	Will not fit
Google Gemma 2 27B27.2B · Google DeepMind	—	—	—	Will not fit
Gemma 4 26B-A4B (MoE)26.5B (3.8B active) · Google DeepMind	—	—	—	Will not fit
Mistral Small 3 24B24.0B · Mistral AI	—	—	—	Will not fit
Mistral Codestral 22B22.2B · Mistral AI	—	—	—	Will not fit
OpenAI gpt-oss-20b20.9B (3.6B active) · OpenAI	—	—	—	Will not fit
DeepSeek R1 Distill Qwen 14B14.8B · DeepSeek AI	—	—	—	Will not fit
Qwen 2.5 Coder 14B Instruct14.7B · Alibaba Cloud	—	—	—	Will not fit
Microsoft Phi-4 14B14.7B · Microsoft	—	—	—	Will not fit
Mistral NeMo 12B Instruct12.2B · Mistral AI & NVIDIA	—	—	—	Will not fit
Gemma 4 12B12B · Google DeepMind	—	—	—	Will not fit

“Needs” is the recommended memory for that quantization, which leaves room for a working context. A 8GB card is treated as ~7GB usable, because the display, framework and CUDA or Metal context reserve roughly 1GB before your model loads.

Machines with about 8GB

We profile 19 machines at roughly this capacity. Their memory bandwidth ranges from 68.25 to 272 GB/s — and since generation speed is bandwidth-bound, two machines with the same 8GB can differ by an order of magnitude in tokens per second. Capacity decides what fits; bandwidth decides how fast.

Mainstream Desktop (RTX 4060 8GB)272 GB/s · ~36.1 t/s est. (8B)ASUS ROG Zephyrus G16 (RTX 4070 Laptop)256 GB/s · ~34 t/s est. (8B)Lenovo Legion 5i (RTX 4060 Laptop 8GB)256 GB/s · ~34 t/s est. (8B)ASUS TUF Gaming A15 (RTX 4060 Laptop 8GB)256 GB/s · ~34 t/s est. (8B)Acer Predator Helios Neo 16 (RTX 4070 Laptop)256 GB/s · ~34 t/s est. (8B)Dell Alienware m16 R2 (RTX 4070 8GB)256 GB/s · ~34 t/s est. (8B)ASUS ROG Zephyrus G16 (AMD AI 9 + RTX 4070)256 GB/s · ~34 t/s est. (8B)Apple Mac mini (M6, 16GB, 2026)153 GB/s · ~20.3 t/s est. (8B)Lenovo Yoga Slim 7i Aura (Core Ultra 7 256V)136 GB/s · ~18 t/s est. (8B)Microsoft Surface Laptop 7135 GB/s · ~17.9 t/s est. (8B)

Image generation on 8GB

6 open image models run at 1024×1024 on a card this size — the best being Stable Diffusion 1.5 at BF16 / FP16, peaking near 4.2GB. Diffusion pipelines load a large text encoder alongside the image model, so offloading it matters more here than it does for language models.

Size a diffusion model for 8GB →

8GB VRAM: common questions

What is the best LLM for 8GB of VRAM?: Meta Llama 3.2 3B Instruct (3.21B) is the largest model with comfortable headroom, at Q8_0, needing about 6GB against the ~7GB usable on a 8GB card. Google Gemma 2 9B (9.24B) is larger and will still load, but only as a tight fit with a short context. 6 of the 30 models in our directory fit at 8GB in some form.
Which small LLMs run well on 8GB?: At 8GB, one model in our directory fits with real room left for context: Meta Llama 3.2 3B Instruct (3.21B, Q8_0). Google Gemma 2 9B (9.24B, Q4_K_M), Qwen3 8B (8.2B, Q4_K_M), Llama 3.1 8B Instruct (8.03B, Q4_K_M), DeepSeek R1 Distill Llama 8B (8.0B, Q4_K_M) will also load, but as tight fits that need a short context. A small model with headroom answers faster and holds a longer conversation than a larger one squeezed in.
Why is only about 7GB of a 8GB card usable?: The display output, the inference framework and the CUDA or Metal context all reserve memory before your model loads. Roughly 1GB is a realistic reservation, so a 8GB card gives you about 7GB to work with. Planning to the full 8GB is the most common reason a model that "should" fit fails to load.
Should I run a bigger model at Q4 or a smaller one at Q8?: Usually the bigger model at Q4_K_M. Above about 4 bits per weight, parameter count matters more than precision, so a larger model quantized harder generally beats a smaller model kept at full precision. Below Q4 that stops being true and quality falls away quickly.
How fast will these run on 8GB?: VRAM capacity decides what fits; memory bandwidth decides how fast it runs, and that varies by device. Among the 19 machines we profile at this capacity, bandwidth ranges from 68.25 to 272 GB/s — which can mean an order-of-magnitude difference in tokens per second for the same model. The hardware matrix gives an estimate per device.
Can I generate images with 8GB?: Yes — 6 open image models run at 1024×1024 on a card this size, the best being Stable Diffusion 1.5 at BF16 / FP16. Diffusion models load a large text encoder alongside the image model, so offloading that encoder to CPU matters more than it does for LLMs.

More memoryBest LLMs for 12GB VRAM →

Guides behind these numbers

How memory requirements are calculated, and which cards give you this much.

All VRAM tiers