Best LLMs for 48GB VRAM

Showing the best quantization each model can use at this capacity — higher precision first, dropping to Q4_K_M only where it has to.

“Needs” is the recommended memory for that quantization, which leaves room for a working context. A 48GB card is treated as ~47GB usable, because the display, framework and CUDA or Metal context reserve roughly 1GB before your model loads.

We profile 4 machines at roughly this capacity. Their memory bandwidth ranges from 307 to 936 GB/s — and since generation speed is bandwidth-bound, two machines with the same 48GB can differ by an order of magnitude in tokens per second. Capacity decides what fits; bandwidth decides how fast.

Image generation on 48GB

11 open image models run at 1024×1024 on a card this size — the best being FLUX.1 [dev] at BF16 / FP16, peaking near 36.5GB. Diffusion pipelines load a large text encoder alongside the image model, so offloading it matters more here than it does for language models.

48GB VRAM: common questions

What is the best LLM for 48GB of VRAM?

Qwen3.6 35B-A3B (MoE) (36B (3B active)) is the largest model with comfortable headroom, at Q8_0, needing about 42.6GB against the ~47GB usable on a 48GB card. Qwen 2.5 72B Instruct (72.7B) is larger and will still load, but only as a tight fit with a short context. 26 of the 30 models in our directory fit at 48GB in some form.

Why is only about 47GB of a 48GB card usable?

The display output, the inference framework and the CUDA or Metal context all reserve memory before your model loads. Roughly 1GB is a realistic reservation, so a 48GB card gives you about 47GB to work with. Planning to the full 48GB is the most common reason a model that "should" fit fails to load.

Should I run a bigger model at Q4 or a smaller one at Q8?

Usually the bigger model at Q4_K_M. Above about 4 bits per weight, parameter count matters more than precision, so a larger model quantized harder generally beats a smaller model kept at full precision. Below Q4 that stops being true and quality falls away quickly.

How fast will these run on 48GB?

VRAM capacity decides what fits; memory bandwidth decides how fast it runs, and that varies by device. Among the 4 machines we profile at this capacity, bandwidth ranges from 307 to 936 GB/s — which can mean an order-of-magnitude difference in tokens per second for the same model. The hardware matrix gives an estimate per device.

Can I generate images with 48GB?

Yes — 11 open image models run at 1024×1024 on a card this size, the best being FLUX.1 [dev] at BF16 / FP16. Diffusion models load a large text encoder alongside the image model, so offloading that encoder to CPU matters more than it does for LLMs.

How memory requirements are calculated, and which cards give you this much.

• How Much RAM and VRAM Do You Need to Run Local LLMs?Model weights, KV cache and OS overhead, tier by tier.Read the guide
• Quantization Guide: Q4_K_M vs Q8_0 vs FP8What each quant level costs you in memory and quality.Read the guide
• Best Budget GPUs for Local LLMs16GB cards, used RTX 3090s and current street prices.Read the guide