Ollama models and VRAM requirements

All 30 models, smallest first

Q4 is what ollama run gives you by default for most tags. Q8 roughly doubles the memory for a quality difference most people cannot pick out in ordinary use — start at Q4 and move up only if you have a reason.

Figures are the minimum to load at that quantization. For a specific context length, the VRAM calculator splits the requirement into weights, KV cache and overhead — long context can add several gigabytes on top of these numbers.

What fits your GPU

Start from the memory you have. Each tier scores every model against what a card that size realistically gives you once the display and the framework have taken their share.

On a Mac the GPU shares unified memory rather than having its own pool — the Mac VRAM optimizer works out how much of your RAM it can address. To start from a specific machine, the hardware matrix profiles real devices against every model.

Ollama or something else?

• LM Studio: An application. Browse, download, chat, inspect — no terminal needed.

  Ollama vs LM Studio

• llama.cpp: The engine. Total control, manual setup, newest features first.

  llama.cpp vs Ollama

• vLLM: Many users, NVIDIA-class GPU, whole model resident.

  Ollama vs vLLM

New to this entirely? The local LLM starting guide covers hardware, runner and first model in order.

Common questions

How much VRAM does Ollama need?

Ollama itself is a thin service; the memory goes to whichever model you load. At Q4_K_M — the default for most Ollama tags — a 3B model needs under 3GB, an 8B model around 6GB and a 70B model over 40GB. The table on this page lists the requirement for every model, and the figure includes the KV cache and framework overhead, not just the weights.

What happens if a model does not fit in my VRAM?

Ollama will still run it. It loads as many layers onto the GPU as fit and runs the rest on the CPU against system RAM, which works but is far slower because those layers run at system memory bandwidth. This is why Ollama can run models larger than your card, and why doing so is often unpleasant.

Which quantization does ollama run use by default?

For most model tags the default is a 4-bit quantization, usually Q4_K_M. You can request another by appending it to the tag. Q8_0 roughly doubles the memory for a quality difference most people cannot detect in ordinary use, so Q4 is the usual starting point.

Does Ollama use the GPU automatically?

Yes, where it can. It detects CUDA on NVIDIA, ROCm on supported AMD cards and Metal on Apple Silicon, and offloads as many layers as the available memory allows. On a Mac the GPU draws from unified memory rather than a separate pool, so the usable figure is roughly 70% of total RAM.

Is Ollama better than LM Studio?

They are not competing on capability — both run the same GGUF models on the same hardware. Ollama is a background service with a command line and an HTTP API, which is what you want when another application calls the model. LM Studio is a desktop app with a model browser and chat window, which is what you want when you are the one talking to it.

Choosing a runner, what each quantization costs, and how memory is sized.

• Best Local LLM Runners: Ollama vs LM Studio vs Jan vs llama.cpp vs vLLMWhich runner suits your hardware and workflow.Read the guide
• Quantization Guide: Q4_K_M vs Q8_0 vs FP8What each quant level costs you in memory and quality.Read the guide
• How Much RAM and VRAM Do You Need to Run Local LLMs?Model weights, KV cache and OS overhead, tier by tier.Read the guide