Local LLM Model Recommender

How the recommendation works

The recommender compares the memory you have against the 4-bit weight size of every model in our directory, then leaves headroom for the KV cache and framework overhead. A model is only suggested if it fits with room to actually use it, rather than merely loading.

Speed estimates use the same formula as the rest of the site: tokens per second is approximately memory bandwidth multiplied by 0.65, divided by the active weights in gigabytes. For a mixture-of-experts model the active weights are far smaller than the total, which is why those models punch above their size on bandwidth-limited machines. These are arithmetic estimates from published specifications, not lab measurements — BFWR does not run a hardware test lab.

Task matters as much as size. A 30B coding model will outperform a larger general model on code, and a reasoning-tuned model will do better on multi-step problems than a chat model twice its size. Pick the task first, then the largest model of that type that fits.

Common questions

How do I know how much memory I actually have for a model?

On a discrete GPU it is the VRAM figure, minus about 1GB for the display and framework overhead. On Apple Silicon and other unified-memory machines the GPU can typically address around 70% of total RAM by default, so a 16GB Mac gives roughly 10-11GB to work with. The hardware matrix lists a GPU-usable figure for every device we profile.

Should I run a bigger model at Q4 or a smaller one at Q8?

Usually the bigger model at Q4_K_M. Parameter count tends to matter more than precision above 4 bits, and Q4_K_M is the size-to-quality sweet spot in practice. Drop below Q4 only when nothing else fits, because quality falls away quickly at Q3 and Q2.

Why does a mixture-of-experts model run faster than its size suggests?

Because only a fraction of its weights are active for each token. A 35B model with 3B active reads roughly 3B worth of weights per token, so it generates at something closer to the speed of a small model while needing the memory of a large one. That trade suits machines with plenty of RAM but modest bandwidth.

Does a longer context need more memory?

Yes, and it is often the thing that pushes a working setup over the edge. Context lives in the KV cache, which grows with every token and competes with the weights for the same memory. A model that fits at 4k may not fit at 32k. The VRAM calculator works out the exact cost.

Which runner should I use once I have picked a model?

Ollama is the simplest starting point and covers most needs. LM Studio adds a GUI, llama.cpp gives the most control, and vLLM is aimed at serving multiple concurrent users rather than single-user local chat.

Sizing memory, picking a runner and choosing a quantization level.

• How Much RAM and VRAM Do You Need to Run Local LLMs?Model weights, KV cache and OS overhead, tier by tier.Read the guide
• llmfit: How the Local LLM Fit Checker WorksIts fit and speed formula, checked against measurements.Read the guide
• Best Local LLM Runners: Ollama vs LM Studio vs Jan vs llama.cpp vs vLLMWhich runner suits your hardware and workflow.Read the guide