Local LLM Models Directory
Detailed VRAM memory footprints, quantization estimates, and hardware compatibility for the world's leading open-source large language models.
Alibaba's newest open mid-size model and one of the strongest dense models you can run on a single 24GB GPU or 32GB+ Mac.
A Mixture-of-Experts model that needs 32GB-class memory but generates text like a 3B model.
A fast MoE coding model for local agents and editor assistants.
A capable small dense model for 16GB machines.
Google's largest dense Gemma 4 model, for 24GB GPUs and 32GB+ Macs.
Gemma 4's MoE model: roughly 26B of knowledge at the speed of a ~4B model.
A well-rounded multimodal model for 16GB machines.
OpenAI's small open-weight reasoning model, designed to run in about 16GB of memory.
OpenAI's large open-weight reasoning model. A great match for 96-128GB unified-memory machines.
DeepSeek's 2026 MoE flagship for workstation-class unified memory.
Meta's MoE Llama 4 model, sized for 96-128GB machines.
The ultimate sweet spot for local chain-of-thought mathematical reasoning and coding.
Blazing-fast reasoning for entry-level 8GB GPUs and standard 16GB laptops.
Frontier-class mathematical and code reasoning for 24GB GPUs and 36GB+ Macs.
The flagship 70-billion parameter reasoning model for Mac Studio and multi-GPU rigs.
Meta flagship open-weights generalist model rivaling GPT-4o.
The global standard compact LLM for consumer laptops and edge devices.
The world benchmark leader for open-weights coding and autonomous software engineering.
The ideal local coding assistant for 16GB developer laptops and GPUs.
Ultra-fast local code copilot for budget GPUs and 8GB/16GB machines.
Joint NVIDIA & Mistral 12B model with Tekken tokenizer and 128k context.
Microsoft state-of-the-art synthetic reasoning model punching far above its weight class.
Google DeepMind high-efficiency architectural masterpiece with sliding window attention.
The 27B powerhouse capable of competing with 70B models on single 16GB–24GB GPUs.
The gold standard for enterprise Retrieval Augmented Generation (RAG) and tool use.
Mistral dedicated generative programming model with fill-in-the-middle (FIM) mastery.
Mistral next-generation compact model beating previous generation 70B benchmarks.
The open-weights frontier flagship rivaling Claude 3.5 Sonnet and GPT-4o.
The global open-weights leaderboard champion for general knowledge and multi-task reasoning.
Meta ultra-compact edge model built for instantaneous local inference on laptops and phones.
How to Read Local LLM Memory Requirements
Q4_K_M (4-Bit)
The industry standard quantization for daily local inference. It reduces model memory consumption by approximately 70% with less than a 1% loss in reasoning perplexity.
Q8_0 (8-Bit)
True 8-bit quantization offering near-zero quality loss. Essential for competitive math, complex logic, and delicate formatting when you have ample VRAM headroom.
KV Cache Overhead
Beyond raw weight size, active conversation context (e.g. 16k or 32k tokens) consumes additional VRAM for the Key-Value attention cache. Always leave 1.5GB to 3GB of headroom.
Model setup guides
Step-by-step instructions for the models in this directory.