Local LLM Models Directory
Detailed VRAM memory footprints, quantization estimates, and hardware compatibility for the world's leading open-source large language models.
Alibaba's newest open mid-size model and one of the strongest dense models you can run on a single 24GB GPU or 32GB+ Mac.
A Mixture-of-Experts model that needs 32GB-class memory but generates text like a 3B model.
A fast MoE coding model for local agents and editor assistants.
A capable small dense model for 16GB machines.
Google's largest dense Gemma 4 model, for 24GB GPUs and 32GB+ Macs.
Gemma 4's MoE model: roughly 26B of knowledge at the speed of a ~4B model.
A well-rounded multimodal model for 16GB machines.
OpenAI's small open-weight reasoning model, designed to run in about 16GB of memory.
OpenAI's large open-weight reasoning model. A great match for 96-128GB unified-memory machines.
DeepSeek's 2026 MoE flagship for workstation-class unified memory.
Meta's MoE Llama 4 model, sized for 96-128GB machines.
The ultimate sweet spot for local chain-of-thought mathematical reasoning and coding.
Blazing-fast reasoning for entry-level 8GB GPUs and standard 16GB laptops.
Frontier-class mathematical and code reasoning for 24GB GPUs and 36GB+ Macs.
The flagship 70-billion parameter reasoning model for Mac Studio and multi-GPU rigs.
Meta flagship open-weights generalist model rivaling GPT-4o.
The global standard compact LLM for consumer laptops and edge devices.
The world benchmark leader for open-weights coding and autonomous software engineering.
The ideal local coding assistant for 16GB developer laptops and GPUs.
Ultra-fast local code copilot for budget GPUs and 8GB/16GB machines.
Joint NVIDIA & Mistral 12B model with Tekken tokenizer and 128k context.
Microsoft state-of-the-art synthetic reasoning model punching far above its weight class.
Google DeepMind high-efficiency architectural masterpiece with sliding window attention.
The 27B powerhouse capable of competing with 70B models on single 16GB–24GB GPUs.
The gold standard for enterprise Retrieval Augmented Generation (RAG) and tool use.
Mistral dedicated generative programming model with fill-in-the-middle (FIM) mastery.
Mistral next-generation compact model beating previous generation 70B benchmarks.
The open-weights frontier flagship rivaling Claude 3.5 Sonnet and GPT-4o.
The global open-weights leaderboard champion for general knowledge and multi-task reasoning.
Meta ultra-compact edge model built for instantaneous local inference on laptops and phones.
Frontier open models: too big for the directory
These open-weight models are listed separately and are not part of the fit counts elsewhere in this section. Each download size is read from the named repository on 2026-10-24; a device counts as able to hold a model when its GPU-usable memory covers the file (or the part that must be in memory) plus 10%.
| Model | Parameters | Smallest 4-bit download | Licence | Devices in our database that hold it |
|---|---|---|---|---|
| Qwen3.8-Flash-NextQwen | 180B6B active | 93.7 GBUD-IQ4_XS GGUF | Qwen Community License 1.0Name attribution above 100M monthly users or US$20M monthly revenue; a separate licence for model-as-a-service businesses. | 6 of 79: Apple Mac Studio (M2 Ultra 192GB), Apple Mac Studio (M5 Ultra, 256GB, 2026), Apple Mac Studio (M5 Ultra, 512GB, 2026) and 3 moreStrata runs it from a 12GB graphics card plus 32-64GB of RAM by keeping most experts in system memory.Read the guide |
| GLM-5.3-FlashZ.ai | 321.3B18B active | 156.8 GBUD-IQ4_XS GGUF | MITPermissive. The larger GLM-5.3 uses a different, custom licence. | 3 of 79: Apple Mac Studio (M5 Ultra, 256GB, 2026), Apple Mac Studio (M5 Ultra, 512GB, 2026), Apple Mac Studio (M3 Ultra, 512GB, 2025)Read the guide |
| DeepSeek V4.1-Flash-1017DeepSeek | 763.2B16B active | 500.71 GB297.9 GB in memoryUD-Q4_K_XL GGUF | MITPermissive. | 2 of 79: Apple Mac Studio (M5 Ultra, 512GB, 2026), Apple Mac Studio (M3 Ultra, 512GB, 2025)Its 202.8 GB of Engram tables can stay on the SSD (unsloth), so 297.9 GB must be in memory. 16B active per token when decoding.Read the guide |
| Muse Spark OpenMeta | 816B42B active | 470.07 GBQ4_K_S GGUF | Muse Model License"Built with Muse Spark Open" attribution; a separate agreement for Model-as-a-Service businesses with group revenue above US$20M. | None of our 79 devices.Meta's own NVFP4 setup uses two 512GB Mac Studios. 2- and 3-bit files (304-372 GB) fit one 512GB Mac Studio.Read the guide |
| DeepSeek V4-ProDeepSeek | 1,598.8B | 799 GB4-bit weights (computed) | MITPermissive. | None of our 79 devices.Read the guide |
How to Read Local LLM Memory Requirements
Q4_K_M (4-Bit)
The industry standard quantization for daily local inference. It reduces model memory consumption by approximately 70% with less than a 1% loss in reasoning perplexity.
Q8_0 (8-Bit)
True 8-bit quantization offering near-zero quality loss. Essential for competitive math, complex logic, and delicate formatting when you have ample VRAM headroom.
KV Cache Overhead
Beyond raw weight size, active conversation context (e.g. 16k or 32k tokens) consumes additional VRAM for the Key-Value attention cache. Always leave 1.5GB to 3GB of headroom.
Model setup guides
Setup guides and comparisons for the models in this directory.
- How to Run Qwen3.8 27B LocallyHardware, VRAM and step-by-step setup.Read the guide
- Meta Muse Spark Open Locally: 816B Weights, the Memory It Takes and the LicenceFile sizes from 176GB, which Macs hold it, and what the $20M clause really covers.Read the guide
- Best Local Embedding Models for RAGDimensions, licences and what vector storage really costs.Read the guide
- Best Local LLMs for CodingModels by VRAM tier, licence and scaffold support.Read the guide
- Gemma 4 12B vs Qwen3.5 9B: Which Small Local Model to RunMemory from 8K to 256K context, estimated speed and vendor benchmarks for two 8-16GB models.Read the guide
- How to Run GLM-5.3-Flash Locally321B, 18B active, MIT: which build fits 192GB to 512GB.Read the guide