Running Giant Open Models Locally: DeepSeek V4, Qwen3.8 & Llama 4 Hardware Sizing Guide

The biggest open-weight models are now Mixture-of-Experts (MoE) giants with hundreds of billions, or even trillions, of parameters. Can you run them at home? Sometimes, if you understand one key idea: MoE models need memory for all their parameters, but only compute a small “active” slice for each token. This guide explains how to size hardware for today’s largest open models, from DeepSeek V4 and Qwen3.8 to Llama 4 and gpt-oss-120b, and compares them with classic 70B-class dense models.

Dense vs Mixture-of-Experts in One Minute

• Dense models (such as the older Llama 3.3 70B and Qwen 2.5 72B) use every parameter for every token. Memory and speed both scale with total size.
• MoE models split the network into many “experts” and route each token through only a few. You still need memory for all the weights, but generation speed depends on the much smaller number of active parameters.

So an MoE model can be huge on disk yet generate text surprisingly fast, as long as you have enough memory to hold it.

Today’s Largest Open Models

DeepSeek also published V4.1-Flash on September 10, 2026, so check its model card for the latest details. Its API price against its real size is covered in DeepSeek API pricing vs running it locally. Memory figures are estimates for weights at ~4.8 bits per parameter, before context. Very low-bit “dynamic” quantizations can shrink these a lot, at a quality cost. See our quantization guide.

What Hardware Can Hold Them?

Apple says Mac Studio systems can be clustered over Thunderbolt 5 with RDMA to pool memory for the largest models. On PCs, multi-GPU setups with runtimes like vLLM or llama.cpp can split models across cards, but VRAM per dollar is poor at 2026 GPU prices.

Estimated Speed: Why Active Parameters Matter

Using tokens/sec ≈ bandwidth × 0.6–0.75 ÷ active weights in GB, these are theoretical upper-bound estimates:

• DeepSeek V4-Flash on an M5 Ultra (~7.8GB active at 4-bit, 1.2TB/s): up to roughly 90–115 tok/s in theory. Real-world MoE efficiency is lower, so expect less.
• gpt-oss-120b on a Strix Halo (~2.7GB active, 256GB/s): up to roughly 55–70 tok/s in theory.
• A 70B dense model on an M5 Max (~42GB, 614GB/s): roughly 9–11 tok/s, because every parameter is active.

These are estimates from specs, not our own measurements. They show why big MoE models are practical on large unified-memory machines while dense 70B models feel slow. Prompt processing (reading long inputs) is compute-heavy, and that’s where Apple’s 2026 Neural Accelerators and NVIDIA GPUs help most.

Should You Run a Giant Model Locally?

• Yes, if you need privacy, offline use or heavy daily usage, and you already have (or can justify) a 128–512GB machine.
• Probably not, if you only use it occasionally. Hosted APIs for these same open models are often far cheaper than the hardware. Compare costs with our API Pricing Break-Even Calculator.
• A good middle ground: mid-size models like Qwen3.8-27B, Gemma 4 31B or gpt-oss-120b cover most local needs on 32–128GB machines.

For more detail, see the Local LLM Directory, our best Mac for local LLMs guide and our DeepSeek local setup guide.

FAQ

Can I run DeepSeek V4 on a gaming PC?

Not realistically. Even V4-Flash needs roughly 175GB of memory at 4-bit, far beyond a single consumer GPU.

Why is an MoE model faster than a 70B dense model?

Only its active parameters are read for each token. For example, DeepSeek V4-Flash activates about 13B, compared with all 70B in a dense 70B model.

Sources

Model cards on Hugging Face: DeepSeek, Qwen, Meta Llama, OpenAI gpt-oss. Also DeepSeek V4 release notes and Apple Mac Studio M5 Ultra.

Updated September 11, 2026: rewritten for DeepSeek V4, Qwen3.8, Llama 4 and gpt-oss, with clearly labelled estimates.