Qwen3.8-Flash-Next is the most interesting open model of the autumn for anyone with a big-memory machine. It has 180 billion parameters in total, but only 6 billion are active for each token, so it can generate text quickly on hardware that would crawl through a dense model of the same size. Qwen calls it an experimental preview of the architecture behind Qwen4. This guide covers what it needs in memory, which build fits which machine, how fast it actually runs, and the licence condition worth reading first.
Quick answer
- Size: the model card gives “125B with 6B activated, plus 51B n-gram embedding and 4B MTP”, 180B in all. It reads images as well as text.
- Memory: about 94GB for a usable 4-bit-class build (unsloth’s UD-IQ4_XS), 111GB for UD-Q4_K_XL, and 120GB for Ollama’s Q4_K_M. A 128GB Mac, DGX Spark or Strix Halo machine can run the ~90–94GB builds; Ollama’s default 4-bit build is too big for them.
- Speed: a community measurement on a 128GB Strix Halo machine found 16.8 tokens/sec without speculative decoding and up to 47.1 with it. That is well below what memory bandwidth alone predicts, because the runtime support is still new.
- Runtimes: llama.cpp supports it (and so LM Studio and unsloth’s tools), and Ollama has builds. Qwen’s own card points production users to SGLang and vLLM.
What makes it different
Most large Mixture-of-Experts models keep almost all their parameters in the experts. Qwen3.8-Flash-Next splits them three ways. The 125B core has 512 experts, of which 10 routed and 1 shared are used per token, which is where the 6B active figure comes from. Another 51B sits in an “n-gram embedding” table indexed by short word sequences, which Qwen’s card says “requires less computation and is more amenable to offloading than Mixture-of-Experts (MoE).” The last 4B is a multi-token-prediction (MTP) head, which runtimes can use to draft several tokens at once.
The context window is 262,144 tokens natively, extensible to about a million. The model thinks before answering by default. Qwen also offers Qwen3.8-Flash on its Qwen Cloud service; the card describes that as the official version “based on Qwen3.8-Flash-Next with more production features.” The open weights are the preview.
How much memory it needs
Sizes below are the downloads themselves, read from the unsloth repository’s file listing and Ollama’s tag page on 12 October 2026, in decimal gigabytes. Leave room on top for the context (KV cache) and the operating system.
| Build | Download size | Where |
|---|---|---|
| UD-IQ1_S (1-bit class) | 72.6 GB | unsloth GGUF |
| UD-Q2_K_XL | 78.9 GB | unsloth GGUF |
| UD-Q3_K_XL | 90.0 GB | unsloth GGUF |
| UD-IQ4_XS | 93.7 GB | unsloth GGUF |
| NVFP4 (MLX, for Macs) | 105 GB | Ollama 125b-a6b-nvfp4 |
| UD-Q4_K_XL | 111.3 GB | unsloth GGUF |
| Q4_K_M | 120 GB | Ollama 125b-a6b-q4_K_M |
| Q8_0 | 188–189 GB | unsloth GGUF / Ollama |
| BF16 | 354–355 GB | unsloth GGUF / Ollama |
unsloth’s own guide puts the total memory needed at 75GB for its 1-bit build, 90GB for 3-bit and 96–114GB for 4-bit, and LM Studio lists a minimum of 96GB of system memory. The vision projector (under 1GB) and the optional MTP file (2.8GB for the shared Q8_0 version) come on top.
Which machine runs which build
- 128GB unified memory (Mac Studio or MacBook Pro with M5 Max, DGX Spark, Ryzen AI Max+ 395 systems): the 3-bit and IQ4_XS builds, at 90–94GB. On a Mac you will need to raise macOS’s default GPU memory limit first (our Mac VRAM Optimizer generates the command); on Windows, AMD’s Variable Graphics Memory caps the GPU at 96GB, while Linux users can allocate more. Ollama’s 120GB Q4_K_M will not fit.
- 192GB (Ryzen AI Max+ PRO 495 systems): room for UD-Q4_K_XL at 111GB. How much of the 192GB the GPU can address has not been published, so check before trying anything larger.
- 256GB and up (Mac Studio M5 Ultra): Q8_0 at about 188GB.
- A PC with one consumer GPU: the weights do not fit in VRAM, so most of the model would sit in system RAM. unsloth argues this model suffers less from that than usual: its guide says that whether you run it “on a CPU with system RAM or on a GPU with VRAM may make relatively little difference.” That is unsloth’s claim, not an independent measurement; you would still need 96GB or more of system RAM.
Our comparisons of the 128GB machines cover the trade-offs in detail: DGX Spark vs Strix Halo and Strix Halo vs M5 Max.
How fast it really runs
Our usual estimate for generation speed is memory bandwidth × 0.65 ÷ the size of the weights read per token. For a 256GB/s Strix Halo machine running the IQ4_XS build, that gives about 53.3 tokens/sec, because only about 3GB of active weights is read per token. (Our tokens-per-second guide explains the formula.)
Real numbers are lower, and the reason is instructive. In a reproducible write-up on llama.cpp’s forum (discussion #27950, August 2026), a user measured single-stream generation on a 128GB Ryzen AI Max+ 395 machine with the UD-IQ4_XS build:
| Workload | No speculation (tok/s) | MTP + n-gram speculation (tok/s) |
|---|---|---|
| File rewrite, 8K context | 16.8 | 47.1 |
| New code, 8K context | 16.8 | 31.7 |
| File rewrite, 24K context | ~15 | 28.6 |
| New code, 24K context | ~15 | 25.4 |
Without speculation the model runs at about a third of the bandwidth estimate. The write-up traces much of that to the model’s sparse-attention indexer, which on AMD’s ROCm fell back to the CPU for long contexts until fixes landed. New architectures usually run below their bandwidth ceiling at first and close the gap as runtimes are optimised, so treat any figure for this model as a snapshot. Speculative decoding with the MTP head is the biggest lever: it roughly doubles to triples throughput on predictable text such as file rewrites, and helps less on genuinely new output.
How to run it
llama.cpp
llama.cpp added the model in pull request #27742 (merged 27 August 2026) and native MTP drafting in #29761 (merged 1 October), so use a current build. unsloth’s guide runs it like this, with the MTP head as a draft model:
llama-server \
-hf unsloth/Qwen3.8-Flash-Next-GGUF:UD-Q4_K_XL \
-md unsloth/Qwen3.8-Flash-Next-GGUF/MTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf \
--spec-type draft-mtp --spec-draft-n-max 5
On a 128GB machine, swap UD-Q4_K_XL for UD-IQ4_XS or UD-Q3_K_XL. llama.cpp also accepts --spec-type ngram-mod, the second lever in the measurements above.
Ollama and LM Studio
Ollama publishes the model as qwen3.8-flash-next, with tags for Q4_K_M (120GB), Q8_0, BF16 and an MLX NVFP4 build for Macs (105GB, run on a Mac with ollama run qwen3.8-flash-next:125b-a6b-nvfp4). Its smallest build is still larger than a 128GB machine’s usable GPU memory by default, so check the limit before you download. LM Studio lists the model with a 96GB minimum and uses the GGUF builds, which gives you the smaller quantizations. For a comparison of the runners, see our local LLM runner guide.
Recommended settings
Qwen’s card recommends temperature=1.0, top_p=0.95, top_k=20 in thinking mode, and temperature=0.7, top_p=0.80, top_k=20, presence_penalty=1.5 with thinking turned off. Long contexts cost memory on top of the weights; our context window guide explains why.
The licence: read clause 2 if you sell AI services
The weights are under the Qwen Community License 1.0, not Apache 2.0 like the smaller Qwen3.8 27B. For most people it is permissive: it allows you to “use, copy, modify, merge, publish, distribute, sublicense, sell, deploy, host, fine-tune, and create derivative works”. Two conditions apply:
- Products with more than 100 million monthly active users or US$20 million in monthly revenue must display the model name prominently.
- If you or an affiliate run “a Model as a Service or AI Work Assistant business”, you need a separate licence from Qwen before any commercial use. The licence exempts internal use that does not make the model or its outputs available to any third party.
Running it on your own machine for your own work is fine. If you want a fully permissive licence instead, GLM-5.3-Flash is MIT-licensed, but it needs about 160GB for a 4-bit build. Building a paid assistant or API on it is the case to check with Qwen first. This is a reading of the published text, not legal advice.
FAQ
Can I run Qwen3.8-Flash-Next on a 24GB or 32GB GPU?
Not in VRAM. Even the 1-bit build is about 73GB. With llama.cpp you would need a system with 96GB or more of RAM and accept CPU-heavy inference.
Update, 16 October 2026: a dedicated engine has changed this answer. Strata, released on 24 September, runs ISTA-DASLab’s 2- and 3-bit builds of this model on a 12GB card with 48 to 64GB of RAM, by keeping the most-used experts on the GPU and computing the rest on the CPU.
Is it better than Qwen3.8 27B?
On the five coding benchmarks in Qwen’s model card it scores higher on all of them, from under a point on SWE-bench Pro (62.5 against 61.7) to 16.5 points on DeepSWE 1.1 (58.7 against 42.2). Those are Qwen’s own figures, and it needs roughly five times the memory. If you have a 24–32GB GPU, the 27B is the practical choice.
Why does Ollama’s build not fit a 128GB Mac?
Ollama’s Q4_K_M is 120GB and its MLX build 105GB, while macOS reserves part of unified memory for the system by default. The 90–94GB GGUF builds through llama.cpp or LM Studio leave room for context.
Sources
- Qwen: Qwen3.8-Flash-Next model card and Qwen Community License 1.0
- unsloth: GGUF builds (file sizes from the repository listing) and run guide
- Ollama: qwen3.8-flash-next tags; LM Studio: model page
- llama.cpp: #27742, #29761, and community measurements in discussion #27950
Updated 12 October 2026. File sizes, tags and pull-request status checked on that date. Speed figures are a community measurement credited above and a bandwidth estimate; BFWR does not run its own benchmarks.
Work out what your hardware can actually run
Free calculators and directories, sized from real model weights — before you download gigabytes or order a card:

