Kimi K3.1 Pro Locally: 2.8 Trillion Parameters and the Memory It Takes

Can you run Kimi K3.1 Pro locally? Not in full on any single machine in the BFWR hardware database. The released checkpoint is 1,561 GB, while the biggest machine we list, a 512GB Mac Studio, gives its GPU 384 GB by default. The smallest file that keeps every expert is 580 GB. What people actually run are compromises: a pruned 336 GB build on a 512GB Mac at about 6 tokens per second, low-bit or full builds streamed from NVMe at 2 to 5 tokens per second, or a cluster of four 512GB Macs.

This guide gives the real size of every build, which machines can hold them, the speeds their publishers report, the exact commands, and what the Kimi K3.1 Pro licence allows. It also examines a “12GB” version on Hugging Face. BFWR has not run this model. File sizes come from Hugging Face file listings, commands and measured speeds come from each publisher’s own card, and hardware fits are arithmetic against the BFWR hardware database.

Quick answer

• Size: 2.78 trillion parameters, 104B active per token. The official download, moonshotai/Kimi-K3.1-Pro, is 1,561.3 GB in 96 safetensors shards.
• Smallest workable combination on one Mac: pipenetwork/Kimi-K3.1-Pro-REAP80-code-MLX-mxfp4-q6, 336.0 GB, on a 512GB Mac Studio. It has only 20% of the experts, so it is a different and weaker model.
• Smallest complete files: 580.4 GB (bloomery only) and 592.7 GB (stock llama.cpp).
• Software: llama.cpp from build b12180 (text only). vLLM needs a nightly build or Docker image. mlx-lm has no kimi_k31 loader yet. Ollama offers only kimi-k3.1-pro:cloud.
• The 12GB repository: not Kimi K3.1 Pro in any meaningful sense, and its code cannot run as published.
• Licence: free for commercial use, with conditions only at large scale (US$20 million revenue for Model-as-a-Service businesses; a display rule above 100 million monthly users).

What it is, and what changed from Kimi K3

Moonshot calls it “our most capable 3T-class model to date”. From the card and config.json: 93 layers, 896 routed experts per layer with 16 active per token plus 2 shared, a 1,048,576-token context, and text and image input. Thinking is set with reasoning_effort (low, high, max), and the default is max.

It is not a new base model. The card says it was “Post-trained from Kimi K3 with large-scale joint reinforcement learning across hundreds of downstream domains”, and the one architectural change is in a footnote: “Compared with Kimi K3, we replace the full-attention Gated MLA layers with Gated DeepSeek Sparse Attention (Gated DSA) layers for more efficient long-context modeling.”

For memory planning, K3.1 Pro and K3 are the same size: identical routed-expert totals, plus 178.5M extra parameters for the sparse-attention indexer. K3 is 1,560.9 GB; K3.1 Pro is 1,561.3 GB.

Moonshot’s benchmark claims are its own. Its blog puts K3.1 Pro at 65.7 on Terminal-Bench 4.0, against 66.4 for Claude Opus 5.5 and 58.2 for GPT-6 Astra, and reports 61.2 against K3’s 55.95 on its internal long-horizon coding test. The same card’s table also shows benchmarks where it trails, such as 31.5 on Program Bench against 65.0 for Opus 5.5.

How big are the files?

Sizes are in GB (one billion bytes), summed from each repository’s file listing on 9 November. The official release is already quantized: the routed experts are MXFP4 (1,446.5 GB) and everything else is BF16 (114.8 GB).

Budget extra memory for the cache: KarLexAI reports about 612 GB of GPU memory in use for its 592.7 GB file at a 131K-token context.

Pruned builds are not the same model. vLLM’s pruned75 card warns: “Expect a large quality regression relative to the source model.” It scored 43.67% on GSM8K, where vLLM’s recipe reports 96.2% for the full model. REAP80 is calibrated on code-heavy text, but pipenetwork says “This is an aggressive prune” and reports a chat test where q6 got 17 times 23 wrong. pipenetwork calls q4 “the fast, lossy tier” and recommends q6.

Which machines can hold it

Against GPU-usable memory in the BFWR hardware database:

• 512GB Mac Studio, M5 Ultra or M3 Ultra (384 GB by default): only the REAP80 builds at 322 to 350 GB. pipenetwork says to raise the GPU limit to 480 GB; our Mac VRAM optimizer explains the setting.
• 256GB Mac Studio (192 GB), 192GB M2 Ultra (144 GB): no published route. The smallest build is 322 GB.
• 128GB machines (DGX Spark and RTX Spark at 110 GB, M5 Max and Strix Halo at 96 GB): no. The BF16 non-expert weights alone are 114.8 GB.

Memory is not the only wall. Each token reads 16 of 896 experts (25.8 GB) plus the 114.8 GB of everything else, about 140 GB. With the formula from our tokens-per-second guide (bandwidth × 0.65 ÷ active GB), a hypothetical M5 Ultra with enough memory would top out near 5.5 tokens per second on the official weights. That is an estimate, not a measurement.

Published speeds

These are the publishers’ own figures, with different prompts and settings, so read them as orders of magnitude.

The model thinks by default. KarLexAI’s run averaged 1,419 tokens per MATH-500 answer and 9,867 per AIME problem, which at 6 tokens per second is about four minutes and more than 27. And as pipenetwork puts it: “Pruning buys memory, not speed”. For REAP80 q6 on the M5 Ultra’s 1,200 GB/s the same formula gives an estimated 10.7 tokens per second.

The 12GB version is not Kimi K3.1 Pro

shashanth-27xA9/Kimi-K3.1-Pro-SuperExpert-12GB claims the model was “compressed to an ultra-portable 12GB footprint” for machines with 16GB of RAM. It had 1,725 downloads on 9 November. Its files do not substantiate the claim:

• Different architecture. Its config.json sets 36 layers, and modeling_kimi_k31_superexpert.py defines plain attention plus one feed-forward block per layer: no experts, no router, no KDA or DSA attention.
• The code cannot run as published. It reshapes a 7,168-wide hidden state into 96 heads of 7168 // 96 = 74 dimensions, and 96 × 74 is 7,104.
• The GGUF is empty. kimi-k3.1-pro-superexpert-12gb-q4_k_m.gguf is 623 bytes of metadata with no tensors, and that is what the card’s ollama run command gets.
• The weight bytes look random. Two 2 MB samples from shards 2 and 10 showed 7.9999 bits of entropy per byte, with all 256 values almost equally common. A 2 MB sample of Moonshot’s expert weights showed 7.51 bits and an uneven distribution. That is a statistical observation, not proof of origin.
• No evidence behind the scores. The “verification scorecard” is a 263-byte JSON file. The card gives the full model 88.1% on GSM8K; vLLM’s recipe reports 96.2%.

Two reports on the repository’s discussion page already call it fake. Do not run its Python: the quickstart uses trust_remote_code=True, which executes the repository’s code on your machine.

How to run each route

llama.cpp (8× H100, or GPU plus lots of system RAM)

Support landed in pull request #31779, merged 6 November; the first release containing it is b12180. KarLexAI’s card gives these commands for the 592.7 GB file, which is “Text only: the vision encoder is not included.”

hf download KarLexAI/Kimi-K3.1-Pro-GGUF --include "KLX-IQ1_S/*" --local-dir .
llama-server -m KLX-IQ1_S/Kimi-K3.1-Pro-KLX-IQ1_S-00001-of-00014.gguf -ngl 999 --fit off -c 131072 -np 4 --jinja --temp 1.0 --top-p 0.95

With less GPU memory, replace -ngl 999 --fit off with --fit on, and llama.cpp decides what stays in system RAM. --n-cpu-moe N keeps the first N layers’ experts on the CPU, and --moe-cache-mib N adds a GPU cache for them. You still need about 600 GB across VRAM and RAM.

Mac Studio 512GB, pruned (macOS only)

From the REAP80 q6 card, using its bundled loader:

sudo sysctl iogpu.wired_limit_mb=480000
pip install mlx-lm

python - <<'PY'
import os, shutil, mlx_lm
from huggingface_hub import hf_hub_download
dst = os.path.join(os.path.dirname(mlx_lm.__file__), "models", "kimi_k31.py")
shutil.copy(hf_hub_download("pipenetwork/Kimi-K3.1-Pro-REAP80-code-MLX-mxfp4-q6", "kimi_k31.py"), dst)
PY

mlx_lm.generate --model pipenetwork/Kimi-K3.1-Pro-REAP80-code-MLX-mxfp4-q6 --trust-remote-code \
    --max-tokens 256 --prompt "Write a Python function that merges overlapping intervals."

inferencerlabs/Kimi-K3.1-Pro-MLXse is the other Mac route. It “offloads the experts to storage”: 114.7 GB of core weights plus 1,446.5 GB of experts on disk, at about 0.37 tokens per second, or 45 minutes for a 1,000-token reply.

Workstation with NVMe streaming (Linux, CUDA)

The midagedev card says “Only bloomery loads this file.” It needs bloomery 0.3.13 or later, which caps context at 2,048 tokens:

hf download midagedev/Kimi-K3.1-Pro-GGUF --local-dir /models/Kimi-K3.1-Pro-IQ1_S
BLOOMERY_HOST_ROOM=232G bloomery-serve --model kimik3 \
  -m /models/Kimi-K3.1-Pro-IQ1_S/Kimi-K3.1-Pro-IQ1_S-00001-of-00031.gguf \
  --place bp --ctx-size 2048

Servers: vLLM and SGLang

Moonshot recommends vLLM, SGLang and TokenSpeed. vLLM’s recipe starts at 8 GPUs (“either two GB300 trays or one B300 node”), and states “Requires vLLM >= 0.34.0 (not released yet — nightly wheel)”, with the image vllm/vllm-openai:kimi-k31. SGLang uses the kimi-k3.1-day-0 branch or the image lmsysorg/sglang:dev-kimi-k31-pro.

What goes wrong first

• unknown model architecture: 'kimi-k31': your llama.cpp is older than b12180 (issue #31778 tracked the request). Update it. DevQuasar’s GGUFs came from a fork before the merge, so we could not confirm that mainline loads them.
• ollama run kimi-k3.1 finds nothing: the library name is kimi-k3.1-pro, and its only tag is kimi-k3.1-pro:cloud, which runs on Ollama’s servers.
• mlx-lm cannot load it: mlx_lm/models/kimi_k31.py is not in the main branch, and issue #2042 on adding it was open on 9 November. Copy the bundled loader as shown above.
• Nothing on port 8080: KarLexAI’s card says to open localhost:8080, but current llama-server defaults to port 9931. Open that, or pass --port 8080.
• Broken multi-turn replies: the model “requires the complete assistant message returned by the API to be passed back to” the message list, including reasoning_content and tool_calls.

Security defaults

llama-server binds to 127.0.0.1 by default. vLLM’s recipe sets neither --host nor an API key, and DaoCloud’s command passes --host 0.0.0.0; use --host 127.0.0.1 and put authentication in front. --trust-remote-code runs Python from the repository and is required for the official model and the REAP80 builds, so read those files first.

The licence: MIT-style with two large-scale conditions

The licence grants the rights to “use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software” and to “run, deploy, fine-tune, or otherwise modify the Software and create derivative works from it”. Apart from the model name, its text matches the Kimi K3 licence.

• Notice: the copyright and permission notice must travel “in all copies or substantial portions of the Software”, which covers GGUF and pruned builds.
• Model as a Service: if you or an affiliate run such a business “and the aggregate revenue of the Licensee and its affiliates exceeds 20 million US dollars (or the equivalent in other currencies) in total over any consecutive 12 months”, you need a separate agreement “before using the Software or its derivative works for any commercial purpose”. On this text the test is total group revenue, not revenue from that one service.
• What counts: giving a third party inference or fine-tuning with “meaningful control over the inputs, parameters, or training data”. Products with model features “solely embedded within specific features or harnesses” are excluded.
• Display: a commercial product with more than 100 million monthly active users, or more than US$20 million in monthly revenue, must show “Kimi K3.1-Pro” prominently.
• Exempt: internal use, meaning “any use that does not make the Software, its outputs, or its underlying capabilities available to third parties”.

There is no territorial exclusion, no acceptable-use policy and no separate outputs clause, and “meaningful control” is undefined. This is a reading of the published text, not legal advice.

Is it worth running locally?

Moonshot’s pricing page on 9 November listed kimi-k3.1-pro at $2.00 per million input tokens, $10.00 output and $0.20 cached input. The older K3 is $3.00 and $15.00.

At 6 tokens per second, a 512GB Mac needs about 46 hours to generate one million tokens from a pruned model. The API charges $10 for that from the full one. The local case is data that cannot leave your machine. Our API pricing calculator runs the comparison for your volumes.

These large open models fit far better: GLM-5.3-Flash (321B, MIT), Qwen3.8-Flash-Next (180B), which Strata runs on a 12GB card with enough system RAM, and Muse Spark Open (816B).

FAQ

Can I run Kimi K3.1 Pro on a 12GB or 24GB graphics card?

No. The non-expert weights alone are 114.8 GB, and the 12GB repository on Hugging Face does not hold the real model. Argodrive DS4 was tested on a 96 GB card with 1.45 TB of experts on NVMe at 1.5 to 3 tokens per second.

Is Kimi K3.1 Pro on Ollama?

Only as kimi-k3.1-pro:cloud, at the same price as Moonshot’s API. There is no local tag.

Related guides and tools

• Mistral Large 4 locally, a 1.05T model that does fit a 512GB Mac at 2-bit
• Meta Muse Spark Open locally
• DeepSeek V4.1-Flash-1017 locally
• Strata: Qwen3.8-Flash-Next on a 12GB graphics card
• llama.cpp model commands
• Open model directory

Sources

• Kimi K3.1 Pro on Hugging Face (model card, config.json), its licence and Kimi K3
• Moonshot tech blog and API pricing
• llama.cpp pull request #31779, issue #31778 and the llama-server README
• vLLM recipe and SGLang cookbook
• KarLexAI GGUF and its benchmarks, midagedev GGUF, DevQuasar GGUF, danielhanchen preliminary GGUF
• pipenetwork REAP80 q6, q4 and q8; vLLM pruned75; DaoCloud pruned75
• AinsleyMarkov 2-bit UVMAX, EXL3 2-bit, MLXse, Argodrive DS4
• SuperExpert 12GB repository (files read directly) and its discussion page
• Ollama tags for kimi-k3.1-pro and mlx-lm issue #2042

Updated 9 November 2026. File listings, parameter counts, the licence text, API prices, the Ollama listing, and llama.cpp, vLLM and mlx-lm support status were read on that date. Speed figures are the named publishers’ own; the two labelled estimates are computed from the BFWR hardware database.