Gemma 4 31B Locally: VRAM, Context Length and Which GPU It Fits

Gemma 4 31B needs a 24GB graphics card or a Mac with about 22GB of GPU-usable memory at 4-bit. On a 24GB card it fits with up to 32K of context at the default cache setting, or 64K if you switch to an 8-bit cache. A 16GB card cannot hold it without spilling to system RAM. It takes images as well as text, and its licence is Apache 2.0.

This guide lists the exact files and Ollama tags with their real sizes, the memory the model takes at 8K to 256K of context, which GPUs fit, the most common failure on 24GB cards, and the MTP and QAT variants. BFWR has not run this model. Sizes come from the Hugging Face API and Ollama’s registry, settings from vendor documentation, and fit figures from the code behind the BFWR tools.

Quick answer

• Default download: ollama run gemma4:31b pulls 20.39 GB: an 18.67 GB Q4_K_M model, a 1.20 GB vision projector and a 0.51 GB drafter.
• Smallest official set: Google’s QAT GGUF, gemma-4-31B_q4_0-it.gguf, at 17.65 GB for text, or 18.85 GB with gemma-4-31B-it-mmproj.gguf for images. In Ollama that is gemma4:31b-it-qat.
• 24GB card (RTX 3090, RTX 4090, RX 7900 XTX): 32K context at the default 16-bit cache, 64K at an 8-bit cache. Not 128K.
• 32GB card: 128K at the default cache, or the full 256K at an 8-bit cache.
• 48GB: the full 256K at the default cache.
• 16GB or less: it spills to system RAM. Use Gemma 4 12B instead.
• Slow on a 24GB card in Ollama? Run ollama ps. If the PROCESSOR column is not 100% GPU, see the fit problem below.
• Licence: Apache 2.0, with no revenue, user-count or territory limit.

What Gemma 4 31B is

It is the largest dense model in Google DeepMind’s Gemma 4 family. The model card gives 30.7B parameters and 60 layers, with a 256K context window; the Hugging Face API counts 31.27B once the vision encoder (about 550M) is included. Every parameter is used for every token, unlike the 26B A4B sibling, which activates 3.8B at a time, so 31B is the slower of the two.

The card lists text and images as its inputs, and video by “processing sequences of frames”. Audio is limited to the E2B, E4B and 12B models, so 31B does not take it. It also has a thinking mode and native function calling.

The files and tags, with real sizes

Sizes are decimal GB (one billion bytes), read on 9 November 2026 from the Ollama registry manifests and the Hugging Face API. Ollama’s page lists the default tag as 19GB to 20GB; 20.39 GB is the GGUF build its registry returned. Google’s repo google/gemma-4-31B-it is ungated.

How much memory each context length takes

The config.json has 60 layers in a repeating pattern of five sliding-window layers and one global layer. The 50 sliding layers only remember 1,024 tokens, so their cache stops growing almost at once: 0.78 GiB at the default 16-bit cache. The 10 global layers keep the whole context, with 4 key-value heads of 512 dimensions (num_global_key_value_heads and global_head_dim). That costs 81,920 bytes per token, or 2.5 GiB for every 32,768 tokens.

The table uses the BFWR VRAM calculator’s formula: weights, plus cache, plus 1.8 GB of engine overhead. The config sets attention_k_eq_v to true, but llama.cpp allocates separate key and value tensors per layer, so both are counted.

The totals add decimal-GB weights to binary-GiB cache, as the calculator does, so they read about 1.3 high against a card rated in binary GB. Converted, Q4_K_M with the default cache comes to 22.6 GiB at 32K, 25.1 GiB at 64K and 30.1 GiB at 128K. The 8-bit cache brings 64K down to 22.2 GiB and 256K to 29.7 GiB. Images add about 1.12 GiB for the projector on top of every figure.

That gives the limits in the quick answer. Q8_0 weights need a 48GB card at any useful context. Treat these as planning figures: one RTX 3090 owner’s ollama ps reported 18.8 GiB to 20.0 GiB for 64K with an 8-bit cache, below the 22.2 GiB here. The guide to context windows and VRAM explains the method, and the VRAM calculator takes your own numbers.

Which GPUs and Macs it fits

These verdicts come from computeFit(), the function behind the Gemma 4 31B page, across the 79 devices in the BFWR hardware list: 25 comfortable, 2 tight, 24 needing CPU offload, 28 unsupported. Comfortable means the model’s recommended 22.1 GB is available, and tight starts at its 20.3 GB minimum; both assume a modest context, so use the table above for longer ones. Speeds are estimates (memory bandwidth × 0.65 ÷ 18.8 GB of weights, or system-RAM bandwidth for offload rows) and cover generation only.

One public measurement sits close to the estimate: the RTX 3090 owner who filed Ollama issue #17099 reported 33.8 tok/s at 64K context on Ollama 0.31.1.

Every 8GB, 12GB and 16GB card in the list needs CPU offload, which is why the estimates collapse. unsloth publishes 2-bit and 3-bit files that would load on a 16GB card, but Google has published no accuracy figures for them. On 12GB or 16GB, Gemma 4 12B (7.2 GB at 4-bit in BFWR’s data) is the more defensible choice; see Gemma 4 12B vs Qwen3.5 9B.

Running it in Ollama

ollama run gemma4:31b
ollama ps

On Apple Silicon, Ollama has run supported architectures on MLX by default since 0.40.0, Gemma 4 included. Context depends on your memory, per Ollama’s docs: 4K below 24 GiB, 32K from 24 to 48 GiB, and 256K from 48 GiB. So a 24GB card starts at 32K. The same docs say that tasks “like web search, agents, and coding tools should be set to at least 64000 tokens”. On a 24GB card that takes the 8-bit cache as well (macOS and Linux shell syntax):

OLLAMA_CONTEXT_LENGTH=64000 OLLAMA_KV_CACHE_TYPE=q8_0 ollama serve

The cache type is a server-wide setting, so it applies to every model you run, and it needs Flash Attention. Ollama “uses Flash Attention automatically when the selected backend and devices support it”. Hermes Agent’s local guide uses gemma4:31b as its example and states it “requires at least 64,000 tokens”. Its Modelfile is FROM gemma4:31b plus PARAMETER num_ctx 64000, built with ollama create gemma4-64k; the Hermes Agent guide covers the rest.

The common failure: 90% GPU, and a sevenfold slowdown

Issue #17099 is the one to know if you own a 24GB card. After Ollama 0.31.2, the reporter’s gemma4:31b at 64K went from 18.8 GiB and 100% GPU to 20.0 GiB and 90% GPU, and generation fell from 33.8 to 4.7 tok/s. Ollama now reserves the projector’s memory plus 1 GiB when it decides what fits on the GPU; the source calls this mmprojOffloadHeadroom. The issue was still open on 9 November.

The workaround from Ollama maintainer dhiltgen was: “you should be able to set LLAMA_ARG_FIT_TARGET=1024 to get back to the prior behavior”. Set it in the server’s environment. A separate request, #19407, notes that setting the variable bypasses Ollama’s automatic padding, so you lose that protection against running out of GPU memory. Alternatives: use gemma4:31b-it-qat (no drafter, 1.54 GB smaller) or lower the context.

Two other errors people hit

• It hangs on long prompts with 0% GPU use. Issue #15350 (April, RTX 3090, since closed) reported this on prompts over about 3K to 4K tokens with Flash Attention on. An Ollama collaborator’s advice then was to disable it: OLLAMA_FLASH_ATTENTION=0. Ollama 0.20.5 later listed “Enable flash attention for Gemma 4 on compatible GPUs”. If you disable it, the quantized cache above is no longer available.
• this model requires macOS. Issue #16019: gemma4:31b-coding-mtp-bf16 needed the MLX runner, then macOS only (May). On a PC use the normal tags.

Running it in llama.cpp

This command is assembled from flags documented in the llama-server README, with file names from unsloth’s repo, for 64K with an 8-bit cache and vision:

llama-server -m gemma-4-31B-it-Q4_K_M.gguf \
  --mmproj mmproj-BF16.gguf \
  -c 65536 -fa on -ctk q8_0 -ctv q8_0

MTP drafters: faster output, same text

Multi-Token Prediction pairs the model with a small drafter. Google’s card says that “the draft model predicts several tokens ahead, which the target model then verifies in parallel”, giving speedups “up to 3x” while “guaranteeing the exact same quality as standard generation”. Ollama’s 0.23.1 notes claim “over a 2x speed increase for the Gemma 4 31B model on coding tasks” on Macs, and 0.31.1 claims “nearly 90% faster on average across a coding-agent benchmark”. unsloth reports about 1.4× to 2.2× for GGUFs. All are vendor figures measuring different things.

Google’s google/gemma-4-31B-it-assistant is 0.94 GB and the 8-bit GGUF 0.51 GB. Ollama’s default tag already carries a drafter, and its source passes it to llama-server with --spec-draft-n-max 4. For llama.cpp, unsloth documents this:

llama-server -hf unsloth/gemma-4-31B-it-GGUF --spec-type draft-mtp --spec-draft-n-max 4

unsloth advises to “plan for ~2 GB additional RAM/VRAM headroom”, which matters on a 24GB card. Drop the drafter first if the model will not stay on the GPU.

QAT builds: Google’s own 4-bit

Google’s card claims Quantization-Aware Training allows “preserving similar quality to bfloat16 while dramatically reducing the memory requirements”, without giving a figure on that card. For 31B there are three forms: the Q4_0 GGUF (17.65 GB), a w4a16 compressed-tensors build for vLLM (23.27 GB), and unquantized QAT weights. Google also warns that with a drafter “the assistant model must also be a QAT checkpoint with the same precision”. So do not pair the QAT GGUF with a standard drafter.

Gemma 4 31B or Qwen3.8 27B?

Qwen3.8 27B is its closest rival on this site: also dense, also 256K, also Apache 2.0. This compares memory only, using the same calculator; BFWR has no quality comparison to offer.

Qwen has 48 linear-attention layers and 16 global ones. The ten devices it adds are 32GB laptops with 19.2 GB usable, where Qwen is tight and Gemma does not fit. On a 24GB card Qwen holds 64K at the default cache (21.4 GiB) where Gemma needs the 8-bit one. See how to run Qwen3.8 27B locally.

Licence

Hugging Face tags google/gemma-4-31B-it as apache-2.0, and the licence link on its card resolves to Google’s Apache License 2.0 page. Gemma 3’s repo, by contrast, is tagged gemma. The text grants a “perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license” and has no revenue, user-count or territory limit. Commercial use and fine-tuned derivatives are allowed.

The conditions are the standard ones. If you redistribute, “You must give any other recipients of the Work or Derivative Works a copy of this License”, and modified files must carry a notice. Patent rights end if you sue claiming the work infringes a patent, and the licence grants no rights to Google’s trademarks. This is a reading of the published text, not legal advice.

Keep it local

“Ollama binds 127.0.0.1 port 11434 by default”, and llama-server defaults to 127.0.0.1 (port 9931), with no API key set unless you pass --api-key. Leave both bound to 127.0.0.1. The model supports function calling, so anything that can reach the port can act through whatever tools you have connected.

FAQ

Does Gemma 4 31B fit on a 16GB GPU?

Not at 4-bit. The smallest 4-bit file is 17.65 GB, so part of the model moves to system RAM, and BFWR’s estimate drops to about 3.3 t/s even on an RTX 5080. The 2-bit and 3-bit files fit, with no accuracy data from Google.

Can a 24GB card run the full 256K context?

No. The 8-bit cache brings it to 29.7 GiB, which needs a 32GB card, and the default 16-bit cache to 40.1 GiB, which needs 48GB. On 24GB plan for 32K, or 64K with the 8-bit cache.

Why is the Ollama download larger than the GGUF?

The default tag bundles a 1.20 GB vision projector and a 0.51 GB MTP drafter with the 18.67 GB model. gemma4:31b-it-qat leaves out the drafter and uses Google’s QAT weights, for 18.85 GB.

Related guides and tools

• LLM context windows and VRAM
• Hermes Agent with Ollama
• How to run Qwen3.8 27B locally
• Gemma 4 12B vs Qwen3.5 9B
• Gemma 4 31B model page
• 24GB VRAM tier
• VRAM calculator

Sources

• Gemma 4 31B model card and config.json
• Google’s MTP assistant card and QAT GGUF card
• unsloth GGUF files, ggml-org GGUF files, unsloth MTP guide
• Google’s Apache License 2.0 page for Gemma
• Ollama gemma4 tags, context length docs, FAQ
• Ollama issues #17099, #19407, #15350, #16019
• Ollama release notes 0.20.5, 0.23.1, 0.31.1; source llama_server.go
• llama-server README, speculative decoding docs, KV cache source
• Hermes Agent local Ollama guide

Updated 9 November 2026. File sizes, tags, licence, documentation and issue status were read on that date; fit and memory figures were computed from the BFWR data files on the same day.