Pi Coding Agent With a Local Model: Ollama, llama.cpp and Context Setup

Pi runs on a local model through Ollama, LM Studio or a llama.cpp router, and it needs Node.js 22.19 or newer plus a model that supports tool calls. A 12GB card can hold Gemma 4 12B at a 64K context (a 7.7 GB download, 10.3 GB in memory), and 24GB fits Qwen3.8 27B. The catch is Ollama: Pi cannot set the context size there, so unless you raise it on the Ollama side, a card under 24 GiB runs at 4K while Pi believes it has far more.

This guide gives the exact models.json for Ollama and LM Studio, the llama.cpp router route, memory per model, what Pi does without asking, and the failures people report. BFWR has not run Pi. The steps come from Pi’s and Ollama’s documentation and source code, and the memory figures are computed from our VRAM calculator’s data.

Quick answer

• Fastest route: ollama launch pi. It installs Pi and writes the config, but it also adds a web search package and does not fix the context.
• Context on Ollama: start the server with OLLAMA_CONTEXT_LENGTH=65536 ollama serve, then set "contextWindow": 65536 in ~/.pi/agent/models.json. The two numbers must match.
• Context on llama.cpp: set it with -c on the router; Pi reads the real value from the server.
• If you leave contextWindow out, Pi assumes 128,000 tokens.
• Permissions: read, bash, edit and write are on from the start, and there is no approval prompt and no sandbox. Use a container, or start with pi --tools read,grep,find,ls.
• Network: prompts stay local with a local model, but Pi reports installs and checks for updates unless you use PI_TELEMETRY=0 and --offline.

What Pi is

Pi is a terminal coding agent from earendil-works/pi on GitHub (badlogic/pi-mono redirects there). Its README says it “ships with powerful defaults but skips features like sub-agents and plan mode”, and leaves those to extensions. Read on 9 November 2026, the repo has about 120,900 stars, 2,072 commits on the main branch in the previous 90 days, a push on 8 November, and is not archived. The latest release is v1.5.0, from 2 November.

The licence is the standard MIT text, so commercial use and modification are allowed if the notice is kept. That is our reading of the published file, not legal advice.

Install it with the official script, or from npm:

# macOS and Linux
curl -fsSL https://pi.dev/install.sh | sh

# Any platform with Node.js 22.19+
npm install -g --ignore-scripts @earendil-works/pi-coding-agent

Older guides use the @mariozechner/pi-coding-agent package name, which is the previous one.

Setting it up with Ollama

The one-command route

ollama launch pi --model qwen3.8:27b

# Write the config without starting Pi
ollama launch pi --config

Ollama’s Pi page says this “installs Pi if needed, configures Ollama as a provider including web tools”. The launcher’s source shows two more things. It installs the package @ollama/pi-web-search, whose tools use Ollama’s web search and fetch APIs, so those calls leave your machine. It also writes each model’s contextWindow as the model’s trained maximum, which is 262,144 for Qwen3.8 27B according to our calculator’s data, and that is rarely what Ollama has loaded.

The manual route

Create ~/.pi/agent/models.json:

{
  "$schema": "https://pi.dev/schemas/models.schema.json",
  "providers": {
    "ollama": {
      "baseUrl": "http://localhost:11434/v1",
      "api": "openai-completions",
      "apiKey": "ollama",
      "models": [
        { "id": "qwen3.8:27b", "contextWindow": 65536, "reasoning": true }
      ]
    }
  }
}

Then set the default in ~/.pi/agent/settings.json:

{
  "defaultProvider": "ollama",
  "defaultModel": "qwen3.8:27b"
}

The apiKey looks pointless but is required. Pi’s docs say “The dummy key makes the model available to Pi; Ollama ignores it.” Without it the model will not appear in /model. Set "reasoning": true only for models that Ollama’s library marks with thinking; qwen3-coder:30b is not one of them. The launcher replaces only the entries it created, so a hand-written entry survives a later ollama launch pi.

The Ollama context trap

Pi talks to Ollama through its OpenAI-compatible endpoint, and Ollama’s documentation is plain about the limit: “The OpenAI API does not have a way of setting the context size for a model.” Pi’s source contains no num_ctx setting at all. Compare Cline, which uses Ollama’s native API and sends a context size with every request.

So contextWindow in models.json does not change what Ollama loads. It only tells Pi what to assume. Ollama picks the real size from your GPU memory:

Ollama’s scheduler can also step an automatic size down if the model does not fit. When the conversation outgrows that size, Ollama’s chat code “truncates any messages that exceed the context window of the model”, removing the oldest messages while keeping system messages and the latest one (MLX models on Apple Silicon skip this step). Pi does not get an error. A comment in Pi’s overflow detector says that silent truncation cannot be caught because “we do not know the expected token count”. The agent starts forgetting what it read a few turns ago, and Pi’s compaction does not start, because the limit it is waiting for is far above the real one.

Ollama’s Pi page gives no context figure, although its pages for OpenCode, Claude Code and Cline do. Its general guidance applies: “Tasks which require large context like web search, agents, and coding tools should be set to at least 64000 tokens.”

There are two fixes. The first sets a server-wide default (macOS and Linux shell shown; the Ollama app has a slider in its settings):

OLLAMA_CONTEXT_LENGTH=65536 ollama serve

The second bakes the size into a named model, which is better if you switch between sizes:

# Modelfile
FROM qwen3.8:27b
PARAMETER num_ctx 65536

ollama create qwen3.8-64k -f Modelfile

Use qwen3.8-64k as the id in models.json. Either way, confirm it with ollama ps, whose CONTEXT column shows the size actually allocated, and whose PROCESSOR column should read 100% GPU.

Setting it up with LM Studio

LM Studio is a compatible endpoint, not a built-in provider. Start the server (default port 1234), load the model at the context you want, and add a provider block alongside any others in models.json:

lms load qwen/qwen3.8-27b --context-length 65536

{
  "providers": {
    "lmstudio": {
      "baseUrl": "http://localhost:1234/v1",
      "api": "openai-completions",
      "apiKey": "lmstudio",
      "models": [
        { "id": "qwen/qwen3.8-27b", "contextWindow": 65536, "reasoning": true }
      ]
    }
  }
}

The id must match the model key LM Studio reports, and contextWindow must match the load setting. LM Studio is safer than Ollama when you overrun it: Pi’s overflow detector recognises its “greater than the context length” error and can compact and retry.

Setting it up with llama.cpp

This is Pi’s built-in local route. Start llama serve with no model argument, which puts it in router mode, then connect from inside Pi:

llama serve --models-dir ~/models --no-models-autoload -ngl all -c 32768

/login llama.cpp
/llama
/model

/login llama.cpp stores the server address, /llama loads a GGUF (it can also search and download from Hugging Face), and /model selects the loaded one. With --no-models-autoload, a model missing from /model has not been loaded through /llama yet. To go to 64K, use -c 65536; -c 0 uses the model’s full window and can need far more memory. Pi’s docs add: “Context is shared across parallel server slots.”

Type the URL at the login prompt. Pi’s docs give the default as http://127.0.0.1:9931, which is where llama serve listens. But the v1.5.0 source falls back to http://127.0.0.1:8080 if you press Enter, so a blank answer points at the wrong port. Enter http://127.0.0.1:9931, or set LLAMA_BASE_URL first.

Pi reads the live context size from the server (n_ctx), so you do not maintain a second number by hand.

How much memory each model needs

These figures use our VRAM calculator‘s formula and data: Q4_K_M weights, a 16-bit cache, plus 1.8 GB of runtime overhead. The Gemma 4 rows use the calculator’s separate head counts for global layers, which keeps their cache small. All tags below carry Ollama’s tools badge.

The smallest workable setup in this table is Gemma 4 12B on a 12GB card. The right-hand column shows roughly what Pi’s default 128,000 assumption would really cost. Liquid AI’s model card says LFM3-1.7B was “post-trained for agentic harnesses such as Hermes Agent, Pi, and OpenClaw”; that is the vendor’s claim.

Compaction on a small window

Pi starts compacting when the context passes contextWindow minus reserveTokens (default 16,384), and it tries to keep the most recent keepRecentTokens (default 20,000). At 64K, that means compaction starts at 49,152 tokens, which works. At 32K, it starts at 16,384, but Pi wants to keep 20,000, which is more than the whole conversation at that point. In the source, compaction then finds nothing to summarise until the history passes 20,000. A per-model override in settings.json fixes this. These values are our suggested starting point, not a figure from Pi’s docs:

{
  "compaction": {
    "modelOverrides": {
      "ollama/qwen3.8:27b": { "reserveTokens": 8192, "keepRecentTokens": 8192 }
    }
  }
}

What Pi does without asking

Pi’s security page says it “does not ask for approval before every tool call”, and the README is more direct: “Pi does not include a built-in permission system for restricting filesystem, process, network, or credential access.” The four default tools run with your user account’s permissions. A command can reach any path you can.

• Project trust is the one prompt you will see. Pi asks before loading a project’s .pi/settings.json, .pi/mcp.json or .pi/extensions. It does not limit tools, and AGENTS.md and CLAUDE.md load even if you decline.
• Extensions and packages run inside the Pi process. The docs warn: “Extensions run with your full system permissions and can execute arbitrary code.” Read a package’s source before pi install.
• Isolation is left to you. Of running Pi entirely inside a container, virtual machine or sandbox, the docs say “This is usually the strongest practical option.”

A read-only start is sensible: pi --tools read,grep,find,ls. For a light guard, the repo ships examples/extensions/permission-gate.ts. Copied to ~/.pi/agent/extensions/, it asks before commands matching rm -rf, sudo or chmod 777. That is a pattern list, not a boundary. If you use llama serve, keep its default 127.0.0.1 bind. See also securing a local LLM.

Telemetry and network calls

With a local model, prompts go only to your server. Pi still makes a few calls of its own. On a fresh install and after each update it sends its version to pi.dev/api/report-install; the setting enableInstallTelemetry (default on) is described as “Enable anonymous install/update reporting and selected provider attribution headers.” It also checks pi.dev for new versions, refreshes its model catalog, and downloads fd and ripgrep if they are missing. To stop all of it, set PI_TELEMETRY=0 and start with pi --offline (or set PI_OFFLINE=1).

How it compares with other agents

Pi is the most stripped-down of the four. It suits people who will write an extension or run it in a container.

Common failures

When reading Pi’s issue tracker, note that “New issues and PRs from new contributors are closed automatically”, so closed does not always mean fixed.

• The agent forgets files it just read (Ollama): silent truncation, covered above. Check the CONTEXT column in ollama ps.
• Repeated context errors after compaction: contextWindow is set higher than the server allows. A Pi maintainer’s reply in issue 11387 says “A window declared larger than the server allows makes proactive compaction unreachable”. Set it to the served size.
• Context size has been exceeded. on llama.cpp: this message is not recognised as an overflow, so auto-compaction does not start (issue 10885, open). Raise -c, avoid sharing it across parallel slots, or run /compact.
• terminated errors with qwen3.8:27b on Ollama: issue 9216, open, traces them to a change between versions 0.84.4 and 0.85.1. A contributor’s synthetic checks did not reproduce it, while a second user reports that going back to 0.84.4 helped.
• The model pastes a file into chat instead of writing it: reported with Qwen3.5 35B on Ollama in issue 10540. The suggested workaround is a short --append-system-prompt telling it to write files to disk.
• Slow first responses get cut off: httpIdleTimeoutMs defaults to 300,000 ms. On slow hardware with a large context, raise it in settings.json.
• The model is missing from /model: the provider has no apiKey, or on llama.cpp the model is not loaded.

FAQ

Does Pi work with Ollama’s default context?

Only briefly on a card under 24 GiB, where the default is 4K. Pi cannot change it through the endpoint it uses, so set OLLAMA_CONTEXT_LENGTH or build a Modelfile with num_ctx.

What is the best local model for Pi?

Pi’s docs do not name one. Any tool-capable model works, so choose by memory: Gemma 4 12B on 12GB, gpt-oss-20b on 20GB, Qwen3.8 27B on 24GB at 64K. Our guide to local coding models compares them by licence too.

Related guides and tools

• Cline with a local model
• OpenCode with a local LLM
• Claude Code with a local model
• LLM context windows and VRAM
• Running Ollama as a server
• VRAM calculator
• Ollama models and VRAM
• llama.cpp models

Sources

• earendil-works/pi on GitHub: LICENSE, models, models configuration, llama.cpp, compaction, security, settings, overflow detection and the llama.cpp provider in v1.5.0
• Ollama: Pi, context length, OpenAI compatibility, launcher source and prompt truncation
• llama.cpp: llama serve
• LM Studio: lms load
• LFM3-1.7B model card

Updated 9 November 2026. Pi v1.5.0, its docs and issues, Ollama’s docs and source, and Ollama library tags were read on that date; memory figures are computed from the BFWR VRAM calculator’s data.