Hermes Agent With Ollama: Running Nous Research’s Agent on Local Models

Hermes Agent is Nous Research’s open-source personal agent. It remembers you across sessions, writes its own skills, runs scheduled jobs and answers through Telegram, Discord, Slack, WhatsApp or Signal. It has more than 256,000 GitHub stars, and it can run entirely on a model served by Ollama on your own machine.

This guide covers the two ways to connect it to Ollama, the 64K context minimum that trips most people up, which local models can actually drive it, and the security defaults to change before you give it a terminal. BFWR has not run Hermes Agent. The setup steps come from Nous Research’s and Ollama’s documentation and source code, and the memory figures are computed with our own VRAM calculator.

Quick answer

• Fastest setup: ollama launch hermes installs Hermes and points it at Ollama. Or run hermes setup and choose a custom endpoint at http://localhost:11434/v1.
• Context: Hermes refuses models with less than 64,000 tokens of context, and Ollama’s default is 4,096 on cards under 24GB. Set OLLAMA_CONTEXT_LENGTH=64000 before starting Ollama.
• Model: Nous’s Ollama guide recommends gemma4:31b. At 64K context we compute it needs about 26 GB, so a 32GB card or a Mac with 36GB or more. On 24GB, Gemma 4 26B-A4B or Qwen3.8 27B fit.
• Not Nous’s own Hermes models: the agent’s code warns that Hermes 3 and 4 “are NOT agentic”.
• Security: commands run directly on your computer by default, and the default approval mode asks your local model to judge which commands are dangerous.

What Hermes Agent is

The project is NousResearch/hermes-agent, MIT-licensed, with v0.22.5 released on 23 October 2026. Ollama’s documentation describes it as “a self-improving AI agent built by Nous Research. It features automatic skill creation, cross-session memory, and 70+ skills that it ships with by default.”

Its tools cover web search, browser automation, terminal commands, file editing, memory, delegation to subagents and scheduled tasks. Two things set it apart from coding agents like OpenCode:

• It keeps a memory of you. Notes go in ~/.hermes/memories/ and every session is stored in a searchable SQLite database at ~/.hermes/state.db.
• It writes and edits its own skills. The docs say “By default the agent writes skills freely”, including from a background review after each turn.

Installation is a script on macOS, Linux and Windows. The macOS installer supports Apple Silicon only, and the installer brings its own Python 3.14. Native Windows works without WSL, according to the README.

Setting it up with Ollama

Option 1: ollama launch hermes

ollama launch hermes

Ollama’s documentation says this installs Hermes if needed, lets you pick a model, “configures the Ollama provider, points Hermes at http://127.0.0.1:11434/v1, and sets your model as the primary”, and can connect a messaging app. In Ollama’s source, the command writes those settings to ~/.hermes/config.yaml. We found nothing in that code that changes Ollama’s context length, so you still need the context step below.

Option 2: Hermes’s own setup

curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
ollama pull gemma4:31b
hermes setup

When asked for a provider, choose Custom Endpoint. Nous’s Ollama guide gives these values:

• Base URL: http://localhost:11434/v1
• API key: “Leave empty or type `no-key` (Ollama doesn’t need one)”
• Model: the tag you pulled, such as gemma4:31b

The resulting entry in ~/.hermes/config.yaml:

model:
  default: "gemma4:31b"
  provider: "custom"
  base_url: "http://localhost:11434/v1"

LM Studio is a built-in provider (lmstudio, at http://localhost:1234/v1), and llama.cpp and vLLM work as custom endpoints. For llama.cpp, Hermes’s docs note that “`–jinja` is required for tool calling”. Their example starts llama-server with --host 0.0.0.0, which exposes it to your network; use 127.0.0.1 unless you mean to share it.

The 64K context minimum

Hermes’s provider docs state the rule: “Hermes Agent requires at least 64,000 tokens of context for agent use with tools. Smaller windows are rejected at startup”. The same limit is in its code, as MINIMUM_CONTEXT_LENGTH = 64_000.

Ollama’s own defaults are well below that. Its context-length page lists “< 24 GiB VRAM: 4k context”, 32K at 24 to 48 GiB and 256K above 48 GiB, and says tasks like “agents, and coding tools should be set to at least 64000 tokens.” (Hermes’s Ollama guide still says the default is 2,048 tokens; that figure is out of date.)

Hermes’s docs also warn: “You cannot set context length through the OpenAI-compatible API”. Set it on the Ollama server instead, in one of these ways:

# For the whole server
OLLAMA_CONTEXT_LENGTH=64000 ollama serve

# Or per model, with a Modelfile
FROM gemma4:31b
PARAMETER num_ctx 64000

For the Modelfile route, save the two lines as Modelfile, run ollama create gemma4-64k -f Modelfile, and use gemma4-64k as the model name in Hermes. Check the result with ollama ps, which shows the context in use.

Hermes also sends its system prompt and the schemas of every enabled tool with each request, so the first reply can be slow on a small GPU. Nous’s guide suggests hermes prompt-size to see the cost and hermes tools to switch off tool sets you do not use.

Which local model can drive it

Hermes does its work through tool calls, and Nous’s Ollama guide is blunt: “Models without tool-call support can only chat; they can’t take actions.” Its model table marks only one model as able to call tools reliably: “gemma4:31b is currently the best local option with tool-call support.” Its Mac guide recommends Qwen3.5-9B for getting started on smaller machines.

The guide lists gemma4:31b as needing “24+ GB” of RAM. That does not account for the 64K context Hermes requires. Computed with our VRAM calculator‘s formula (4-bit weights, 16-bit cache, plus runtime overhead):

Gemma 4 31B’s cache at 64K is 5.7 GB, against 1.4 GB for the 26B-A4B mixture-of-experts version, because the larger model has more full-attention layers. On a 24GB card, the 26B-A4B model is the closer substitute. Our Gemma 4 12B vs Qwen3.5 9B comparison covers the smaller end.

Do not use Nous’s own Hermes models

It sounds natural to run Hermes models in Hermes Agent, but the agent advises against it. Its startup code prints this warning when it detects them: “⚠ Nous Research Hermes 3 & 4 models are NOT agentic — they lack reliable tool-calling for agent workflows (delegation, cron, proactive tools).” The docs mention Hermes models only as a tool-call format that some servers parse.

Security: what the defaults allow

Nous’s SECURITY.md is unusually frank: “The only security boundary against an adversarial LLM is the operating system. Nothing inside the agent process constitutes containment — not the approval gate, not output redaction, not any pattern scanner, not any tool allowlist.” Three defaults follow from that:

• Commands run on your computer. The default terminal backend is local, which the security docs describe as “None — runs on host”. Docker, SSH and several cloud sandboxes are available instead.
• The approval mode is smart. Commands are checked against a list of dangerous patterns; matches go to an auxiliary model, which approves low-risk ones, denies dangerous ones and asks you about the rest. By default, auxiliary tasks run on your main chat model, so with Ollama, your local model is judging its own commands. Set approvals: mode: manual to be asked about every flagged command.
• Memory and skills are written without asking. Set memory.write_approval and skills.write_approval to true if you want to review them.

Some defaults are already safe. The messaging gateway denies everyone until you add an allowlist: “If no allowlists are configured and `GATEWAY_ALLOW_ALL_USERS` is not set, all users are denied.” The optional API server is off by default, listens only on 127.0.0.1:8642, and requires a key even there.

The strongest single change is to run commands in a container: set terminal.backend: docker. Note that the dangerous-command checks are then skipped, because Hermes treats the container as the boundary. Our guide to securing a local LLM explains why an agent that reads web pages needs that boundary.

Nous has published no security advisories on the repository itself, but the GitHub Advisory Database lists 10 reviewed advisories for the hermes-agent package, published between 26 May and 17 June 2026: one high, six medium and three low. The high one, a DNS rebinding flaw, was fixed in 0.16.0. Keep Hermes updated with hermes update.

What stays local

Hermes’s FAQ says: “Hermes Agent does not collect telemetry, usage data, or analytics. Your conversations, memory, and skills are stored locally in `~/.hermes/`.” Two things in Nous’s own Ollama guide can still send data out:

• Cloud fallback. The guide suggests a fallback_providers entry pointing at OpenRouter, so “only the hard tasks hit the paid API”. Leave it out if your data must stay local.
• Some tools need a service. Web search, image generation and text-to-speech run through Nous Portal’s tool gateway or your own API keys, not your local model.

FAQ

Why does Hermes say my model’s context window is too small?

Hermes requires 64,000 tokens, and Ollama defaults to 4,096 on GPUs under 24GB. Start Ollama with OLLAMA_CONTEXT_LENGTH=64000 or create a model with num_ctx 64000.

What is the best local model for Hermes Agent?

Nous recommends gemma4:31b, which needs about 26 GB at 64K context by our calculation. On a 24GB card, Gemma 4 26B-A4B or Qwen3.8 27B fit with room for the context.

Can I use Hermes 4 with Hermes Agent?

You can, but the agent warns against it: Nous says Hermes 3 and 4 are chat models without reliable tool calling.

Does Hermes Agent work offline?

The model, memory and chat can, with Ollama and no cloud fallback. Web search, image generation and speech need online services or your own keys.

Related guides and tools

• OpenClaw vs Hermes Agent on a local model
• OpenCode with a local LLM
• Claude Code with a local model
• Running Ollama as a server
• Ollama models and VRAM

Sources

• NousResearch/hermes-agent on GitHub (README, SECURITY.md, source)
• Hermes docs: local Ollama setup, providers, security, memory and FAQ
• Ollama: Hermes integration and context length
• GitHub Advisory Database

Updated 24 October 2026. Versions, defaults and advisories were read on that date; memory figures are computed from the BFWR VRAM calculator’s data.