How to Run Ollama as a Server (and in Docker): Network Access, Security and Tuning

Ollama is already a server. The moment it is running, it answers HTTP requests on 127.0.0.1:11434 — the desktop app, the command line and every tool that “connects to Ollama” are all just clients of that API. So the real questions are narrower: how to keep it running as a proper service, how to run it in Docker, and how to let other machines reach it without opening a model server to your whole network. The same binding question comes up when you put an automation platform in front of it — see self-hosting n8n with local AI, where the container holds credentials for everything it touches.

This guide covers all three. Every command and default below is taken from Ollama’s own documentation and Docker’s documentation, checked on 14 September 2026 against Ollama v0.34.0 (released 5 September 2026). The bind, concurrency and keep-alive defaults were re-checked on 10 October 2026 against v0.40.2 and are unchanged.

Quick answer

• Default address: 127.0.0.1:11434 — reachable from the same machine only.
• Linux service: run it under systemd and set options with sudo systemctl edit ollama.
• Docker: ollama/ollama, with --gpus=all for NVIDIA or the :rocm tag for AMD. There is no GPU acceleration in Docker Desktop on macOS.
• Other machines: set OLLAMA_HOST=0.0.0.0:11434, or better, keep Ollama on loopback and put a reverse proxy in front.
• The trap: Docker’s -p 11434:11434 publishes the port to the outside world, not just your machine.

What “Ollama server” actually means

Ollama runs an HTTP API. Its native endpoints include /api/generate, /api/chat and /api/tags (your downloaded models), and it also exposes OpenAI-compatible endpoints — /v1/chat/completions, /v1/models, /v1/embeddings and /v1/responses — which is why so many apps built for OpenAI can point at it with a one-line change. For a framework rather than a raw client, see Ollama with LangChain in Python.

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "llama3.2", "messages": [{"role": "user", "content": "Hello!"}]}'

If you have not picked a runner yet, our comparison of local LLM runners covers where Ollama fits against LM Studio, llama.cpp and vLLM, and the Ollama hub lists every model’s memory cost and pull command. For a server you can carry as a single file, our llamafile guide does the same job from one executable, with an OpenAI-compatible API on port 8080.

Option 1: Run Ollama as a Linux service

Ollama’s Linux documentation recommends running it as a systemd startup service. Install it, then start and check the service:

curl -fsSL https://ollama.com/install.sh | sh
sudo systemctl start ollama
sudo systemctl status ollama

Ollama is configured through environment variables, and on a systemd install they belong in the service, not your shell profile. Open an override with:

sudo systemctl edit ollama.service

and add one Environment line per variable under [Service]:

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"

Then reload and restart:

sudo systemctl daemon-reload
sudo systemctl restart ollama

Logs go to the journal (journalctl -e -u ollama), and on Linux downloaded models live in /usr/share/ollama/.ollama/models.

On macOS and Windows

The desktop app reads environment variables differently. On macOS, use launchctl setenv OLLAMA_HOST "0.0.0.0:11434" and restart the app. On Windows, quit Ollama from the task bar, add the variable under Edit environment variables for your account, and start it again from the Start menu.

Option 2: Run Ollama in Docker

The official image is ollama/ollama — over 174 million pulls on Docker Hub. The named volume is what keeps your downloaded models when the container is replaced; leave it out and every upgrade starts from an empty model list.

CPU only

docker run -d -v ollama:/root/.ollama -p 127.0.0.1:11434:11434 --name ollama ollama/ollama

NVIDIA GPU

Install the NVIDIA Container Toolkit first, configure Docker to use it with sudo nvidia-ctk runtime configure --runtime=docker and restart Docker, then add --gpus=all:

docker run -d --gpus=all -v ollama:/root/.ollama -p 127.0.0.1:11434:11434 --name ollama ollama/ollama

AMD GPU

Use the rocm tag and pass the GPU devices through:

docker run -d --device /dev/kfd --device /dev/dri -v ollama:/root/.ollama -p 127.0.0.1:11434:11434 --name ollama ollama/ollama:rocm

Vulkan support is also bundled in the standard image and is enabled when the container can see the GPU devices.

Once the container is up, pull and run a model inside it:

docker exec -it ollama ollama run llama3.2

On a Mac: Ollama’s FAQ is explicit that GPU acceleration is not available in Docker Desktop on macOS, because there is no GPU passthrough. On Apple Silicon, run the native app instead.

The Docker port trap

Ollama’s own Docker examples use -p 11434:11434. Docker’s documentation is blunt about what that does: publishing container ports “is insecure by default” — the port becomes available “not only to the Docker host, but to the outside world as well.” That is why every command above uses -p 127.0.0.1:11434:11434 instead, which Docker documents as restricting the port to the host.

One caveat from the same page: in Docker releases older than 28.0.0, other machines on the same network segment could still reach ports published to localhost. Check docker version before relying on it.

Reaching Ollama from another machine

There are two ways, and the second is the one to prefer.

1. Bind Ollama to your network

Setting OLLAMA_HOST=0.0.0.0:11434 (as above) makes Ollama listen on every interface. It is simple, and it means anything that can reach that port can use your models. Restrict it with your firewall to the machines that should have access.

2. Keep Ollama on loopback and put a proxy in front

Ollama’s documentation gives an Nginx example. It sets the Host header to localhost:11434, which the FAQ describes as a required header when Ollama itself is not exposed on the network — exactly this setup:

server {
    listen 80;
    server_name example.com;
    location / {
        proxy_pass http://localhost:11434;
        proxy_set_header Host localhost:11434;
    }
}

This is the better design because the proxy is where TLS and access control live. Don’t count on Ollama for that: its documentation describes no built-in authentication, and its own OpenAI-compatible examples note that the API key is “required but ignored”. Whatever sits in front of Ollama is what decides who gets in.

If a browser page or extension calls Ollama directly, you will also need OLLAMA_ORIGINS. Ollama allows cross-origin requests from 127.0.0.1 and 0.0.0.0 by default; extensions need their origin pattern added, for example chrome-extension://*.

For the wider picture of what running models yourself protects and what it doesn’t, see our self-hosted AI guide.

Tuning it for more than one person

A server that several people or apps hit at once needs different settings from a laptop. These are Ollama’s documented defaults:

Raising parallelism costs memory. Ollama’s documentation spells this out: memory scales with OLLAMA_NUM_PARALLEL × context length, so a 2K context serving 4 parallel requests behaves like an 8K context. Work out whether that still fits your card with the VRAM & KV cache calculator before you raise it.

Two other habits help on a shared server:

• Keep the main model loaded so the first request of the day isn’t slow: set OLLAMA_KEEP_ALIVE=-1, or preload it with an empty request — curl http://localhost:11434/api/generate -d '{"model": "llama3.2", "keep_alive": -1}'.
• Check where it actually loaded with ollama ps. The PROCESSOR column shows 100% GPU, 100% CPU, or a split — a split means the model did not fit in VRAM and will be noticeably slower.

If you are serving many simultaneous users, that is the point where a dedicated serving engine earns its keep — see Ollama vs vLLM and the vLLM hub. Size the hardware first with how much RAM and VRAM you need.

Putting a chat interface in front

Most people who run Ollama as a server want a ChatGPT-style interface on top of it. Open WebUI is the usual choice: it detects a local Ollama on port 11434 automatically and adds accounts, permissions and document search. Turning that document search into a working assistant over your own files is covered in building a private RAG chatbot. The Open WebUI hub pairs it with what to run behind it.

FAQ

What port does Ollama use?

11434, bound to 127.0.0.1 by default. Change the address and port with OLLAMA_HOST.

How do I let other devices on my network use Ollama?

Either set OLLAMA_HOST=0.0.0.0:11434 and firewall the port, or keep Ollama on loopback and expose it through a reverse proxy that handles TLS and access control. Binding to loopback is one of the containment controls in securing a local LLM, which covers what self-hosting does and does not protect against. The proxy is the safer choice.

Does Ollama have authentication?

Its documentation describes none, and its OpenAI-compatible examples say the API key is “required but ignored”. Put access control in whatever sits in front of it.

Can I use my GPU with Ollama in Docker on a Mac?

No. Ollama’s FAQ states GPU acceleration is not available for Docker Desktop on macOS. Use the native app on Apple Silicon.

Why does my model run slowly on a server with a good GPU?

Run ollama ps. If the PROCESSOR column shows a CPU/GPU split, the model — or the extra context from parallel requests — no longer fits in VRAM.

Sources

• Ollama FAQ — OLLAMA_HOST, per-OS environment variables, proxy and origins, concurrency defaults, keep-alive, model storage, Docker on macOS
• Ollama Docker docs — CPU, NVIDIA, AMD ROCm and Vulkan commands
• Ollama Linux docs — install script, systemd service, logs
• Ollama OpenAI compatibility — /v1 endpoints, “required but ignored” API key
• Docker: port publishing — “insecure by default”, binding to 127.0.0.1, the pre-28.0.0 caveat
• Ollama releases — v0.34.0, 5 September 2026; v0.40.2, 8 October 2026 · ollama/ollama on Docker Hub

Updated 14 September 2026. Commands and defaults verified against Ollama’s and Docker’s documentation on that date; defaults change between releases, so check the sources before relying on them for a production server.