Jan is a free, Apache-2.0 desktop app from Menlo Research that downloads and runs GGUF models through a llama.cpp engine built into the app. Version 0.8.8, released on 28 October 2026, bundles llama.cpp build b11669 from 15 October, runs on Windows 10 or later (x64 with AVX2, or ARM64), Apple Silicon Macs on macOS 13.6 or later, and the main Linux distributions, and on Windows and Linux asks for 8GB of RAM and, for GPU offload, 6GB of VRAM. Install 0.8.8 or newer rather than anything older: GitHub lists seven security advisories, one rated critical, that affect 0.8.7 and earlier. Jan’s OpenAI-compatible server runs on http://127.0.0.1:1337/v1 once you start it, and it should stay on 127.0.0.1.
This guide covers installing Jan, its engine and GPU backends, importing models, where files live, the API server and its security settings, MCP, what leaves your machine, and the most common failures. BFWR has not run Jan. The steps and figures come from Jan’s documentation, release notes, source repository, GitHub issues and security advisories, and the Hugging Face API, all read on 15 November 2026.
Quick answer
- Version: v0.8.8 (28 October 2026), a security release. Check the releases page for anything newer.
- Engine: llama.cpp ships inside the app and updates with it. NVIDIA cards use CUDA 13; AMD, Intel and older NVIDIA cards use Vulkan; Macs use Metal, with an experimental MLX engine.
- Your own GGUF: Settings, Model Providers, Llama.cpp, Import. Jan links the file in place.
- API server:
127.0.0.1:1337, prefix/v1, no API key unless you set one. Set one anyway. - Not local by default: web search is on and uses Exa’s keyless endpoint; analytics are asked for at first launch.
- Intel Macs: local models stopped working in v0.8.5.
Install and requirements
Download Jan from jan.ai or the GitHub release. The requirements below are from Jan’s install pages, and the file sizes from the v0.8.8 release assets (MB = 106 bytes).
| System | Requirements | v0.8.8 file | Size |
|---|---|---|---|
| Windows x64 | Windows 10+, AVX2 CPU (Intel Haswell 2013+, AMD Excavator 2015+), 8GB RAM (16GB recommended), 6GB VRAM for GPU use, 10GB free | Jan_0.8.8_x64-setup.exe |
571.7 MB |
| Windows on ARM | As above, ARM64 | Jan_0.8.8_arm64-setup.exe |
335.6 MB |
| macOS | macOS 13.6+, Apple Silicon, 10GB+ free | Jan_0.8.8_universal.dmg |
115.9 MB |
| Linux | Debian/Ubuntu, Fedora/RHEL, Arch, openSUSE; AVX2; 8GB RAM; 6GB VRAM | Jan_0.8.8_amd64.AppImage / Jan_0.8.8_amd64.deb |
712.8 MB / 1,062.7 MB |
The Windows and Linux packages carry the CUDA 13 engine, so no CUDA Toolkit is needed: “Jan bundles its CUDA 13 engine and runtime libraries.”
On Macs, the docs’ rough guide is 3B models on 8GB, 7B on 16GB and 13B on 32GB. “Local models do not run on Intel Macs since v0.8.5”; the app still opens there for cloud providers.
How Jan runs models: the llama.cpp engine
Since v0.8.5 the engine is not a separate download. In the docs’ words, “The llama.cpp engine is built into Jan and updates together with the app, so there is no separate engine version to download or pick.” It runs in its own worker process, and v0.8.8 moved it to llama.cpp b11669. The version, build and commit are shown under Settings > Model Providers > Llama.cpp.
Backends are chosen from what is bundled, under Settings > Hardware > GPUs:
- NVIDIA on Windows and Linux: CUDA 13. Cards that cannot run it, such as Pascal GTX 10-series, use Vulkan.
- AMD and Intel Arc: Vulkan, with a current driver. There is no ROCm download; for HIP, run your own
llama-serveras a custom endpoint. - Apple Silicon: Metal by default. The separate MLX engine needs macOS 14+, and its page says “MLX support in Jan is experimental”; it has no embeddings, so use llama.cpp for document chat.
Two defaults matter for memory. The context size defaults to 8192 tokens per model, and can go up to 131072. The KV cache type defaults to f16 for keys and values; switching KV Cache K Type and KV Cache V Type to q8_0 roughly halves its size. Our context window guide and VRAM calculator show what each costs.
Getting models in
From the Hub. The Hub tab browses Hugging Face, and each model carries a fit label, Fits, May be slow or Won’t fit, worked out for the machine Jan is running on. The docs add: “No data is downloaded to determine fit status.” You can also paste a Hugging Face model ID, such as the docs’ example TheBloke/Mistral-7B-v0.1-GGUF, into the Hub search, or use Use this model > Jan on a Hugging Face model page. Gated models need a token under Settings > General > Others > HuggingFace Token.
Your own GGUF file. Open Settings > Model Providers > Llama.cpp and click Import. “Jan links to the file in place without copying it”, so a model you already use in LM Studio or llama.cpp does not take up space twice, and deleting it in Jan leaves the original alone. For a split GGUF, point Jan at shard 1 and it picks up the rest.
Jan’s own models are small: Jan-v3-4b-base-instruct-Q4_K_XL.gguf, the example in its data-folder docs, is 3.00 GB, tagged Apache-2.0, and built on Qwen/Qwen3-4B-Instruct-2507, also Apache-2.0. For a step up on an 8–12GB card, see our Gemma 4 12B vs Qwen3.5 9B comparison and the model directory.
Where Jan keeps its files
- Windows:
%APPDATA%\Jan\data, which isC:\Users\<YourUsername>\AppData\Roaming\Jan\data - macOS:
~/Library/Application Support/Jan/data - Linux (.deb or AppImage):
~/.local/share/Jan/data - Linux (Flatpak):
~/.var/app/ai.jan.Jan/data/Jan/data
Downloaded GGUF models sit under llamacpp/models/<org>/<repo>/ as model.gguf beside a model.yml. Chats are under threads/ as messages.jsonl and thread.json, and the log is logs/app.log. To move it, use Settings > General > Jan Data Folder; since v0.8.8 a drive root such as D:\ is refused, so pick a folder like D:\Jan.
The local API server
Jan can serve a local model to other programs over an OpenAI-compatible API, and needs one loaded to start. Go to Settings > Local API Server, choose the model under Default Model, and click Start Server. With the default settings the base URL is http://127.0.0.1:1337/v1. The docs’ test request:
curl http://127.0.0.1:1337/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_API_KEY" \
-d '{
"model": "YOUR_MODEL_ID",
"messages": [{"role": "user", "content": "Tell me a joke."}]
}'
Omit the Authorization line if you have not set a key. Any OpenAI-compatible client can then use that base URL.
Security settings, and what to change
The defaults, from the server page and the v0.8.8 release notes:
- Server Host
127.0.0.1: “the most secure option for personal use”.0.0.0.0opens it to your network. - API Key empty. The docs say “Leave the field empty to disable authentication”, and that this is not recommended on
0.0.0.0. - Trusted Hosts: since v0.8.8, a web page can only call the server from your own machine, Jan itself, or a host you list.
- CORS on. The v0.8.8 notes say that before this release the switch “was shown but had no effect”.
- Execute Tools on Server off, so MCP tools do not run on requests that come in through the API.
Keep Server Host at 127.0.0.1 and set an API key anyway. If another device must reach it, set a key first and list only that device in Trusted Hosts. Two advisories show why. GHSA-x6p8-7cp8-c3p6 (high) found that on versions 0.8.0 to 0.8.4, binding to 0.0.0.0 “silently discards the Trusted Hosts allowlist”, fixed in 0.8.5. GHSA-66pg-h682-8cmf (high), a cross-site request forgery on the local API, affected 0.6.9 to 0.8.7. A further change, merged on 30 October in pull request #9505, makes Jan refuse a non-loopback address without a key. It is not in 0.8.8, so for now that is your job.
Of the eleven advisories published on 29 October 2026, seven affect 0.8.7 and earlier and are fixed in 0.8.8, among them GHSA-j6hv-vhmg-vpj6, a critical “One-click RCE” through malicious MCP or tool content.
MCP tools
Jan is an MCP host. Add servers under Settings > MCP Servers with the + button, giving a server name, a command and arguments; the docs’ example is Browser MCP, with command npx and argument @browsermcp/mcp, which needs Node.js. For a local model, turn on Tools under its capabilities.
By default “you must approve each tool call individually”, in a panel that shows the arguments the model wants to pass. Allow All MCP Tool Permissions pre-approves everything; leave it off, or a tool call prompted by text hidden in a web page runs unseen. Our prompt injection guide explains that risk. Since v0.8.7 the chat toolbar also has a Shell switch for the sandboxed shell tool, which is on by default for tool-capable models.
What leaves your machine
Three things can reach the internet:
- Web search, on by default. “The default provider (Exa) uses a free, keyless endpoint.” When a tool-capable model calls
web_searchorweb_fetch, the query or URL goes to that provider. Turn it off in Settings > Web Search, per chat with the globe icon, or switch the provider to your own SearXNG instance. - Analytics, by choice. The privacy page says “You’ll choose tracking preferences at first launch.” If allowed, it counts active users and retention against a random ID, held by PostHog in the EU.
- Cloud providers. OpenAI, Anthropic, Google, Groq, Mistral and OpenRouter are built in under Settings > Model Providers, with your own API key. As the privacy page puts it, “Cloud models (like GPT, Claude) need to see your messages to work.”
Common failures and fixes
Errors appear as a banner in the chat, with the cause in logs/app.log on lines containing jan-llama-worker:. The troubleshooting page lists these:
- “This model’s architecture is not supported by the current engine.” In the log:
unknown model architecture: '<arch>'. The bundled llama.cpp is older than the model. Update Jan, or run a newerllama-serveryourself (for examplellama-server -m /path/to/model.gguf --alias my-model --port 8080 -ngl 99 --jinja) and add it as an OpenAI-compatible custom provider athttp://localhost:8080/v1. Qwen3 hit this in April 2025 (#4948); v0.5.17 followed in May. - Out of memory (
failed to allocate ... buffer,cudaMalloc): pick a smaller quantization, cut the context size, lower GPU Layers, or switch the KV cache toq8_0. - Corrupt or partial file (
failed to load model,missing tensor): download it again. - “The model took too long to load and was stopped.” The load timeout is 600 seconds; raise Timeout for llamacpp or move models to faster storage.
Three hardware cases from the issue tracker. On a GTX 10-series card, v0.8.4 picked a CUDA 13 build that has no Pascal code, so every load failed (#9091); since v0.8.5 those cards use Vulkan. On a GTX 16 or RTX 20 card, the error “the provided PTX was compiled with an unsupported toolchain” in v0.8.5 is fixed by v0.8.6. On Linux, the v0.8.5 AppImage crashed in libvulkan.so.1 on every load (#9173) because it shipped its own Vulkan loader; v0.8.6 uses the system’s. The reporter still saw crashes on 0.8.8 with MangoHud installed, an open case; a maintainer’s suggested fallback is the .deb, which uses system libraries.
The licence
Jan’s LICENSE is the Apache License 2.0, “Copyright 2025 Menlo Research”, with one extra line: “Attribution is requested in user-facing documentation and materials, where appropriate.” That is a request, not a condition, so you can use, modify and redistribute Jan, including commercially, under the normal Apache terms. Each model carries its own licence, which Jan does not change; check it in our model licence checker. This is our reading of the published text, not legal advice.
FAQ
What port does Jan’s local API use?
1337 on 127.0.0.1, with the prefix /v1, so chat requests go to http://127.0.0.1:1337/v1/chat/completions. Port, host and prefix can be changed under the server’s Configuration button.
Can Jan use my Ollama models?
Yes, through a custom OpenAI-compatible provider with the base URL http://localhost:11434/v1, which Jan’s docs list for Ollama. Ollama then runs the model and Jan is the chat window. Plain GGUF files can be imported instead.
Why is Jan not using my GPU?
Check that the GPU’s switch is on in Settings > Hardware > GPUs, that a current driver is installed, and, for AMD and Intel, that it supports Vulkan.
Does Jan work on an Intel Mac?
Only with cloud or custom providers. Local models have not run on Intel Macs since v0.8.5.
Related guides and tools
- Best local LLM runners: Ollama vs LM Studio vs Jan vs llama.cpp vs vLLM
- Gemma 4 12B vs Qwen3.5 9B: which small model to run
- LLM context windows and what they cost in VRAM
- Securing a local LLM: prompt injection
- Open WebUI: setup and backends
- VRAM calculator
- Model directory
- llama.cpp models and commands
Sources
- janhq/jan:
LICENSE, README, the v0.8.8 release and its assets, release notes for v0.8.0 to v0.8.8 indocs/src/pages/changelog/, pull request #9505, and issues #4948, #9091 and #9173 - Jan docs: Windows, Mac and Linux install, llama.cpp engine, MLX, model management, data folder, local API server, MCP, web search, privacy, custom endpoints and troubleshooting
- Jan security advisories, read through the GitHub API
- Hugging Face API: janhq/Jan-v3-4B-base-instruct-gguf, janhq/Jan-v3-4B-base-instruct and Qwen/Qwen3-4B-Instruct-2507
- llama.cpp release b11669
Updated 15 November 2026. Jan v0.8.8, its release assets, release notes and security advisories, the live docs at docs.jan.ai, the issues cited and the Hugging Face model data were read on that date.
Work out what your hardware can actually run
Free calculators and directories, sized from real model weights — before you download gigabytes or order a card:

