oMLX: A Local LLM Server for Apple Silicon Macs

oMLX is an open-source server for running language models on Apple Silicon Macs, built on Apple’s MLX framework and managed from a menu bar app. It appeared in February 2026 and has gathered about 22,900 stars on GitHub, which makes it one of the most-watched local AI projects on the Mac this year.

Its pitch is narrower than Ollama’s or LM Studio’s: it is built for long, repetitive workloads such as coding agents, where the same large prompt is sent again and again. This guide covers what it does differently, how to install and secure it, and when it is worth using over the tools you may already have. BFWR has not run oMLX; everything here is from its repository and documentation.

Quick answer

• What it is: an OpenAI- and Anthropic-compatible server for MLX models, with a macOS menu bar app and a web dashboard. Apache 2.0.
• What is different: it keeps the processed prompt (the KV cache) in memory and on your SSD, so a repeated prefix can be restored instead of recomputed, even after a restart. It also batches concurrent requests and serves several models at once.
• Who it suits: people using Claude Code, OpenCode or similar agents on a Mac, or serving several models to a few users.
• Requirements: macOS 15 (Sequoia) or later and an Apple Silicon Mac.
• Caveat: it is young (version 0.7), changes quickly, and is largely one developer’s work.

What oMLX does

The project describes itself as an “LLM inference server with continuous batching & SSD caching for Apple Silicon”. Its main features, from the README:

• Tiered KV cache. A hot tier in RAM and a cold tier on SSD. When the hot cache fills, blocks move to disk, and “On the next request with a matching prefix, they’re restored from disk instead of recomputed from scratch – even after a server restart.”
• Continuous batching. Concurrent requests are batched together, with a default limit of 8 at a time.
• Multi-model serving. Language, vision, embedding and reranker models in one server, with least-recently-used eviction, pinning and a per-model idle timeout.
• API compatibility. It works as a “Drop-in replacement for OpenAI and Anthropic APIs”, at http://localhost:8000/v1.
• Admin dashboard at /admin, with a model downloader, built-in chat, per-model settings and a one-click benchmark that measures prompt processing and generation speed.
• Integrations. One-click setup for OpenCode, Codex, Copilot and several other agents from the dashboard.

Its author explains the reason for the cache in the README: keeping all past context cached and reusable “making local LLMs practical for real coding work with tools like Claude Code”. It also has a specific Claude Code fix: it reports the model’s real context window so Claude Code compacts the conversation at the right point, and keeps the connection alive during long prompt processing.

Why the cache matters on a Mac

Macs generate text well because of their memory bandwidth, but they typically process long prompts much more slowly than a high-end NVIDIA card. A coding agent can send tens of thousands of tokens of instructions and code with every request, and most of that is identical to the last request. A server that recomputes it each time makes you wait for the whole prompt again; one that reuses the cached prefix only has to process what changed. oMLX’s addition is writing that cache to disk, so it survives model swaps and restarts.

By default the SSD cache can use up to half of your free disk space plus whatever it already holds, which on a nearly full drive is worth knowing. You can set an explicit limit, and the in-memory tier has its own size setting:

omlx serve --model-dir ~/models --paged-ssd-cache-dir ~/.omlx/cache --hot-cache-max-size 20%

Installing it

There are three routes. The macOS app is the simplest: download the .dmg from the GitHub releases page, drag it to Applications, and follow the welcome screen, which sets a model folder, starts the server and downloads a first model. The app updates itself.

With Homebrew:

brew tap jundot/omlx https://github.com/jundot/omlx
brew install jundot/omlx/omlx
omlx start

One detail matters for speed. oMLX has native Metal kernels that some model families need; the README warns that “GLM-5.2, MiniMax M3, and Qwen3.5 need them”, and without them those models “silently fall back to much slower generic paths”. The app ships them precompiled. A Homebrew or source install builds them only with the --with-custom-kernel option or make install, which needs full Xcode, not just the command-line tools. If you use Qwen3.5 models, use the app or install Xcode.

Models go in a folder, one subfolder each, in MLX format. oMLX detects their type automatically, and you can also download MLX models from Hugging Face in the dashboard. The server needs Python 3.11 to 3.13 if you install from source.

oQ: its own quantization

oMLX can shrink full-precision models itself. Its oQ quantizer “measures how much each layer suffers from quantization, then spends extra bits where they help most”, and an enhanced mode, oQe, uses an importance matrix to round the most important weights more accurately. The output is a standard MLX model that also loads in mlx-lm and other MLX apps. The documentation recommends starting with oQ4e, which works out at about 4.6 to 4.7 bits per weight; a 9B model at oQ4 “comes out at about 5.6 GB”. It advises against 2-bit levels for dense models.

Security

oMLX listens only on your own machine by default, and the README says it “refuses to start on any non-loopback address without a main API key”. To share it on your network, set a key first:

OMLX_API_KEY=your-secret-key omlx serve --model-dir ~/models --host 0.0.0.0

That is a safer default than many local servers. There is a setting to allow inference without a key on a network address; leave it off unless you trust everyone who can reach the machine. Our guide to running a local model server covers the general case.

oMLX, Ollama or LM Studio?

All three now run MLX models on Apple Silicon. Ollama’s v0.40.0 release on 25 September 2026 announced “Models run on MLX on Apple Silicon by default” for supported architectures, and LM Studio has offered MLX models for longer. So MLX itself is no longer the reason to switch.

• Use oMLX if you run coding agents or other long, repeated prompts on a Mac, want several models served at once, or want to quantize your own MLX models.
• Use Ollama if you want one tool across Mac, Windows and Linux, the largest ready-made model library, or the simplest setup. Our Claude Code with Ollama guide covers the agent setup there.
• Use LM Studio if you want a polished desktop app for browsing and chatting with models.

Our comparison of local LLM runners covers the wider field.

Things to weigh before relying on it

• Pace of change. It is at version 0.7, with development builds most weeks: v0.7.0 on 30 September and v0.7.1rc1 on 15 October 2026. Expect settings and behaviour to move.
• Maintainers. Its creator has made about 2,100 commits; the next-largest contributor has 85. GitHub shows about 650 open issues and pull requests. That is a lot of activity, and also a lot riding on one person.
• Mac only. If you might move to a Linux box or a PC with a graphics card later, your setup will not carry over.
• Memory limits still apply. macOS caps how much unified memory the GPU can use. oMLX has its own memory guard, with safe and balanced tiers, but the system limit is separate; our Mac VRAM optimizer explains how to raise it.

FAQ

Is oMLX free?

Yes. It is open source under Apache 2.0.

Does oMLX work on Intel Macs?

No. It requires Apple Silicon and macOS 15 or later.

Can I use oMLX with Claude Code?

Yes. It speaks the Anthropic API and includes specific handling for Claude Code’s context compaction. The model still needs tool-calling support.

Is oMLX faster than Ollama?

For a single short prompt, both run on MLX, so we would not expect a large difference. oMLX’s advantage is on repeated long prompts, where its cache avoids reprocessing, and on concurrent requests. Its website hosts a community benchmark database, with results submitted by users, if you want figures for your chip.

Related guides and tools

• Mac mini M6 for local AI: which configuration to buy
• Best Mac for local LLMs
• OpenCode with a local LLM
• Apple Silicon vs Snapdragon X for Local AI (2026)
• llamafile in 2026
• LM Studio Bionic

Sources

• jundot/omlx on GitHub (README, releases and contributors) and its oQ quantization guide
• oMLX community benchmarks
• Ollama v0.40.0 release notes

Updated 18 October 2026. Repository details, versions and documentation were read on that date; oMLX changes quickly.