oMLX

oMLX turns your Mac into a full LLM inference server, run from the menu bar. It serves text, vision, OCR, embedding, and reranker models with continuous batching, plus a RAM and SSD tiered KV cache that survives restarts. The result: Claude Code and Cursor respond in about 5 seconds instead of 90 when using local models.

The key innovation is the persistent KV cache. When an agent loops back with follow-up context, most local setups recompute the entire conversation from scratch. oMLX writes the KV cache to SSD, so old context comes back in milliseconds even after a restart. This eliminates the biggest bottleneck in local model agent loops.

Built in native Swift, not Electron. Supports OpenAI and Anthropic compatible APIs so it drops straight into existing toolchains. Apache 2.0 licensed and open source on GitHub. Over 21,000 stars and daily shipping since February 2026.