Profile
Back to NewsBack
GitHub Trending 11 min
Reader Mode
joshuaswarren/omarchy-mlx: MLX-compatible Vulkan and ANE backends for Apple Silicon running Linux.

joshuaswarren/omarchy-mlx: MLX-compatible Vulkan and ANE backends for Apple Silicon running Linux.

17 hours ago

omarchy-mlx

No Mac? Start here for NVIDIA, Intel, AMD and software Vulkan development paths.

Results so far: docs/TESTED-HARDWARE.md.

MLX on the Apple GPU under Linux, and Core ML models on the Apple Neural Engine.

MLX is Apple's array framework for machine learning on Apple silicon. Upstream runs on macOS through Metal and on Linux through CUDA or CPU. omarchy-mlx runs the same import mlx.core as mx code on an Apple silicon Mac running Omarchy Linux, on the Apple GPU. The GPU driver is Mesa's Borealis Vulkan stack (Aurora's rename of the Honeykrisp driver; both names are accepted). Tensors stay on the GPU. There is no Metal and no CPU fallback.

It is for people who run Linux on an M1 or M2 Mac. You get local chat, serving, and speech to text, with no macOS and no cloud account.

The projects

Part of the Omarchy on Apple Silicon ecosystem:

  • omarchy-ane — Linux driver for the Apple Neural Engine: DRM kernel modules and libane.
  • mil-hwx-compiler — compiles Core ML MIL into ANE programs on Linux (M1/M2 targets).
  • metal2vk — translates Metal compute kernels to Vulkan SPIR-V with open tools.
  • mesa-1 — Mesa fork carrying the Borealis Vulkan driver work for Apple silicon (the honeykrisp-omarchy branch).
  • omarchy-cluster — one local model split across Macs on macOS and Omarchy Linux.
  • omarchy-apple-dev — build and ship iOS apps from Omarchy Linux, no Xcode or Mac.
  • omarchy-egpu — eGPU setup, diagnostics and measured local-inference results on Omarchy Linux.
  • ane-linux-experiments — Neural Engine on Linux: verified primitives, receipt-backed macOS parity.
  • coreglass — profiler that shows where local inference loses speed on Apple Silicon Linux.
  • omarchy-inference-fleet — reproducible, pinned single-node MLX inference baselines.
  • remnic — local-first memory and context for AI agents.
  • Platform upstream: omarchy-mac — Omarchy on Apple Silicon Macs.

What runs on Omarchy Linux

mlx-lm, oMLX, mlx-serve, TensorFold, sushi, llama.cpp (Vulkan) and MCDMA have been tested on an Apple silicon Mac under Linux. The status of each, with the commit or measurement behind it, is in docs/ENGINES.md.

What you get

  • The mlx Python module with the Vulkan GPU backend. The distribution
name is mlx-omarchy; the module stays mlx. Upstream source is fetched at a pinned commit, and this project's code lives in overlay/ and patches/. The tree stays a small patch-set, not a fork.
  • MLX Chat, a local web app for chat and comparing options, with a
browser UI and a terminal mode. The install registers it in the launcher and sets up a user service that keeps the loaded pair ready across logins.
  • mlx-omarchy-serve, a serving CLI with catalog, plan, and
serve commands, memory admission, and approve-first model downloads. The same install ships the Laya typed-decision server and the Bonsai2 packed-runtime server.
  • Core ML on the Neural Engine. mlx-omarchy-parakeet transcribes audio
(download, verify, transcribe). mlx-omarchy-coreml inspects a Core ML package. The aarch64 wheel ships the ANE worker and the pinned Parakeet encoder bundles.
  • An opt-in wake word. mlx-omarchy-chat --wake-word hey_jarvis
listens through openWakeWord ONNX models on the CPU (SHA-256-pinned files, downloaded once on first use) and hands the microphone to the chat when it hears the phrase. It is off by default. The score threshold is 0.5 and lives in one place (wake_word.WAKE_THRESHOLD_DEFAULT). Measure it on your own microphone with python -m mlx_omarchy_assistant.wake_eval --manifest clips.json --clips-dir clips --model --out results.json (receipt).

Chat models download from Hugging Face the first time you use them. You approve each download first. The default pairs: Everyday is Qwen3.5-9B 4-bit plus the Laya model. Compact is Qwen3-4B-Instruct-2507 4-bit plus Laya. Quality is Qwen3.8-27B 4-bit plus Laya, and it needs a 96 GB machine.

Supported hardware

| Chip | GPU | ANE | |---|---|---| | M1 | Tested | Parakeet islands | | M1 Max | Tested | Whole encoder | | M2 Max | Tested on Linux | Whole encoder (bit-exact on the M2, see M2 Max ANE) (results) | | M1 Pro | Tested on Linux: MacBook Pro 16-inch, 16-core GPU, release wheel on the Borealis Vulkan driver, Qwen3-4B 4-bit decode 31.5 tok/s | Untested | | M2 | Tested on Linux: MacBook Air 13-inch, 8-core GPU, release wheel on the Borealis Vulkan driver, Qwen3-4B 4-bit decode 19.4 tok/s | Untested | | M1 Ultra, M2 Pro, M2 Ultra | Untested | Untested | | M3 | Experimental: aurora mesa-m3 graphics; compute not certified | Data-only; h15 bring-up module | | M3 Pro | Tested on Linux: MacBook Pro 14-inch, 14-core GPU, release wheel on the Borealis Vulkan driver, Qwen3-4B 4-bit decode 43.9 tok/s; standing battery 747/747 cases on source build a893cae5c | Untested | | M4 | Experimental: Mac mini with the out-of-tree G16G driver and Honeykrisp's Apple9 compute port | Data-only; h16 bring-up module | | M4 Pro | Tested on Linux: MacBook Pro 16-inch, 20-core GPU, release wheel on the Borealis Vulkan driver, Qwen3-4B 4-bit decode 49.3 tok/s; runtime 59/60 on source build ec849019 (independent-streams ratio 0.876 against that build's 0.95 floor; main has the 0.80 Borealis floor, not re-run) | Untested | | A18 Pro | Tested on Linux: MacBook Neo, 5-core GPU, release wheel on the Borealis Vulkan driver, Qwen3-4B 4-bit decode 13.5 tok/s; runtime 59/60, known issue: the independent-streams timing check fails against the 0.80 Borealis floor, copies are byte-exact | Untested |

The supported installer targets are M1-class Macs and the M2 Max. They share one GPU class and driver path. The Tested rows are the machines this project measures on. The M4 row needs the out-of-tree G16G driver stack named in its receipt. The M1 Pro, M2, M4 Pro and A18 Pro rows run the release wheel on Fedora with aurora's Borealis Vulkan driver (Linux 7.3); the setup is in docs/INSTALL.md. The M3 Pro row runs the release wheel on Omarchy with the linux-aurora 7.2.11 kernel and the mesa-aurora and vulkan-aurora Borealis packages through the system ICD. Each of these rows is one machine with the GPU core count shown; the other core counts of those chips are not measured. On an M3 or M4 Mac, run the tester kit, bash scripts/m3m4_kit.sh (see scripts/m3m4_kit.md), to send us the numbers and ANE state that only real silicon gives. On an M4 that runs Linux the kit runs the collector only, and the ANE is not supported there yet. The beta path for M4 on Linux, with its steps and what is not measured, is in docs/m4-linux.md.

M5 users: use the edge channel for now. The M5 is not in the table above and has no receipt in this repo yet. The Omarchy rc and stable channels still carry omarchy-mlx 0.7.28, which cannot use an M5 GPU (it accepts only the stock Honeykrisp driver id; the Aurora driver id the M5 reports was added in v0.7.39). Edge carries 0.7.40-3, which ran on an M5 Max GPU in a lab run with the system driver. Switch back when rc and stable move.

The ANE is separate hardware from the GPU. It has its own driver and its own tested state. Chip-by-chip ANE status, including what each untested chip needs, lives in the chip coverage table of omarchy-ane.

Install

Run one command on an Arch install for an M1-class Mac or an M2 Max. The script picks the latest release, checks the wheel against the release SHA256SUMS, and requires Python 3.14 on aarch64.

curl -fsSL https://raw.githubusercontent.com/joshuaswarren/omarchy-mlx/main/install.sh | bash

The other ways to install (the Omarchy package, the script flags, a release wheel) are in docs/INSTALL.md.

Quick start

After any install path, this works:

mlx-omarchy-info                                                   # GPU and driver state
mlx-omarchy -c "import mlx.core as mx; print(mx.default_device())" # Device(gpu, 0)
mlx-omarchy-chat                                                   # chat in the browser

The first chat run asks you to approve a model download. Then it streams the answer from the Apple GPU.

Usage

See docs/USAGE.md.

Other Vulkan GPUs (development)

Not a supported target. With MLX_OMARCHY_ALLOW_NON_APPLE=1 the backend admits a Vulkan 1.3 device that exposes timeline semaphores and a compute queue, for development on a normal Linux box. The flag and the llvmpipe note are in Development override. Driver selection with VK_DRIVER_FILES is in Selection. The non-Apple refusal without the flag stays.

Observed, from PR #111, run record in receipts/2026-10-11-non-apple-gpu-run/README.md:

| GPU | Driver | Result | |---|---|---| | Intel UHD 770 | Mesa ANV 26.2.4 | 55/55 omarchy_runtime_tests; op check clean | | RTX 5060 Ti | NVIDIA 615.71.09 | 54/55; the independent-streams timing check (0.78-0.88 vs a 0.95 floor), see Non-Apple GPUs |

AMD (RADV) has no qualifying run.

How close to macOS

Same MacBook Pro (M1 Max, 64 GB), same models, same prompts, MLX on both sides with the same mlx-lm (0.32.0), measured 2026-10-09 to 2026-10-11. The Linux column runs the mlx-lm patches the installer applies. Where macOS is shown, the percentage is Linux as a share of macOS: 100% means equal, higher is faster on Linux.

Decode is the median of 10 greedy generations and prefill is the median of 5 runs, except two Linux cells: gemma-4-E4B is the mean of three runs and Qwen3.6-35B-A3B the mean of two. Those two sets of runs started with a load average above 0.5, so the numbers may be low. The Qwen3.5-9B macOS cells are earlier sealed runs.

MLX 4-bit through the installer's mlx-lm patches, tokens per second. A cell shows Linux, or Linux / macOS and the percentage where macOS is shown:

| Model | Decode | Prefill 512 | |---|---|---| | Qwen3.8-27B (dense 27B) | 13.5 | 87.4 | | gemma-4-26B-A4B (MoE, 4B active) | 12.61 | 116.50 | | gemma-4-31B (dense 31B) | 7.3 | 62.4 | | Qwen3.5-9B (dense 9B, gated-delta) | 37.1 / 59.4 = 62% | 299.4 / 339.0 = 88% | | Qwen3.6-35B-A3B (MoE, 3B active) | 13.62 | 116.37 | | gemma-4-E4B (dense 4B) | 30.5 | 366.5 | | gpt-oss-20b (MoE, attention sinks) | 2.2 | 1.9 |

macOS shares are printed only where both sides produce identical tokens.

Driver build: the two mixture-of-experts rows (gemma-4-26B-A4B and Qwen3.6-35B-A3B) and the gemma-4-E4B row ran on main at e57cb301 with the private Honeykrisp ICD at Mesa git 6543eeb7df7. For the other rows the driver build was not recorded. The full tables, per chip, with the wheel, the driver and the start load of each run, are in docs/BENCHMARKS.md. Differences from macOS and models that did not run are in docs/KNOWN-ISSUES.md.

Decode on the 13-inch M1 changed with the CPU governor with no ANE work running. See Set the CPU governor for local AI.

Qwen3.5-9B fused GDN decode

See docs/DRIVER-AND-KERNEL.md.

Muse Spark Open

The mlx-community Muse Spark Open checkpoints load through the adapter in serve/mlx_omarchy_muse. Their 3-bit and 2-bit routed experts run on the GPU. See docs/muse-spark.md.

How it works

See docs/DRIVER-AND-KERNEL.md.

Contribute hardware data

The fastest way to help is a capture from your machine. Owners of the untested chips above are the unblock for those rows. Both collectors print a fully redacted payload. Nothing is sent until you pass --submit.

git clone https://github.com/joshuaswarren/omarchy-mlx.git
cd omarchy-mlx
python3 scripts/collect_quick.py    # quick capture, Linux
python3 scripts/collect_deep.py     # deep capture, Linux and macOS

add --submit to publish. It already targets the public endpoint; the

deep run also needs --out FILE:

python3 scripts/collect_deep.py --out mlx-omarchy-deep.tar.gz --submit

On a dual-boot Mac, run collect_deep.py under macOS and under Omarchy, and submit both. The full guide, including what gets collected and how redaction works, is docs/contribute-data.md.

Apple Neural Engine

Measured M2 Max results: docs/ANE-LINUX.md. To enable the ANE on your chip: docs/ANE-LINUX.md.

Troubleshooting

See docs/TROUBLESHOOTING.md.

Support

If omarchy-mlx is useful to you: sponsor on GitHub, a Lightning donation to [email protected], star the repo, or stock the lab at joshuawarren.com/sponsors.

Contributing

Code contributions start at CONTRIBUTING.md. Hardware captures start at docs/contribute-data.md. Dev machines without an Apple GPU can run the module under software Vulkan. GPU kernel changes still need real hardware before a release. Any other Vulkan GPU (NVIDIA, Intel, AMD): the contributor kit is docs/ANY-GPU.md and the results so far are in docs/TESTED-HARDWARE.md.

Releases

See docs/RELEASING.md.

License

MIT, see LICENSE. Prepared MLX source keeps Apple's MIT license and copyright notices. Other bundled components keep their own notices under LICENSES. Not affiliated with Apple.

Contributors

Everyone credited so far: CONTRIBUTORS.md.

Chat with me