Local inference engine · M1 and M2 · community port

Splash M1

Splash is a local inference engine built around Qwen3.8-27B. It serves the model to coding agents and any OpenAI or Anthropic compatible client, entirely on the Mac. This is a community port for M1 and M2: the Metal kernels are rewritten to use formats those GPUs support natively, so a 2021 M1 Max decodes at 39 tok/s instead of 19, and prefills at 7.4 TFLOPS instead of 2.4.

or copy the one-line install:

Prefer a server with a dashboard? Run it inside oMLX · all files on GitHub

Read the write-up

macOS 26 or newer · Apple Silicon (M1, M2) · 16 GB RAM or more

Built on Inco's Splash

The engine, the model-specific kernels and DFlash speculative decoding are Inco's work.

This is an unofficial community port. It is not affiliated with, endorsed by or supported by Inco. If you have an M3 or newer Mac, use upstream Splash: it is faster there and it is the version Inco supports.

Benchmarks

Five prompts, same M1 Max (32 GPU cores, 64 GB)

npanj's Splash benchmark, Qwen3.8-27B 4-bit, temperature 0, fresh server, no prompt cache. "Before" is stock Splash kernels on the M1, "after" is this port.

PromptStock on M1This portM5 Pro (for scale)
Math (bat and ball)27.1 tok/s57.0 tok/s83.3 tok/s
Coding (merge_intervals)20.1 tok/s43.4 tok/s75.5 tok/s
Constraint reasoning19.7 tok/s38.2 tok/s59.3 tok/s
Domain knowledge11.7 tok/s25.3 tok/s39.0 tok/s
Writing (3 sentences)15.9 tok/s31.6 tok/s46.2 tok/s
Average18.9 tok/s39.1 tok/s60.7 tok/s

Against two MLX engines, one 44K-token prompt

Same Mac, same model. Decode speed, GPU temperature, energy per token and memory after the prompt.

EngineDecodeGPU tempEnergy per tokenMemory
oMLX19.8 tok/s77.7 C2.20 J23 GB
MTPLX25.9 tok/s78.8 C1.69 J35 GB
Splash M141.5 tok/s75.3 C1.06 J21 GB

Where the speed comes from

Measured on the M1 Max, not taken from the spec sheet.

MetricStockThis portHardware limit
Prefill matmul2.4-3.5 TFLOPS7.4 TFLOPS10.4 TFLOPS
Decode, 4 parallel requests18 tok/s65 tok/smemory bound
35B-A3B decoden/a144 tok/smemory bound
Top-1 token agreementn/a99.92%reference

Two ways to run it

Command line

The engine itself: one binary, an OpenAI-compatible server on localhost. This is what scripts and coding agents talk to.

  1. Install the binary

    curl -fsSL https://github.com/paperniuk/splash/releases/latest/download/install-m1.sh | sh

    Downloads the latest release, puts splash-m1 on your PATH and leaves any official splash alone.

  2. Check it runs

    splash-m1 --version

    Prints the version and the GPU it picked. Needs an M1 or M2 Mac with macOS 26 or newer.

  3. Serve a model

    splash-m1 serve --model mlx-community/Qwen3.8-27B-4bit

    Downloads the 4-bit Qwen3.8-27B (about 21 GB) on first run, then serves it. Decodes at about 39 tok/s on an M1 Max.

  4. Call it like the OpenAI API

    curl localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"qwen3.8-27b","messages":[{"role":"user","content":"hello"}]}'

    Any client that speaks the OpenAI API works: OpenCode, Claude Code, your own scripts. Point them at http://localhost:8080/v1.

GGUF files work too: splash-m1 serve --model unsloth/Qwen3.8-27B-GGUF:Q4_K_M.

oMLX

Run Splash models through the oMLX server. Pick Splash as the external engine.

  1. Install Splash M1 (the engine)

    curl -fsSL https://github.com/paperniuk/splash/releases/latest/download/install-m1.sh | sh

    oMLX looks for splash-m1 on PATH and in /opt/homebrew/bin, /usr/local/bin, ~/.local/bin, and prefers it over the official build on M1 and M2 Macs.

  2. Check the integration

    SettingsIntegrationsExternal engines

    The card shows splash-m1, its path and version. From oMLX 0.7.1 it fills in by itself.

  3. Serve a model through Splash

    Model SettingsAdvancedUse external engine

    The button next to the toggle downloads the DFlash2 draft the model needs; the toggle stays off until it is there. GGUF models have the external engine on already.

Qwen3.8-27B and Qwen3.6-35B-A3B GGUFs run on Splash too, and the oMLX downloader fetches them.

Latest release

Splash 1.7.0-m1 for M1/M2

Splash 1.7.0-m1 for M1/M2: upstream 1.7.0, faster decode with several requests at once, and the Neural Engine pair is now calibrated by itself.

curl -fsSL https://github.com/paperniuk/splash/releases/download/1.7.0-m1/install-m1.sh | sh

Already on 1.6.1-m1.x? Run the same line; it installs next to it and switches splash-m1 to the new one. Desktop app: download Splash-1.7.0-m1.dmg and replace the old Splash.app (same first-open steps as before).

The first start of each model takes longer once: Splash 1.7 measures decode on your Mac, and on the dense 27B the Neural Engine split is calibrated again, now with its pair of chunks (about 5 minutes on an M1 Max). Later starts reuse it. Persistent caches from 1.6.x start cold once.

From upstream 1.7.0: draft trees and prompt-lookup candidates verified in one pass, INT8 cached states (about a third of the size), KV that keeps decoding while it spills to SSD, --host unix:PATH, --reasoning-effort, --chat-template-file. Details in Inco's notes: https://github.com/incoai/splash/releases/tag/1.7.0

New on M1/M2: - The Neural Engine pair from 1.6.1-m1.1 no longer needs a share by hand. The first start times it and keeps it for prompts of 769 rows or more, single chunks below (#66 by @wtfsayo, the follow-up to #64). On an M1 Max it picked 0.74 on the 27B MLX and 0.79 on UD-Q5_K_M. - MLX 4-bit decode on the register tile is faster: affine units split by a fitted wave law (#69) and fp16 subnormal operands in the 4-bit chains (#70), both by @wtfsayo. - Short appends prefill faster where a projection has too few tiles for the GPU: the M1/M2 prefill tile now splits K there (#73 by @ghisguth). On Flash-Next a 128-token append to a 16.7K history answers in 691 ms instead of 758, a 384-token one in 1392 instead of 1487, and decode with 2 and 4 requests at once is 5 to 6% faster. - Flash-Next's IQ2_XXS and IQ2_S experts (the UD-IQ1_S pack) run on the column tile in 8-row decode passes (#72 by @ghisguth). Not timed here, the UD-IQ1_S pack is not downloaded. - Port fixes for 1.7's 16-row trees: MoE MMA tiles now address rows from their tile's first grouped row, and a decode step of three or four lanes (48 or 64 rows) on the prefill MMA tile reads and writes its own rows only. Without them the MoE decode tests gave wrong values at four lanes and the engine's decode warm-up of three or four lanes failed.

M1 Max 64 GB, packaged builds, ABBA (two runs each), --max-cache-disk 0, greedy, reasoning off, 400 tokens on four short code prompts. Decode tok/s at 1 request / 2 at once / 4 at once, cold prefill tok/s at 2.1K / 16.7K tokens, 1.6.1-m1.1 -> 1.7.0-m1:

Model 1 2 4 prefill 2.1K prefill 16.7K
27B MLX 4-bit 57.0 -> 60.5 (+6%) 62.5 -> 70.8 (+13%) 65.4 -> 80.0 (+22%) 209 -> 218 (+4%) 199 -> 227 (+14%)
27B UD-Q5_K_M 49.5 -> 49.0 (-1%) 57.0 -> 59.7 (+5%) 63.4 -> 70.0 (+10%) 199 -> 209 (+5%) 193 -> 224 (+16%)
Flash-Next Q2_0 81.1 -> 81.4 101.1 -> 100.7 113.3 -> 124.0 (+9%) 459 -> 449 (-2%) 434 -> 422 (-3%)

This table is without #72 and #73. Flash-Next's cold prefill is 2 to 3% slower than on 1.6.1-m1.1; not tracked down yet. With #72 and #73 on top (same Mac, ABBA against the table's build): Flash-Next decode at 2 / 4 requests 97.0 -> 101.5 / 116.5 -> 123.6 tok/s, appends of 128 / 384 tokens on a 16.7K history 758 -> 691 ms / 1487 -> 1392 ms; 27B UD-Q5_K_M prefill 2.1K 202 -> 209 tok/s; 27B MLX unchanged. One request on Flash-Next moves by up to 10% per prompt either way, since #73 changes prefill sums and with them the greedy text; a decode cycle's GPU time stays within 1.3%.

Thank you: @wtfsayo for #66, #69 and #70, @ghisguth for #72 and #73.

Not fixed yet: decode slowing down after about a day of uptime with "growth paused" in the log (#63).

Checks on an M1 Max 64 GB: test-engine-metal passes in full (shader validation, with the ane-ffn kernels, split, faults and program), and the CPU and Python tests pass. The packaged build answers text, a cached follow-up and an image correctly with the 27B MLX 4-bit, 27B UD-Q5_K_M and Flash-Next Q2_0 at default settings, and Flash-Next with --expert-memory 16G.

This is an unofficial community port. Splash, its models and draft models are built by Inco (https://github.com/incoai/splash); official Splash runs on M3 and newer.

Built from commit dac0b207 (branch apple7-m1-1.7) on macOS 26.7.3 with Apple clang 21.0.0.

File SHA-256
splash-1.7.0-m1-arm64-macos26.tar.gz 852e7427a85e27dfaadcab1928bf13b5abffbb148940b5e193ca4750f546a561
Splash-1.7.0-m1.dmg bd3faf1a3d1023d8494fbed3958ce1a75bfc808ad3594e814a69b3739744f724
install-m1.sh 2a19e4f8ba59a57f23b8a1c5a4892b93c4c33165f9b826a13a84c65fd5b1b39e
engine splash 894b5a1027851475c0caca5c163948f15c1c59827dd841d64a607d894e5cd068
splash.metallib 753551182071b8e5df17757c7d14d906525f37c0cdfdb6e58c04ca885ecd7881

Every release and changelog on GitHub

Questions

Does Splash run on an M1 or M2 Mac?

Not upstream: Splash requires an M3 or newer because its kernels use bfloat16 and Apple's MPP matrix library. This port rewrites those kernels for the M1 and M2, so the engine runs on Apple7 and Apple8 GPUs.

How fast is it compared to the original on an M3 or M4?

On npanj's five-prompt benchmark an M1 Max reaches 39.1 tok/s, against 60.7 tok/s for the same model on an M5 Pro. The M1 Max has 400 GB/s of memory bandwidth, so it keeps a large part of the newer chip's speed.

Which models does it run?

Qwen3.8-27B in 4-bit MLX and GGUF, and Qwen3.8-35B-A3B at up to 144 tok/s. It is the same one-model-at-a-time design as upstream Splash.

Should I use the app, the CLI or oMLX?

The desktop app (the .dmg button above) is for chatting with a model without a terminal. The CLI is one binary and an OpenAI-compatible server, which is what scripts and coding agents talk to. oMLX is a menu bar app and server that runs Splash as its external engine, so you get batching, an SSD KV cache and a dashboard on top of it. All three use the same splash-m1 binary underneath.

Does oMLX find Splash M1 automatically?

Yes, from oMLX 0.7.1. It looks for the launcher on PATH, then in /opt/homebrew/bin, /usr/local/bin and ~/.local/bin, and prefers splash-m1 over the official build on M1 and M2 Macs. Install Splash M1 first, then turn on "Use external engine" in the model's Advanced settings.

Does it change what the model outputs?

No. Every product is exact and only the order of additions changes: 99.92% top-1 token agreement with the reference path, and perplexity moves from 0.45044 to 0.45045.