01
Swiftlet — the 80B model that fits in 4.3 GB
A Swift + Metal runtime (Apache-2.0) for the Qwen3-Next and Qwen3.5/3.6 MoE families that keeps only the model's dense core resident and streams the routed experts from SSD on demand. The README's own table: the 35B runs in 2.6 GB of peak RAM at 7–11 tok/s on an M5 Mac; the 80B in 4.3 GB at 4.5–5 tok/s; the 35B also runs on an iPhone 17 in about 2.5 GB at roughly 1 tok/s. It ships repacked model containers on Hugging Face, a repack CLI that also converts raw MLX checkpoints, an OpenAI-compatible loopback server — and an actual App Store app. Created Sunday, 236 stars, 214 points on HN.
This is the fourth expert-streaming engine in five days (TurboFieldfare 07-30, WASTE and runNburn 07-31), and the watch asked whether the technique would consolidate into llama.cpp or get a head-to-head; instead it went down-market — the first engine targeting phones, and the first shipping as a consumer app rather than a repo. The honesty artifacts carry it: the README credits ANEMLL's 397B iPhone proof-of-concept as prior art, states that the decode loop is currently dispatch-bound rather than IO-bound, and volunteers the catch itself — about 3B parameters are active per token, so these models chat and write like large models but recall facts like small ones. The thread supplied the sharper objection: prefill, not decode, is the wall, with one commenter putting 10k tokens of context at half an hour on an M5.
Reach for it when you want a large open model on hardware you already own, for work that tolerates 5 tok/s — overnight batch jobs, private drafting, anything where custody beats latency. Delete the 128 GB RAM line item from the Mac you were about to spec for local inference. Tradeoffs: the thread's prefill estimate rules out agent loops and long-context coding today; ~3B active parameters is a real ceiling on recall, stated by the author; the repo is a day old; and the qpack containers are the author's own repacks, so you're trusting his conversion until you run the repack CLI on a checkpoint yourself.