01
Janas — runs mixture-of-experts models bigger than your RAM by streaming experts off the SSD, and prints the speeds that cost it
Janas (GPL-3.0, C, 229 stars, one contributor, repo created September 20, 164 points on HN Monday evening) is an inference engine in C with no dependencies beyond the C library. Its first component, Janas-LLM, keeps the weights on disk, streams the experts each token needs from an NVMe drive and keeps the ones that come back in memory. Linux and Windows x86-64 builds are on the releases page, and ./janas-chat lists the models it knows, downloads one and checks it. On the author's laptop (Core Ultra 9 185H, integrated Arc GPU, 32 GB, one NVMe SSD) it wrote 24 tokens a second from the 48 GB Qwen3-Next-80B against llama.cpp's 2.8, or 37 with the model's own draft block, and beat llama.cpp on all 13 models in its Q4_K_M table. The exception sits in a second table: unsloth's 3-bit Qwen3-30B reads prompts slower than llama.cpp (100 against 112) while writing three times as fast.
The README is unusually plain about method. Each figure is the mean of three rounds, the llama.cpp build and flags are listed, and the 80B comparison needed a llama.cpp build without weight repacking just to load. The draft-speed column was wrong, and too high, until September 26, and the README says so and explains why. janas-bench measures your machine, and the author asks for those reports because the engine has run on exactly one laptop. In the thread, one commenter said there are many stream-experts-from-SSD projects now and another pointed to Strata. A third argued that expert caching is model- and prompt-dependent and "doesn't look like a solved problem yet." A fourth read the catalog's older Qwen3 models as a sign the code was written by an agent that didn't know the newer ones.
Reach for it if you have an x86 laptop, a fast SSD and a mixture-of-experts model that doesn't fit in RAM. Delete the plan to buy 64 GB for a model you'd run twice a week. Tradeoffs: 62 tokens a second of prompt reading on the 80B is slow for agent work; the README warns the engine fills free memory and can push a busy desktop into swap, and that a GPU driver reset once took the desktop down; AMD CPUs, discrete GPUs and Apple Silicon are untested or unsupported; the command line changes several times a week; and janas-get pulls conversions from the author's Hugging Face account, though converting a GGUF yourself is documented.