01
Strata — a 125B model on a 12 GB graphics card, with quality described in words
Strata (MIT, C++, 12,523 stars, 845 commits, 68 contributors, created September 24, v0.1.39 on Sunday) runs Qwen3.8-Flash-Next, a 125B mixture-of-experts model, on a Windows or Linux gaming PC with 12 GB of VRAM and 32 GB of RAM. The model has 24,576 experts and uses 10 per token, so Strata keeps the busiest experts on the graphics card, all of them in RAM, has the processor work on the rest at the same time, and leaves a 29 GB lookup table on the SSD. A small draft model guesses ahead and the big one checks, which the README credits with 1.6-1.8x. On the README's RTX 5070 with 64 GB of RAM, it writes 94 tokens per second at Q2_0 and 53 at IQ3_S, and reads a 32K prompt at 1,620 to 2,650 tokens per second. It serves OpenAI's /v1, the Responses API and Anthropic's /v1/messages, so ANTHROPIC_BASE_URL=http://127.0.0.1:8080 points Claude Code at it. Windows users double-click START-HERE.bat, which downloads about 70 GB.
The HN thread (826 points, 365 comments) was mostly about what those speeds cost. The sizes are rated "good", "better", "great" and "best"; only IQ3_S is said to match the full model on published tests, and the Coder build drops half the experts to reach 91% of the full model's SWE-bench Verified score, measured by the people who pruned it. One commenter cited a paper finding that 2-bit quantization "often causes broad degradation", another said these projects "conveniently" leave accuracy out of their speed numbers, and a third, running Q1 and IQ2 quants in llama.cpp, saw the model invent URLs. On the other side, a user switched to IQ3_XXS and reported 60 tokens per second on Strata against 21 in llama.cpp, with better output. Asked why the expert caching isn't in llama.cpp, a commenter pointed to its rule that a human must understand any contributed code, which keeps AI-written kernels like these in their own forks.
Reach for it if you have a 12-24 GB card, 64 GB of RAM, and coding work you'd rather keep off a vendor's servers. Delete the cloud key your agent uses for the tasks you'd run locally if a local model were good enough. Tradeoffs: the fast sizes are 2- and 3-bit quants with no accuracy numbers beside the speed table; the PC can stop responding for 1-3 minutes while 35-55 GB loads; it answers one request at a time by default; and the suggested install is to paste the repo URL into your coding agent and let it follow docs/AI_SETUP.md.