01
Soup — fine-tune an 8B on a 4 GB laptop GPU
A Python CLI (Apache-2.0, pip install "soup-cli[train]") that fine-tunes language models on hardware nobody would call training hardware. The mechanism shipping in v0.72 is layer streaming: the frozen base model sits in system RAM as 4-bit NF4 weights and is fed to the GPU one decoder layer at a time, with two ~113 MB VRAM buffers doing all the compute. Measured, not projected: Llama-3.1-8B trains at 119.6 tok/s with a 3.32 GB peak on an RTX 3050 Laptop with 4 GB of VRAM, and the raw measurement records sit in the repo's benchmarks/ directory. 130 points on Show HN, 240 stars — and a hundred-plus PyPI releases since March 2, so this is a five-month-old tool crossing our pool for the first time, not a launch.
The streaming thread (TurboFieldfare 07-30, WASTE and runNburn 07-31, Swiftlet 08-04) spent a week proving you can run big models from storage they don't fit in; Soup is the first to carry the trick across to training, and it arrives with a stricter verification standard than anything the inference side published: bit-exact logit matching against a resident reference — maximum difference 0.0 — across nine architectures and two precisions, running as CI. The author's reason is the best line in the thread: "streaming fails silently." Loss keeps going down while autograd is quietly broken, so a convergence curve proves nothing. The thread also holds the honest boundaries: embeddings plus one layer plus logits must fit resident, so 14B doesn't go; streaming overhead measured 1.43x against resident training at the one size where both fit the card; GRPO and PPO are excluded on purpose, because generation re-reads every layer per token.
Reach for it when you have a dataset, a laptop, and no appetite for renting an A100 to make an 8B speak your schema. Delete the assumption that fine-tuning starts at a 24 GB card, and the cloud-GPU line item for adapter-sized jobs. Tradeoffs: layer streaming is labeled BETA by its own author; DPO pays 1.52x layer reads for its reference model; what fits in 4 GB is QLoRA adapters on quantized bases, not full fine-tunes; and the author disclosed mid-thread that his early replies were LLM-drafted before he switched to writing his own — the disclosure is to his credit, and it still tells you how young this project's community is.