The weights got announced; the bill underneath them is still yours.

A Rust storage engine built on io_uring whose headline use is boring in the best way: it moves the KV cache off the GPU into host RAM and NVMe, and it does so without touching your serving code. Install the connector, run openlaked, then start vLLM with a --kv-transfer-config block naming the OpenLake nodes — that is the whole integration. The payoff is prefill you stop repeating: the inference engine writes a prefix once and reads it back in milliseconds, and in a multi-host configuration a prefix computed on one GPU box is served to every other box from the shared pool over RDMA. The project's own benchmark puts time-to-first-token at 66x faster on a cached 128K-context request, serving Gemma 4 31B on an H100, and the Show HN frames the same mechanism as roughly halving long-horizon inference cost. The rest of the engine is the unglamorous fleet plumbing that comes with owning models: an S3-compatible object store for checkpoints, fast small-file reads for training, and a place to park the massive conversations agents accumulate.

Reach for it when you self-host inference and the same system prompts, repo context, and conversation histories get re-prefilled all day across a fleet. Delete the line item where the fix for "the KV cache doesn't fit" was another accelerator. Tradeoff: every number here is the project's own, on their hardware, and the benefit scales with how much prefix you genuinely reuse — a fleet of cold, unrelated prompts recovers close to nothing, and you are adding a distributed storage daemon to the list of things that can page you at 3 a.m. Apache-2.0, 2,289 stars since April.

An Apache-2.0 runtime that treats local inference as a system service rather than an app: it serves models on your own hardware, speaks OpenAI-compatible on port 6777 so Aider, Continue, LiteLLM, or any OpenAI SDK points at it unchanged, ships its own Wayland compositor, and sets its floor at 8GB of RAM. The VRAM manager checks fit before anything loads and places each model on gpu, cpu-offload, or cpu-only; a 10GB-plus CUDA card unlocks speculative decoding with a 0.8B draft model, which the project's own tier text puts at roughly 40% faster replies. The genuinely interesting part is what happens when the local model is the wrong tool: a turn it should not take is handed whole to a peer whose model is bigger — not a draft sent for review, the whole turn — with capability advertising opt-in per node and a fall-through to local when nobody on the network has opted in. That is routing again, the thread we have been following since 07-24, but pointed at a neighbor's GPU instead of a vendor's meter. The README is the reason a 44-star repo earns a slot: it names the file implementing each claim, and it documents its own install failure — 24 pinned packages have no Python 3.12 wheel, with the issue number for the fix — rather than letting you discover it.

Reach for it when most of your day's questions do not need a frontier model and you would rather they never left the building. Delete the subscription for the majority of turns that were never hard. Tradeoff: 44 stars and 5 forks is a rounding error of adoption; requirements.txt pins 191 packages and pulls torch, transformers, and onnxruntime, so budget gigabytes on first run; and the federation story is only as good as your neighbors — on a network where nobody advertises, this is a local model with a great deal of extra machinery. The Nunba desktop frontend does ship signed installers for Windows, Linux, and Android, built today.

The agent traces you already collect in OpenTelemetry are, in this tool's framing, an unspent measurement. wmo build ingests them; wmo optimize route sweep scores every model you have registered against held-out tasks drawn from your own traces; route fit turns those measurements into a k-nearest-neighbor routing policy; wmo serve puts an endpoint in front of it; and route report --baseline gpt-5.5 tells you what the swap actually bought, which is the step most tools in this category skip. The claim is frontier-quality output at 40-plus percent lower cost, and the pool it routes across can include OpenRouter's full published catalog alongside anything you self-host. Two more verbs sit beside routing: optimize distill trains a small specialist from frontier open models and drops it into the pool, and optimize harness optimizes the scaffold itself — which is a direct, empirical reply to the "harness engineering is not enough" argument that trended alongside our 07-24 slate. Against Echo that day, which allocated across a pool using a general policy, the distinction is the whole pitch: this one refuses to guess, and fits the policy to your traffic.

Reach for it when you have months of agent traces sitting in an observability bucket and a model choice nobody has re-derived since it was made. Delete the routing heuristic that was somebody's hunch in a config file. Tradeoff: there is no license — not in the API metadata, not at the repo root, not in pyproject.toml, not on PyPI, where the only LICENSE in the tree covers a vendored dependency — so "open source" here describes code you may read, not code you have permission to use, and that is a one-line fix the authors have not made. Beyond that, the CLI has a wmo login to a hosted platform and an E2B backend for evaluation, the 40% figure is the vendor's own, and a routing policy fitted to last quarter's traces is a policy that ages.

The day's biggest mover in GitHub trending, and the closing line item on the bill: autonomous penetration-testing agents that run your code dynamically, find vulnerabilities, and validate them with working proofs-of-concept rather than the false-positive drizzle of static analysis. It ships a full toolkit — reconnaissance, exploitation, validation — orchestrates teams of agents against a single target, generates remediation patches and compliance-ready reports, and, in the addition that matters most here, runs as a GitHub Actions step so insecure code gets blocked at the pull request instead of discovered at the annual audit. Full disclosure: this crossed our pool on 06-28 and is not new — it is a rocket from August 2025 that gained 490 stars today, and the question worth asking is why now. Two days ago this same pool carried a UK AISI and CAISI preliminary assessment of Kimi K3's cyber capabilities and a report of K3 exploiting a current Redis server. Cheap frontier-class weights are not a one-sided event, and the honest version of every argument above is that the person probing your app is reading the same release notes you are.

Reach for it when your only adversarial testing is a scheduled engagement and everything shipped between engagements goes unexamined. Delete the assumption that the quarterly pentest is the security budget. Tradeoff: this is an autonomous exploitation tool, so point it only at systems you are authorized to test, and understand that a multi-agent pentest against a real application is itself a substantial inference bill — which is the joke this entire edition has been telling. Apache-2.0, 44,791 stars, installable as strix-agent.

Off the thread but worth knowing: HeyZoku (closed, macOS, one-time purchase, #22 on Product Hunt at 71 votes) orchestrates fleets of Claude, Codex, and Cursor by voice — spawn, redirect, and interrupt agents while your hands are elsewhere — with speech handled on-device by NVIDIA Parakeet. That is the 07-20 voice-stack watch landing, though not in the form it predicted: the watch expected a harness to wire push-to-talk natively, and instead the local STT engines got wrapped in a commercial layer above the harnesses. wmux (MIT, 295 stars since March, wmux.app) runs fleets of Claude Code, Codex, and Gemini in parallel panes, or fans one prompt into N isolated git worktrees you review hunk by hunk, with approval gates, agent-to-agent channels, a browser the agents drive, and a daemon that brings the whole workspace back mid-conversation after a full OS reboot — the cohabitation thread (07-22, 07-23) with an unusual property for this category: it is native on Windows, not macOS-only. OpenWork (17,287 stars since January, MIT except the /ee directory under a Fair Source license, stated cleanly in its own LICENSE file) is an open-source alternative to Claude Cowork built on opencode, and two details stand out: its MCP exposes exactly two tools, search_capabilities and execute_capability — the progressive-disclosure pattern again, after Conduit (06-23) and Ratel (07-17) — and its install instructions are a prompt you paste into your existing agent, which is the procurement-by-agent thread (07-16, 07-17) arriving at the desktop app. Also noted and left alone: scriptc (Vercel Labs, Apache-2.0, 1,160 stars in five days, 195 points) compiles ordinary TypeScript to a 178KB native binary with no JavaScript engine inside — off this edition's axis entirely, but the third Vercel Labs repo to cross our pool in ten days after just-bash (07-17) and deepsec (07-20).