← AI Hacker Daily

Edition

03

picks

Two frontier models launched today, and the menu got bigger and pricier — not simpler.

Two frontier models launched today, and the menu got bigger and pricier — not simpler. GPT-Live and Grok 4.5 both shipped, and the eval crowd spent the day arguing about measurement (OpenAI's "separating signal from noise in coding evals," Databricks benchmarking agents on a multi-million-line codebase). The more practical question for anyone paying the bill is which call needs the frontier at all. Today's two picks answer it without asking you to route your traffic through a new company: Frugon reads your logs and shows where the bill leaks; Foreman is a gateway you run yourself that sends each call to the cheapest model that can handle it. Both are pointedly honest about savings — Foreman refuses to quote a number, Frugon flags its own estimate as unverified until you measure — which is exactly the tell that separates them from the hosted proxies in the footer. Dropped as news or launches: GPT-Live, Grok 4.5, TypeScript 7, the Bun-in-Rust rewrite, both eval essays. The hosted trading desks (Auriko, Opper, Gate) and Microsoft's Flint are down below.

01

Frugon — find where your LLM bill leaks before you pay it

A free, MIT, fully local CLI that reads your LLM call logs and tells you which calls a cheaper model could have handled. It runs entirely on your machine — no network, no LLM calls, just tokenizing, applying provider pricing, and doing arithmetic — so nothing about your prompts leaves the box. Feed it two ways: point it at existing OpenAI-format JSONL, or run `frugon capture` as a local proxy that logs your traffic with no added latency. `uvx frugon analyze ./logs.jsonl` and you get a per-model breakdown; the demo run flagged $205/mo in potential savings (37%) on a $549 baseline, using the 30–50% range from the RouteLLM research it's built on. Reach for it before you touch any gateway config — it's the audit that tells you whether routing is even worth it for your specific workload, rather than assuming it is. Delete the eyeball-the-dashboard guesswork about which endpoints are burning tokens. The honest part is the caveat Frugon writes itself: "quality is not verified — 'within tolerance' is an offline estimate; run `--measure` to confirm it on your real outputs before you switch." That `--measure` flag samples your traffic through candidate models using your own keys sent straight to the provider, never through Frugon — the verification step most cost dashboards quietly skip. Tradeoff: three weeks old, 53 stars, and the savings it estimates are a starting point to test, not a promise.
github.com/Rodiun/frugon

02

Foreman — a gateway you run yourself that buys the cheapest model that works

A single Go binary (Apache-2.0) that sits between your coding agents and every provider, keeping your keys and traffic inside your own network. It routes on a three-tier cheap/mid/frontier system: policy floors force high-risk tasks to a frontier model, `X-Foreman-Task-Type` / `X-Foreman-Risk` headers let the caller hint, and otherwise it "picks the cheapest healthy model the tier allows." It translates protocols (Claude-format clients can run against OpenAI-compatible upstreams with full tool-call translation), pins a session to one model so you don't thrash the prompt cache, and fails over on a circuit breaker. Every request writes a ledger entry with measured tokens and cost, and `foreman trace <id>` shows exactly why it picked what it did and what it rejected. `brew install northwood-systems/tap/foreman`, then `foreman init && foreman serve`. Reach for it once Frugon's audit says routing is worth it and you'd rather not hand a hosted proxy your keys and prompts. Delete the OpenRouter-or-LiteLLM-plus-a-hosted-dashboard stack where a third party sees every request. The tell that it's built by someone serious: it "never publishes a savings number" — the cost figures come from your actual ledger, not a marketing estimate. Tradeoff: this is one day old (created yesterday, 16 stars) — the freshest bet on the slate. The routing and failover read production-minded, but nobody's run it at scale yet, and anything that can't cross protocols (images, documents) gets an explicit 400 rather than a silent flatten — correct behavior, but you will hit it.
github.com/Northwood-Systems/foreman

03

The hosted alternative to Foreman is a crowded desk, and today three of them launched. **Auriko** (auriko.ai) bills itself as a "trading desk for LLM calls" — ex-quant-trader founders scoring every provider on cost, latency, throughput, and success rate, claiming ~30% off; closed and hosted, your traffic through their box. **Opper AI** (opper.ai) is the grown-up version: one key to 300+ models, EU-hosted with no prompts stored by default, per-team spend caps, 50k+ developers, no token markup (3% on top-ups) — also closed, but a real company with real traction. **Gate** (Constellation) is the security-first entry — #1 across 16 prompt-injection benchmarks (97.4% F1, methodology on arXiv) with lossless prompt compression bolted on for a claimed 20–50% token savings — again a hosted proxy you point `base_url` at. All three want to sit between you and your model; that convenience is the tradeoff. Off the thread but worth knowing: **Flint** (github.com/microsoft/flint-chart, MIT) is Microsoft Research's intermediate visualization language — an agent emits a compact, human-editable chart spec and Flint compiles it to Vega-Lite, ECharts, or Chart.js, deriving the scales, axes, and spacing so the model stops hand-tuning low-level config and produces charts that render reliably and stay editable. `npx -y flint-chart-mcp` drops it in as an MCP server, or `npm install flint-chart` for the library. The cleanest first-party answer yet to "my agent's charts come out broken."

One of these,
every weekday.

Free. Unsubscribe by replying with one word. No tracking pixels in the email.