We ran the benchmark we kept saying we would run, and it returns 74 against 8.

A runtime gate for agent tool calls: pip install doberman-core, Apache-2.0, Python 3.11+, deployable as an MCP proxy wrapping tool servers or as native pre-execution hooks into Claude Code, Codex CLI and OpenClaw. Every call resolves to exactly one verdict before it runs — PASS, AUTH (pause for a human), BLOCK — with "fail closed" and "raise-only learning" stated as non-negotiable. Yesterday we put it top of the slate for its documentation. Today it stays there for a better reason: the documentation is true. From a cold clone at commit 7cd8c91, pip install -e ".[dev]" then python -m tests.benchmarks.run --corpus reproduces the published table cell for cell. Secrets 1.00 TPR on 7 rows. Destructive 0.80 on 10. Encoded/smuggling 0.82 on 79. Exfiltration 0.375 on 8. Natural-language injection 0.00 on 8 — zero detected, exactly as their docs say. Overall 0.741 across 112 attacks, false-positive rate 0.00 on 25 benign rows, precision 1.00. This is the first pick this newsletter has carried whose vendor number we checked ourselves, and it held.

What the reproduction adds is the part a reader would not get from the table: how wide the gap is between "caught" and "stopped." Overall tpr_strict — BLOCK only, no human involved — is 0.080. In the encoded category, 65 of 79 attacks are detected and zero are blocked. The synthetic before/after run is starker still: with no guardrail, 3 of 3 attacks execute; with Doberman, 3 of 3 are mitigated and 0 of 3 are blocked, so attacks_stopped reads 1.00 and attacks_stopped_strict reads 0.00. The same run computes what happens to that if the operator is tired — asr_under_fatigue 0.80, and effective_asr_approve 1.00, meaning a human who approves everything gets exactly the security of having installed nothing. Doberman ships that metric itself. Set it beside the 66.3% approval accuracy across 409,000 clicks from our 08-07 edition and Anthropic's own figure that users approve 97% of permission prompts (08-10), and the honest reading of a 0.74 detection rate is closer to a tenth of that in practice. None of this is a knock on the project — it is the project's own framing, published before anyone asked, and the reason it is still the top pick is that we could only write this paragraph because they made it reproducible.

Reach for it if agents hold credentials against something you care about and your current control is a permission prompt at 4pm. Delete the hand-rolled regex denylist in your hook script, and delete the belief that an approval dialog is a control rather than a queue. Tradeoffs: tpr_strict 0.08 means almost all of the protection is contingent on a human actually reading and refusing; natural-language injection scores a flat 0.00 and is a structural gap in the deterministic layer, not a tuning problem; the AgentDojo suite needs an operator-supplied package, so only synthetic and corpus reproduce from a cold clone, and the AgentDojo results table in the docs is still an empty placeholder; the corpus is 137 rows with synthetic payloads authored by the same project, so this measures the engine against its own threat model rather than against the field; 210 stars, 23 forks and zero watchers with 43 open issues; PyPI still reports no license field while the repo is plainly Apache-2.0; and the repo was pushed at 10:18 UTC this morning, so pin the version you measured.

Electron's safeStorage writes to the legacy file-based macOS Keychain, where access is governed by per-item ACLs. That is the mechanism behind the dialog everyone has clicked: an app wants to read an item, macOS asks, you say yes because you are in the middle of something. keychain-store moves Electron and Node apps onto the modern Data Protection Keychain instead, passing kSecUseDataProtectionKeychain: true to SecItem, where access is decided by code-signing entitlement rather than by a prompt. Either the binary asking for the secret carries the right team identifier and entitlement, or it does not get the secret. Nobody is asked. As a side effect, items stop being dumpable via the security CLI, which is the standard way a script — or an infostealer, or an agent with shell access — walks the legacy keychain. pnpm add keychain-store, MIT.

This is first on the ladder because it is the cleanest example of the thing today's top pick measures the absence of: a check whose answer does not route through a human's attention. There is no fatigue parameter for an entitlement. It also has options for when you do want a person in the loop, and they are explicit rather than ambient — biometrics-only gates on Touch ID, user-presence on device-owner authentication, none on neither — which is the right shape, because the prompt becomes a deliberate choice for the few secrets that warrant one instead of the default for all of them. The relevance to this audience is direct and slightly uncomfortable: a large share of the local AI tooling shipping right now is Electron, holding provider API keys, on the default path.

Reach for it if you ship an Electron or Node app on macOS that stores an API key, OAuth token, or anything a support agent would call "your credentials." Delete safeStorage and delete the assumption that the keychain dialog is doing access control rather than collecting a reflex. Tradeoffs: macOS only, and it requires a valid Apple code signature, so unsigned dev builds and anything shipping outside the Apple signing path cannot use it; it does not auto-migrate existing legacy Keychain items, so you own the migration and the fallback logic; biometrics-only fails outright on Macs with no enrolled Touch ID, which you will discover in the field rather than in CI; it requires a user-login context where the legacy keychain did not, which breaks daemon and background-service use; and the repo was created yesterday — 9 stars, 1 fork, zero watchers, no track record whatsoever, so read the source before it holds anything real.

Analytics that runs entirely inside one Cloudflare Worker and one D1 database on your own domain, MIT, deploy button or npx wrangler deploy, 2.1 KB script tag. The claim worth reading twice: "No third-party requests from any browser — not on your site, not in the dashboard. The fonts and the world map ship inside the Worker." Visitors are identified as sha256(daily_salt + site_id + ip_address + user_agent) with the salt destroyed daily, so there is no cookie, no fingerprint, and no stable identifier at rest to subpoena or leak. The engineering underneath is the fun part and is genuinely clever about D1's pricing model: raw events land in per-hour tables with no indexes at all, one write per pageview, and those tables are never deleted from — they are rolled up into daily aggregates and then DROPped, because DROP TABLE is DDL and costs nothing against a row-write quota. That gets to roughly 1.2 row-writes per pageview and about 20,000 visits a day inside the free tier.

Second rung for the same reason as the first: the guarantee is structural, not procedural. A consent banner is the purest form of the pattern this edition is about — a real decision, delegated to a tired person, thousands of times a day, with a known answer. Edgemetry's response is to remove the thing being consented to. It is also the direct constructive answer to our 08-17 lead, where a developer switched nameservers and found 31KB of beacon.min.js in a site with no JavaScript, on by default, with the dashboard requiring you to enable Web Analytics before it would let you disable it. Same vendor's platform, opposite default. And the README does the thing that has been earning picks all quarter — it lists what it gets wrong, at length and unprompted.

Reach for it if you want per-site numbers, own the data, and would rather not add a cookie banner to a blog. Delete Google Analytics, delete the consent modal, and delete the monthly Plausible or Fathom line if your traffic fits the free tier. Tradeoffs: multi-day unique-visitor counts are summed daily totals, so a returning reader is counted again each day and your "uniques" number is wrong in a specific, upward direction the README names outright; no per-user timezone; filters narrow by entry point rather than all pages in a visit; time-on-site excludes the final page because there is no unload beacon, which biases every session short; screen sizes bucket into four ranges; login attempts are unthrottled and the docs tell you to add edge rate-limiting yourself, which is a real gap you must close before exposing the dashboard; ~20k visits/day is a hard practical ceiling before Cloudflare's free D1 quotas bite; and it is 37 stars, 4 forks, zero watchers, created three weeks ago.

Every local-model workflow contains the same unmeasured judgment call: you have 24 GB, someone says try Q4_K_M, you try it, it feels fine. Shoehorn replaces the guess with an optimization. Give it a hardware budget and it computes a per-tensor mixed-precision assignment targeting near-total budget utilization — routinely 99.99% — rather than applying one uniform quantization level to the whole model, then emits standard GGUF v3 that llama.cpp loads unmodified. The quantizer is written from scratch in Rust with no llama.cpp code linked. macOS on Apple Silicon, Linux x86-64 with NVIDIA or AMD, Windows x86-64 with NVIDIA; Homebrew, a release binary, or cargo. MIT, with a GUI for finding and building models if you would rather not think about any of this.

The reason it sits on this ladder rather than in the footer is one interface decision: the tool displays a perplexity number for what the fit cost you. That is a small thing that quietly changes the category. The standard quantization experience returns a file and leaves quality assessment entirely to the user's impression of a few completions — which is a human check, unmeasured, performed while distracted, and therefore the weakest kind. Attaching a number to the tradeoff at the moment the tradeoff is made is the same move keychain-store makes with entitlements and Edgemetry makes with third-party requests: take the decision out of the tired person's hands and put it somewhere it can be inspected. It is the weakest instance of that on today's list, because a per-fit perplexity readout is a diagnostic and not a published benchmark, and the blurb should say so rather than dress it up.

Reach for it if you run local models on a machine whose memory does not match anyone's recommended quantization tier, or if you have been picking quant levels by folklore. Delete the guesswork about which Q suffix your card fits and delete the second and third re-download when the first guess did not. Tradeoffs: there is no published benchmark — no comparison of Shoehorn's mixed-precision assignment against standard llama-quantize presets at matched memory, on any named model or dataset, which is exactly the artifact that would prove the whole premise, and its absence means the central claim is unverified; the perplexity readout characterizes your own fit and is not a quality guarantee; llama.cpp/GGUF only, so vLLM, MLX and TensorRT users get nothing; 99.99% budget utilization is a statement about packing efficiency, not about output quality, and conflating the two is the easy misread; and it is 43 stars, 4 forks, zero watchers, created six days ago, pushed at 00:19 UTC this morning.

A free in-browser scanner for the artifacts AI chat interfaces leave in text you paste around: hidden HTML class names containing "claude" from the web UI, zero-width joiners and byte-order marks, non-breaking and exotic spaces, smart punctuation, and sentence-length uniformity. It returns exact counts and byte offsets for each finding. No account, no upload, detection engine MIT on GitHub. On the face of it this is a small utility. It is here for what it declines to do.

Our 08-17 edition opened on Anthropic's text watermark — a keyed bias on word choice, applied to responses over 200 tokens, readable only with Anthropic's keys — and set a watch: the completing artifact is anyone publishing a detection result against the actual scheme. This is the closest thing the pool has produced, and it resolves the watch in the negative, on purpose, in the vendor's own marketing copy: "Anthropic's statistical watermark is a keyed bias on word choice, and verifying it needs a key that has not been released to anybody. Any tool claiming to detect or confirm it is guessing." A product named Claude Watermark, whose entire growth path runs through people searching for a Claude watermark detector, leads with the fact that it is not one. Then it goes after the heuristic its own category is built on, with a measurement: "We measured ten pre-computer novels and found the rate swings from zero per thousand words in Austen and Stoker to twenty-six in Melville." That is an empirical refutation of the em-dash tell, published by the party who profits most from you believing it. Compare the honest structure here — evidence with byte offsets, no probability score, no verdict — against the entire AI-detection industry, which sells a confident percentage for exactly the inference this tool says cannot be made.

Reach for it before pasting model output into a CMS, a PR description, or anything that will be diffed, and use it to strip invisible characters that break your linter or leak provenance. Delete any AI-detector subscription that returns a confidence percentage, on the grounds that this free page explains why the number is fabricated. Tradeoffs: it finds artifacts of a chat interface, not authorship — clean text proves nothing, and paste-through-a-text-editor defeats every signal it looks for; the sentence-length-uniformity check is the same class of stylistic inference the page correctly debunks elsewhere, and it is applied without a published false-positive rate; smart punctuation and em dashes are counted despite the tool's own data showing they prove nothing, which invites the misreading it argues against; the GitHub repo has 0 stars, 0 forks and 0 watchers, was created four days ago and has not been pushed since 15 August; and the hosted page and the MIT engine are separate artifacts, so what runs in your browser is not necessarily what you can read.

Visual UI testing where an agent drives a real Playwright browser and interacts like a user — clicking, typing, scrolling — with no scripts and no CSS selectors to maintain. A five-stage pipeline (Validator, Comprehender, Explorer, Strategist, Executor) runs on Gemini 2.5 Flash by default, capturing screenshots, timelines and structured reports. The positioning is precisely today's thesis stated as a product: agents write more UI than QA can cover, so point an agent at it. Setup is uv sync --dev, uv run playwright install chromium, a frontend build and a GEMINI_API_KEY.

It closes the list because it is the honest bottom of the ladder, and this is a placement rather than a dismissal. Every pick above it moved a decision somewhere inspectable — an entitlement, an absent network request, a solver, a byte offset. Argus moves it to a model, which is a legitimate answer to a problem that deterministic selectors genuinely have not solved, but it inherits every property the top pick measures. There is no published benchmark: no pass rate, no flake rate, no false-positive rate, no comparison against a scripted Playwright suite on the same app. So the question "how often does the tester approve a broken page?" has the same answer today that "how often does a human approve a bad tool call?" had before somebody measured it — nobody has looked. This is the category the 08-10 comment nailed and it still applies: if something is truly a rule, there should be code that deterministically enforces it. A visual regression is not a rule, which is the argument for this approach; but a test suite with no measured reliability is a check whose result you have to take on faith, and that is where the whole edition started.

Reach for it if you have exploratory UI coverage at zero and a scripted suite is not going to get written this quarter. Delete nothing yet — run it alongside whatever you have rather than in place of it. Tradeoffs: no accuracy, flake or false-positive numbers published at all, which for a testing tool is the one metric that matters; it is not open source — GitHub reports NOASSERTION because the license is ASAL-1.0, source-available, free under 100 members, commercial license required for enterprises, converting to MIT after three years, so read it before it enters a company repo (the seventh time this quarter a license badge has misreported, and the second where NOASSERTION concealed something less open than it looked); every run bills Gemini tokens, with no cost-per-run figure published; the README states there is no authentication or multi-user isolation in the current release, so do not expose the service; and it is 25 stars, zero forks, zero watchers, created two days ago.

Also on the board. Turbovec (15.5k⭐, MIT) is the strongest artifact we did not pick — a Rust implementation of Google Research's TurboQuant with real reproducible benchmarks against FAISS on GloVe and OpenAI embeddings, 3.4× QPS at 4-bit, a 10M-document corpus going 31 GB to 4 GB, and disclosed losses where it trails at 2-bit and low dimension. It lost on theme, not merit. Worth one caveat before you install: 15,538 stars against 65 watchers and zero releases is the same ratio that made us flag Paperclip on 08-10. • Avouch — "I canceled my AI code reviewer and wrote a free local one," 22 points, 24 comments — was a pick until the thread was read. It contains no AI: 17 AST rules over changed files. Fine idea, but commenters asked what it adds over bandit, ruff and black's differential flags and got no answer; benatkin noted it "seems to be mixing up linting with code review"; ramon156 called the README "fully generated… a bunch of sentences that say nothing" and the author conceded "I just formatted and fixed it using ai"; doc_ick found it is a repost after the original account was flagged; and the author's roadmap is "later its gonna have ai intergartion as well." The tool that cancels the AI reviewer plans to become one. • Mojo is now open source under Modular, which Qualcomm acquired. • Solo, a .so loader for static Linux binaries, took 404 stars in five days. • Impersonate-proxy (MIT) does JA3/JA4 and HTTP/2 fingerprint control via uTLS presets for authorized WAF testing — and publishes no evidence its fingerprints actually match the browsers it names, pointing you at tls.peet.ws to check yourself.

Reproduction note: Doberman figures above were produced locally on 2026-08-19 from commit 7cd8c91, --suite corpus and --suite synthetic --profile before_after, balanced mode, no network and no model calls. They match the project's published table exactly.