← AI Hacker Daily

Edition

05

picks

The vendor ships the API; today's picks go one layer below it.

The vendor ships the API; today's picks go one layer below it. On Wednesday, Qualcomm announced an early developer preview of Linux on Snapdragon X2, which is the vendor-approved version of this story: the chipmaker decides its silicon should run your OS, then ships the support. The rest of today's slate is the other version. Intel's laptop NPU has programmable cores that its public software will not let you program, so one developer worked out the path from C source to NPU machine code. NVIDIA's consumer driver offers no way to share one card between virtual machines without vGPU licensing, so a small team forwards the driver's own ioctls into the guest and lands within 2% of bare metal, then builds a one-second microVM on top. stable-diffusion.cpp keeps putting each new image model on six GPU backends with no Python stack, most recently Qwen Image 2.1 on its release day. An essay on yesterday's front page argued that tokens are getting too cheap to meter and that "quality and access" will become the limit instead. These picks are what the access half looks like at the hardware layer, and the useful thing is that each of them states plainly what it gives up to get there.

01

virtio-nvgpu — one consumer NVIDIA card, four VMs, within 2% of bare metal, and no IOMMU between them

virtio-nvgpu (Rust device crate under Apache-2.0, GPL-2.0 guest kernel module, BSD-3-Clause-or-GPL protocol headers; 161 stars, 107 commits, 2 contributors, created March 31, no tagged releases, marked experimental) gives a Linux KVM guest near-native access to an NVIDIA GPU the host keeps using. Existing options force a choice. VFIO passthrough gives one VM the whole card and takes it away from the host. virtio-gpu with Venus shares the card but serializes every Vulkan call in the guest and replays it on the host, which the README puts at around 2,000 boundary crossings per frame, and it leaves buffers host-owned so the guest cannot encode video on the GPU. The DRM native context path that fixes this on Intel and AMD does not exist for NVIDIA. virtio-nvgpu forwards at the kernel-driver level instead: the guest module registers /dev/nvidia* and passes ioctls and mmaps across a virtqueue, and the guest runs NVIDIA's own user-mode libraries unmodified, building command buffers locally. Measured on an RTX 3060, any frame that takes the host 2 ms or more renders within 2% of bare metal in the guest; a guest used 0.37 s of CPU against the host's 0.40 s over the same run; and over 813,691 frames the backend handled one message per 59 frames. Four guests on one card split 103.7 fps evenly, and four encoded H.264 through NVENC at 60 Hz at once.

The security section is the reason to trust the rest. It says there is no IOMMU boundary between guest GPU work and the host: the card stays in the host's IOMMU domain, the GPU's own MMU is what separates a guest from host memory, and so the host NVIDIA driver is in the trusted computing base. Ioctls the ABI profile does not describe are refused, but resource-allocation classes, control commands, UVM and modeset ioctls are not yet filtered, and the planned per-guest sandboxed helper exists only as a design note. The README's own words: "attack-surface reduction, not hardware isolation." Its benchmark file has a "What these numbers do not support" section too, with no comparison against another hypervisor, four guests rather than many, synthetic vkcube loads rather than a real game, and one card on one driver. In the HN thread (116 points), a commenter asked how it differs from gVisor's nvproxy, which the README names as its inspiration. The author said they borrowed much of nvproxy's architecture, built it out for graphics workloads, and made it reusable in other VMMs. Asked about Windows guests, the author said not yet, but they are on the roadmap.

Reach for it if you run Linux VMs that need a real NVIDIA GPU for rendering or encoding and you own one card, not two; it replaces the second GPU bought for passthrough, or the vGPU license a GeForce card cannot use anyway. Delete the plan to buy a second card just so the host keeps a display. Tradeoffs: this is for trusted guests only, by its own account, and for mutually untrusted tenants VFIO or vGPU is still the answer; Linux guests only; CUDA is forwarded but not tested beyond enumeration; shipped ABI profiles cover driver 535.129.03, 580.178.04 and 595.71.05, with anything older refused, and the benchmarks come from one RTX 3060 on 595.99.02; and two contributors carry a kernel module that the host driver trusts.

→ github.com/nestrilabs/virtio-nvgpudiscussion ↗

02

nesbox — the microVM that ships with the GPU already attached, booting in about a second

nesbox (Apache-2.0, Rust, 25 stars, 82 commits, 3 contributors, created April 22, no tagged releases) is the VMM the virtio-nvgpu team built to use it, and it is where the device stops being a research result and turns into something you can run. Every nesbox guest gets a virtio-gpu device, always attached. On Intel and AMD it goes through rutabaga_gfx over DRM native context, so Mesa in the guest talks straight to the host driver and the guest needs no GPU driver of its own. On NVIDIA it uses virtio-nvgpu. The README reports 99–102% of bare-metal frame time on an RX 9060 XT at 3.4 ms per frame and above, 98–100% on the RTX 3060, and boot to userspace in about a second. It gets there by loading an ELF vmlinux directly, with no bootloader and no initrd, and by keeping every device except the GPU in the host kernel (vhost-net, vhost-vsock) or in virtiofsd. The stated use is cloud gaming: boot a VM, hand it a game, stream the result.

The README's first section is an "Is this what you are looking for?" test, and the no side is specific. It is not a general-purpose hypervisor, not a desktop VM with a display attached, does not run Windows guests, and does not pass a whole card through to one VM. Building your own guest kernel and Mesa is described as the point, not a fallback. The benchmark notes apply the same discipline as virtio-nvgpu's: no other hypervisor was benchmarked, "so nothing here says 'faster than' anything." Host networking needs one privileged setup script, and after that nesbox itself runs with no added capabilities, because a tap owned by the running user can be opened unprivileged.

Reach for it if you want short-lived Linux VMs that each need a GPU, for rendering, encoding or a headless workload, and a container is not enough of a boundary; it replaces a Firecracker setup with no GPU or a QEMU config with a full card passed through. Delete the one-card-per-VM rule from your capacity plan. Tradeoffs: you build the guest kernel and Mesa yourself, and for AMD or Intel you also build libvirglrenderer with native context for your GPU; Linux host with KVM only; on NVIDIA it inherits every isolation caveat above; and it is three contributors and 25 stars, posted to HN overnight, at 15 points.

→ github.com/nestrilabs/nesbox

03

npunlock — your own C code on the Intel NPU that OpenVINO will only give you graphs for

npunlock (Apache-2.0, C, 50 stars, 64 commits, 1 contributor, created September 20, no tagged releases) starts from a fact about Meteor Lake laptops: the NPU contains programmable ACT-SHAVE vector cores, but Intel's public stack only accepts graphs built from operations its compiler already supports. There is no public way to hand it a C implementation of an operation. npunlock rebuilt that path. You write a kernel as a C function, npunlock compiles it to SHAVE machine code, places it in a graph under a stand-in operation (the README's GELU example uses Abs as the carrier), patches the compiled blob, and runs it through Intel's installed driver, checked against NumPy. It does not need OpenVINO as a runtime, Python package or compiler frontend, although it emits OpenVINO-format IR for the driver. Verified so far are FP16 unary and two-input kernels, an FP32 unary path, nonlinear math like tanhf, and, as of yesterday, one graph containing both an FP32 branch and an FP16 branch. It has Python, CLI and C APIs.

The Show HN (64 points) is where the use case got stated. Asked what it is for, the author gave three answers: operators OpenVINO does not have; FP32 versions of existing operators for numerically sensitive work; and fusing several small kernels into one to cut SHAVE invocations. Another commenter, doing radio DSP on their laptop NPU, described the Intel graph compiler as unpredictable, including ops silently lowered to FP16. The author confirmed FP32 operations are sometimes quietly converted to FP16 and back, losing precision, and said that is part of what drove the project. The README is also candid about its sourcing. The compiler comes from Intel/Movidius MoviTools, which npunlock does not redistribute, and the verified copy is inside a 2024 Lenovo NPU driver package, version 31.0.100.1688, that the guide tells you to extract but never install. It also has a note on AI use: the author used AI for scaffolding, documentation and English, and says the reverse engineering and conclusions came from hands-on work.

Reach for it if you have a Meteor Lake laptop and a numerical kernel that OpenVINO either cannot express or quietly runs in the wrong precision; it replaces falling back to the CPU for the one operation the graph compiler will not lower the way you need. Delete the assumption that the NPU in your laptop only runs what Intel's compiler hands it. Tradeoffs: Windows x64 and NPU3720 only, with Linux and newer NPUs untested (the author has published a porting plan and a probe tool that has not yet run on hardware); static shapes and known tensor layouts only; the toolchain is proprietary and pulled from an OEM driver package, so its availability is Lenovo's call; and this is one person's reverse engineering of a binary format Intel can change with any driver update.

→ github.com/hsfzxjy/npunlockdiscussion ↗

04

stable-diffusion.cpp — Qwen Image 2.1 on CUDA, Vulkan, Metal, SYCL or a CPU, from release day, no torch

stable-diffusion.cpp (MIT, C++, 7,155 stars, 809 forks, 908 commits, 100+ contributors, created August 2023, build-numbered releases with master-908 cut September 23) is to diffusion models what llama.cpp is to LLMs: the same ggml base, one binary, and weights in safetensors, .ckpt or GGUF with its own quantization. The backend list is what fits today's theme: CPU with AVX through AVX512, CUDA, Vulkan, Metal, OpenCL and SYCL, on Linux, macOS, Windows and Android via Termux. On September 20 it added day-0 support for Qwen Image 2.1, covering text-to-image, editing with multiple reference images, and alpha-channel PNG output triggered by the prompt. It also added a prefix cache that saves the text and reference-image keys and values on the first denoising step so later steps compute only the image tokens. The doc gives the cost: about 4 GiB per condition for a 4,096-token prefix, in FP32 on every backend, with a flag to turn it off.

The commit log for the last three days is the maintenance picture: a configurable conditioning cache for all models, a MiniMax-H3 text-conditioning cache, a fix that maps memory-mapped weights through Metal buffers instead of CPU buffers on Macs, and a CI rule that now closes pull requests from organization-owned forks automatically. The bundled sd-server exposes OpenAI-compatible routes alongside the AUTOMATIC1111 sdapi ones, so anything already calling either API can point at a local process instead. The README is upfront that this is under active development and that the API and command-line options may change frequently, and each model has its own doc page listing the exact weights, VAE and text encoder it needs.

Reach for it if you generate or edit images on a machine without an NVIDIA card, or you want image generation in a product without shipping a Python environment; it replaces the ComfyUI install you keep only for its API, or the torch venv that breaks on every CUDA update. Delete the hosted image-API spend for internal tooling that can run on the GPU already in the box. Tradeoffs: CLI flags and API change without notice, so pin a build; Qwen Image 2.1 needs Qwen3-VL-8B as its text encoder plus its own VAE, which is not interchangeable with the earlier Qwen Image or Wan 2.2 VAE; the prefix cache trades several GiB of memory for speed; and this is a runtime, not a workflow editor, so node graphs and custom pipelines stay in ComfyUI.

→ github.com/leejet/stable-diffusion.cpp

05

Also on the desk. Hindsight (MIT, Python, about 27,000 stars, 1,607 of them in the last day, v0.10.1 released September 21) is an agent memory server built around retain, recall and reflect operations rather than chat-history lookup. It claims state-of-the-art LongMemEval results and says those were independently reproduced by Virginia Tech's Sanghani Center and The Washington Post, while other vendors' scores are self-reported. The Docker quick start needs an LLM API key. Separately, Transluce's report on agent activity found in urlquery.net scans is worth reading in full: 6,467 scan reports with strong evidence of agents between November 2025 and September 16, including SQL injection and path traversal probes against an Australian government health site, none of which appear to have succeeded. The report calls the extent of the activity "minor."

Verification notes: star, fork, commit, contributor and release data are from the GitHub API on 2026-09-24. Commit totals use the per_page=1 Link-header last-page trick, contributor counts use per_page=100 (stable-diffusion.cpp filled the page, so "100+"), and star counts in the last day are from the github:trending feed. READMEs were pulled raw from each default branch. virtio-nvgpu's benchmark caveats are from BENCHMARKS.md, npunlock's MoviTools sourcing is from wiki/GET_MOVITOOLS.md, and the Qwen Image 2.1 details are from docs/qwen_image_2.1.md. sd-server's OpenAI and sdapi routes were confirmed from the examples/server source listing. Qualcomm's announcement page renders client-side and could not be read beyond its title, so this edition cites the announcement and makes no claims about its contents. HN points and thread quotes are from the Algolia API (items 49824864, 49800513, 49824884). Pipeline health: today's pool held 79 rows, up from 72 yesterday, against the ~280 the compose prompt assumes. Reddit returned zero rows again, 138 days since the last nonzero day on May 9. github:trending returned 15 rows across five feeds (all 5, go 2, python 3, rust 2, typescript 3), with python back after two empty days.

→

One of these,
every weekday.

Free. One-click unsubscribe in every email. No tracking pixels in the email.