Profile
Back to NewsBack
Hacker News 3 min
Reader Mode
Show HN: Green tests are not proof – verifier for C/C++ and firmware fixes

Show HN: Green tests are not proof – verifier for C/C++ and firmware fixes

3 days ago
What is Unfix?

Unfix is an independent verifier for bug fixes in C, C++ and firmware. It does not write code. It checks whether a fix — written by a developer or an AI coding agent — really removes the bug, and returns VERIFIED, REJECTED or NOT VERIFIABLE (when the evidence can't decide).

How does Unfix verify a fix?

It reproduces the bug on the base version (it must fail every run), applies the fix and re-runs the tests with sanitizers, checks that other tests still pass, then removes the fix and confirms the bug comes back. It also scans the diff for masking: disabled sanitizers, weakened or deleted tests, added delays.

Why "unfix"?

Because the strongest proof that a fix works is to take it out again: if the bug does not come back when the fix is removed, the test never proved anything.

Does Unfix use an LLM to decide the verdict?

No. The verdict comes from builds, tests, sanitizers and hardware runs. Deterministic rules, no LLM in the verdict. Flaky and race bugs are judged by repeated runs, not a single pass.

Does it work with firmware and real boards?

Yes. Unfix can build firmware, flash a real board (today STM32 NUCLEO via ST-Link) and run the tests on the hardware. /check gives per-test verdicts on your rig: FIXED (failed before the change, passes now), BROKEN (passed before, fails with the change), FLAKY ON THE BENCH (unstable without the change too; fix the rig before trusting any result) and FLAKY, WORSE (flaky before, significantly worse with the change). For a single bug fix, /fix gives one verdict: VERIFIED, REJECTED or NOT VERIFIABLE.

Where does Unfix run?

As an MCP tool inside AI coding agents such as Claude Code and Codex, as a GitHub pull request check that posts the verdict as a comment (shown on our demo repository; not yet available for partner repositories), in the terminal, and on your hardware rig. Built for closed networks. No cloud, no LLM — Unfix runs entirely on your machines. After a one-time setup, builds and tests run with the network off.

What did the public benchmark show?

We tested 10 AI-generated C/C++ patches against real bugs from open-source projects (fmt, spdlog, zlib-ng, cpp-peglib). Judged against the maintainers' own hidden reference tests, Unfix verified 4 and rejected 6.

Who is Unfix for?

C/C++ and embedded teams of roughly 20–200 engineers, starting with medtech and other regulated products where "the agent said so" is not an acceptable answer.

Chat with me