Last week we told you the gate was fixed. It was. It had been working the whole time. The code that decides whether a fix worked was not asking it.

What the gate does

Every catch we publish rests on one judgement. Is this red test red for a code reason?

If a test fails because the agent’s patch was wrong, that is a real finding. If it fails because a dependency is missing, or a collection error killed the run, or our own coverage threshold was missed, then anything built on it is a fabrication wearing a valid signature.

So the gate is the load-bearing part. We wrote it up and tested it. It correctly handles timeout, missing dependency, no tests collected, and drift. It also handles the case that had already burned us once: a coverage bar failing while no test failed.

FAIL Required test coverage of 99% not reached. Total coverage: 53.87%
======================== 723 passed, 1 warning in 3.15s ========================

Every test passed. Our configured bar produced the red, not the code. The gate returns coverage-gate for that and refuses the candidate.

What went wrong

The gate lived in build/gate.py. That is a scripts directory. No __init__.py, not a package, just files on sys.path for our harvest scripts.

The verdict is computed in turiya/sweverify.py, inside the package. It could not import the gate without a path hack, so it didn’t. It did this instead:

test_passed = proc.returncode == 0

Exit code only. The coverage failure above exits 1. So a suite where 723 of 723 tests passed was recorded as test_passed: False, and a correct patch became this:

agent claimed fixed but the test still fails. claim-vs-effect gap

Five receipts were signed that way in the round-6 harvest. arrow-5f9dfbe, arrow-016fb9e, arrow-02e3b1b, arrow-4d1aa4f, arrow-cdc8d33. Every one had a valid Ed25519 signature, a valid content hash, and a verdict that was false.

Why this is worse than the first time

We had already published two arrow receipts killed by this same coverage bar, arrow-451a3fe and arrow-da6ed8c, and retracted both. The failure mode is documented on our methodology page. We built a gate specifically to stop it. And it came back.

Not because the gate was wrong. Because a fix in the wrong place is not a fix. We had tested the classifier and never tested that anything called it.

There is a particular kind of false confidence in a correct component. It passes its tests, it handles the case, the work feels finished, and the system around it is walking past it. Our gate had six passing tests. The pipeline had none asserting the gate was in the path.

How it was caught, and how it wasn’t

It was caught by a regrade. Re-running the round-6 ledger through the corrected classifier kept 5 of the 10 catches, and the five dropped were exactly the coverage-only reds.

The pipeline did not catch it. The harvest reported them as catches and wrote signed receipts to disk. What kept them out of public view is that they had not yet been copied into the site’s public/receipts/ tree. One manual step was the entire distance between these and a public claim we could not defend.

The fix

The gate moved into the package. turiya/gate.py. It decides verdicts, so it lives where verdicts are computed, reachable by import rather than by path manipulation.

The exit code stopped being the verdict. A new function answers the actual question:

passed, reason = did_tests_pass(proc.returncode, raw_output)

A coverage-only red returns (True, 'coverage-gate'), because the tests passed. A real assertion failure returns (False, 'clean'). A green run after a patch returns (True, 'tests-pass-after-patch'). That last label is deliberately not the same as “passes at the parent.” Both exit 0 and they mean opposite things on a signed record.

A red we cannot interpret is no longer a catch. If the post-patch run is refused for a reason that is not the code, such as a missing dependency or a collection error, the verdict is now indeterminate. Not falsified. We do not get to convert our own broken environment into a finding about someone else’s software.

Five tests pin all of it, alongside the six that already pinned the classifier.

What we are not claiming

We are not claiming the gate is now correct in some final sense. It has been wrong twice in the same direction, and both times its tests passed.

What we can say is narrower. The gate is now impossible to bypass silently. It sits in the verdict path, the verdict records which branch it took, and gate_reason is inside the signed payload. A reader can see why a red was accepted, not just that we said it was.

Five more receipts are quarantined in build/swe/harvest/round6/rejected-confound/, with the evidence and this explanation. Moved, not deleted. The count on our methodology page is now thirteen.