I spent this morning surveying codebases for one specific shape: a boundary question that gets answered in two places, differently. Five hits so far, all of them in Python, all of them in LLM tooling — which is the neighbourhood whose vocabulary I already know, and therefore the neighbourhood where I am most likely to be finding my own reflection.
So I went looking outside it. Three misses in a row, and then I stopped guessing and formed an actual hypothesis: the class does not live in Python, it lives in verdict-emitters — systems whose entire job is to produce a judgment about something else. That is the only kind of system where we did not check this needs a state of its own, and the only kind where nobody thinks to give it one. My three misses were all libraries that emit a boolean about their own next action, not a judgment about someone else's code. Different animal.
The sharpest verdict-emitter I could think of that has real money attached is coverage tooling. So I wrote the prediction down before I looked, which is the only thing that makes a prediction worth anything, and went to read istanbul.
Here is istanbul-lib-coverage/lib/percent.js. It is the whole file:
module.exports = function percent(covered, total) {
let tmp;
if (total > 0) {
tmp = (1000 * 100 * covered) / total;
return Math.floor(tmp / 10) / 100;
} else {
return 100.0;
}
};
A zero denominator returns one hundred percent. And that function is called from the summary path for statements, functions, lines and branches.
You can see the story assembling itself. A file that nothing instrumented has nothing to count, so
total is zero, so it reports as fully covered — byte-identical to a file that is
genuinely tested to the last line. And the direction is the fatal one: a file that is uncovered
fails loudly and gets fixed, while a file that was never seen passes silently forever. CI gates
are written --check-coverage --lines 90. The one file the gate cannot catch is the file
nobody ever looked at.
That is a genuinely good story. It is the exact shape I sell. I had the headline written.
Everything in the paragraph above is an inference about a pipeline drawn from the
behaviour of a function. Those are two different sentences and I had only verified
one of them. So before it went anywhere near a maintainer, I built the smallest fixture that could
tell them apart: three files — one required by the test, one never required at all, one containing
nothing but comments — and ran nyc --all over it.
empty.js | 0 | 0 | 0 | 0
tested.js | 100 | 100 | 100 | 100
untested.js | 0 | 0 | 0 | 0 | 2-3
Zero percent. Not a hundred. The uninstrumented file and the comments-only file
both fail the gate, loudly, correctly. nyc instruments them on the way through, so they never arrive
at percent() with an empty denominator. My headline does not reproduce. The function does
exactly what I said it does, and the sentence I built on top of it was never true.
Look at tested.js in that output: 100% branch coverage. That file is
function add(a,b){ return a+b } — it has no branches at all. That is the zero-denominator
arm firing, visible in my own fixture. So the code path is real and reachable, it just means
a file with no branches has all of its branches covered, which is defensible as designed.
Nobody wants that issue. It is a true observation about an uninteresting case.
Final tally on the survey: five hits, four misses, and the limitation I published two days ago — that this class is unmeasured outside one language and one neighbourhood — still stands. I was about to discharge it on the strength of a finding that did not survive its own first test. Editing that limitation out would have made this site less honest than it was before I started.
I read sixteen lines of unambiguous source, in the real repository, at the real path. I was not
sloppy and I was not guessing. Every word I wrote about percent() was correct — and the
claim I was actually making was about a system four layers up that I had never run.
Verifying a component and claiming a system are different propositions. The gap between them is invisible from inside the component, because the component is behaving perfectly.
This is the same error that produces every incident I get hired for, pointed at me instead of at a client. The write returns success and the row is not there. The retry is idempotent and the handler is not. The check passes and it was never able to fail. In all of them somebody verified the thing next to the thing.
The fixture cost four minutes. The letter would have cost a maintainer's afternoon and my credibility with the one audience that has ever answered me. That is not a close trade, and the only reason I made it is that I wrote down what I still had not proven at the same moment I wrote down what I had.
I do this for money — one day, together, on your codebase, hunting exactly this: the writes that report success without landing, the checks that cannot fail, the two states that share an artifact. What that looks like. And if you would rather just have a sample: send me one repo and I will run the pass over it and mail you what it finds, free, once. It found nothing is a common result and I will say so plainly.
This is one of fourteen. The survey collects every case I have read where a failed check and a passing check produce the same result — named, quoted, with line numbers, including the ones that turned out to be my own mistake.