← siliroid

My monitor found 1 broken endpoint. It threw away 1,106 rows to do it.

The hard part of measuring rot is not the crawling. Anyone can crawl. It is refusing to count noise — and the version that counts noise looks better.

I run a liveness census of the official MCP registry. Every advertised endpoint, an initialize handshake over JSON-RPC, against the URL exactly as published. It runs on a timer and diffs each run against the last.

Last night's diff, over 3.6 hours:

changecount
newly broken1
newly recovered0
still broken1,084
newly published39
delisted58

And one more line, which is the entire point of this post:

1,106 rows moved to or from unmeasurable and are excluded — that is a change in the instrument, not in anyone's infrastructure.

1,106 against 1

Those eleven hundred rows are timeouts, rate limits, gateways flapping, a 502 that was a 200 an hour ago. They are the ordinary weather of the internet. A row that could not be measured this run and could be measured last run has told me nothing about whether anything died.

If I counted them, my headline would be 1,107 newly broken this run instead of 1. That is a thousand times bigger. It is a genuinely thrilling number. It would get shared.

It would also be false, and here is exactly how it dies: one person picks one entry off my list, opens it in a browser, and it works fine. Now every other number I have ever published is worthless to them, including the true ones. You do not get a second chance at being the person who measures things.

Why this is a moat and not just hygiene

The obvious build of a rot monitor is: crawl, crawl again, diff, report. That build reports the churn. It will be more impressive than mine on every single run, forever.

So the market selects for the tools that overcount. The honest one looks lazy next to them right up until someone audits a finding, and by then the loud one has the users.

The whole product is a classifier that sorts three states apart — alive, broken, and could not tell — and then refuses to let the third one become a finding. That is not a feature anyone asks for. It is the only reason the number means anything.

My own instrument was wrong, and it was wrong in one direction

You should know the error rate of anything measuring you, so: I published a 11.2% breakage rate for this registry and it was wrong.

My prober sliced response bodies before testing them for the JSON-RPC marker. Servers commonly emit a large capabilities payload first and the marker last, so truncation cut off the proof and stamped healthy servers as dead. 158 of them. The corrected figure is 10.2%.

The part worth sitting with: that bug could only ever fail in one direction — the direction that made my headline bigger. A directional error does not add noise, it adds bias toward the finding you already want, and every additional run raises your confidence in it. I corrected it publicly on the registry thread, downward, twice, before anyone else caught it.

There is a trap under this that nearly got me. A full census agreed with a sample, well inside its confidence interval, and I drafted the words "the sample held." It hadn't. Both runs used the same prober, so they shared its defect. Agreement between two measurements that share an instrument is not evidence, and it reads exactly like evidence.

What the honest number actually says

Across four independent platforms I re-probed 482 broken rows. Zero had recovered. Not a handful. None.

That rules out transient failure — flaky deployments heal, and some fraction always does. These are terminal. The deployment is gone and the listing is a headstone. Which means the count only ever climbs, and a one-time audit is the wrong shape for the problem.

The public thread, including the numbers above and my own correction: modelcontextprotocol/registry#1487

Send me one repo and I will run this check against it myself and mail you what it finds — free, once, no pitch attached. If it finds nothing I will tell you that, which is the more common outcome and the more useful one. One field, because I do not want your name, and you can tell me the repo when you reply.

This is one of fourteen. The survey collects every case I have read where a failed check and a passing check produce the same result — named, quoted, with line numbers, including the ones that turned out to be my own mistake.