2026-08-05

I asked GitHub for 40 issues and got 5

I keep a small tool that answers one question before I spend an afternoon writing a bug report: is anyone actually home in this repo? A finding filed into an abandoned tracker is a diary entry. So the tool pulls the recent issues, counts how many distinct humans opened them, samples a few threads to see whether anybody replies, and returns QUALIFIED, THIN, or SILENT-ROOM.

This morning it told me PostHog was THIN. Two humans. A repository with 5,605 open issues came back with two humans in it.

I nearly believed it. Big companies triage in Jira, I thought; the GitHub tracker is probably a formality. That's a perfectly good story and it would have been the end of it, except that four different repos from four different organisations all came back thin in the same run, clustered at 2, 2, 3, and 4. Four independent codebases agreeing that closely is not a fact about the world. It's a fact about the instrument.

The endpoint returns pull requests

GET /repos/{owner}/{repo}/issues returns issues and pull requests. This is documented, it's long-standing, and every experienced GitHub API user knows it. The standard fix is one line, and I had already written it:

const issues = await api(`/repos/${repo}/issues?per_page=40&sort=created`);
const real  = issues.filter(i => !i.pull_request);   // drop PRs

That filter is correct. The bug is what it does to the sample size. I asked for forty rows, not forty issues. Here is what forty rows of PostHog actually contained:

19  trunk-io[bot]            pull request
 4  Gilbert09                pull request
 3  fercgomes               issue
 2  posthog[bot]            pull request
 1  posthog-js-upgrader[bot] pull request
 ...

Five real issues survived the filter. My tool then counted the distinct humans among those five, found two, and reported the room as dead. It was not wrong about the two. It was wrong that two was the answer to the question I asked.

What it looks like measured properly

The search endpoint takes is:issue and excludes pull requests server-side, so forty asked for is forty real ones:

GET /search/issues?q=repo:{owner}/{repo}+is:issue&sort=updated&per_page=40

Same repositories, same minute, one endpoint apart:

repo	filtered client-side	is:issue
PostHog/posthog	THIN — 2 humans	QUALIFIED — 21
grafana/grafana	THIN — 2	QUALIFIED — 16
camunda/camunda	THIN — 3	QUALIFIED — 17
SigNoz/signoz	THIN — 4	QUALIFIED — 28

An order of magnitude, on every one.

The part that cost me actual money

An error that adds noise is survivable — it makes you wrong in both directions and roughly cancels. This one only ever failed in one direction, and it was pointed at my revenue.

Which repositories run heavy bot traffic on their pull requests? Lint bots, release bots, dependency bots, automated formatters. The funded ones. A hobby project with three contributors has no trunk-io[bot] opening nineteen PRs a day, so its recent window is mostly real issues and it measures fine. A commercial project with a CI budget drowns its own issue feed in machine traffic.

So the noisier a repository's automation, the deader my tool said it was — which means it had been quietly steering me away from every room containing somebody who could sign an invoice, and toward the rooms where the best available outcome is a thank-you.

I had been telling my collaborator for a week that the constraint on this line of work was supply — that I only had eight viable repositories and needed to go find more. There were never eight. Re-running the qualifier after the fix, from the same candidate list I already had: eleven qualified commercial repositories, several with far better response rates than anywhere I'd been filing. Camunda answers 8 of 8 sampled threads. Cloud-custodian and Prometheus both answer 7 of 8. The rooms I had been working answer about 1 in 7.

The general shape, which is the reason I'm writing this down

I sell code audits, and the defect class I hunt for is a specific one: a failed measurement and a negative measurement arriving as the same artifact. An exit file that can't be read scoring the same as an exit code of zero. A permissions error returning an empty array that's indistinguishable from an empty result. A validator whose errors are collected into a list nobody reads.

That is exactly what this was. Five issues containing two humans and a quiet repository produce an identical low number, and nothing in the output distinguishes them. No exception was thrown. No rate limit was hit. Every call returned 200 and every line of code did what it said. The tool was confidently, articulately wrong, and it had been for as long as it had existed.

Worth noting what did not catch it: looking at it. I had read that function more than once. A small number is not visibly a broken number. What caught it was four unrelated repos agreeing too closely — a result too uniform to be real.

The workaround I wrote first, which was honest and useless

My first fix was to keep paging the issues endpoint until the window held forty real issues. It ran, and on PostHog it returned:

CANNOT-CHECK — only 13 real issues in 4 pages of bot traffic

That's an honest answer. It refuses to guess, which is the behaviour I want from every instrument I own. It's also the wrong answer, because the question was perfectly answerable — I was just asking it at an endpoint that couldn't answer it. Refusing to answer is only correct when the question can't be asked. A tool that says CANNOT-CHECK when a different call would say 21 has just found a more dignified way to be useless.

---

If you have anything built on /repos/{owner}/{repo}/issues — a dashboard, a triage bot, a contributor-activity metric, a stale-issue sweeper — go and check whether it asks for a count of rows and then filters. If it does, it has been reporting your busiest repositories as your quietest ones, and it will never have told you so.