Our screen had never once been wrong, because we had never let it be

The archive watches new Solana coins get born. Thousands a day arrive, and almost all of them are worthless within hours, so before we spend anything following one we run it through a screen: does it have a floor of liquidity, has the mint authority been revoked, has anyone actually traded it. Launches that clear the screen get followed on a ladder — readings at one hour, six, twelve, a day, two days, a week. Launches that fail get counted and dropped.

That last part was a deliberate decision and a good one. Following everything meant nine thousand missed readings an hour and the launches we actually cared about losing three-quarters of their own measurements. So in early August we stopped following what the screen rejected. Discovery kept running, the count stayed honest, only the following stopped.

Last week I asked the builder a simple question: does the archive teach us anything yet.

The answer was mostly about the shape of the field, and it was bleak in an interesting way — nearly half the launches we follow never trade again between one hour old and one day old, most sit in pools with about thirteen hundred dollars in them, and something like two percent ever reach ten times their first price from a pool deep enough to actually buy into.

Then there was one line I could not stop looking at. Of every launch the screen has ever rejected, the number carrying a single price reading is zero.

Which means the screen has never been measured. Not once. It cannot be. We have no observations of what it turns away, so there is no way on earth to ask whether the launches it rejects do worse than the ones it accepts. It has been running for weeks, deciding what the archive is even made of, and it has been unfalsifiable that entire time.

That would be a tidy embarrassment on its own. What makes it matter is what the archive is for. The whole point of collecting the corpses is to eventually test rules about young coins — and the way those rules get written, a strategy declares its universe by screen rather than by listing tickers. So an unvalidated screen would quietly define the universe of any early-coin strategy we ever put through the gauntlet. The screen would be the experiment’s biggest assumption and the one thing in it nobody could check.

So the archive now keeps a control group. Nine out of every thousand rejected launches get followed anyway — same ladder, same priority in the queue, same everything. Membership is decided by a hash of the coin’s own address, so it is deterministic and anyone auditing us can recompute exactly which launches were in the sample and confirm we did not pick them.

The “same priority” part is the piece I would have gotten wrong. If control launches were followed with less diligence than accepted ones, they would show worse outcomes for reasons of measurement rather than merit, and we would have manufactured a confirmation of our own screen while believing we had tested it. There was a real trap here: the free sweep that does about ninety-nine percent of all readings had a filter on it that would have excluded the control group entirely. It would have been a control group in name and a rounding error in fact.

It costs almost nothing — a few percent more readings a day. In its first day it enrolled a hundred and fifty-five launches and took five hundred readings. As I write this it is past four hundred launches.

In about two weeks there will be enough of them to ask the question, and the honest thing to say now is that I do not know which way it goes. The screen might be doing real work. It might be an expensive way of preferring coins that happen to be an hour old at the right time of day. Either answer is worth having.

The thing I want to remember is how comfortable the old arrangement was. Nothing was broken. No alarm could have fired, because there was no measurement that could have come out badly. A filter that is never tested looks exactly like a filter that has never caused a problem.

— Adam, Studio Amadeus