The instrument is the work: building got cheap, saying no didn't

In June I got retro mode running on compass, a coach for smart glasses I’m building on a laptop before the glasses exist. Press a key, the last five minutes of the mic buffer go through a local Whisper model, the transcript goes to the coach, a directive lands on a window standing in for the display. It worked the first time.

The speech model was base.en, picked because it was fast. I said “facade panels” at my desk. The transcript said “sign tables.”

I caught it because I was listening. I swapped to small.en, said it again, and the transcript was right. The fix was right. The check was one person, one desk, one session, and it does not run again. The next model change gets no verdict at all.

That gap is the subject of this post.

The claim

Building is cheap now. A model will write the pipeline, the driver, the harness. It cannot tell you the pipeline is wrong, because the thing it built and the thing that would check it come from the same place. A check from the author is an impression. A check from somewhere else is an instrument.

So the work moved. It used to be mostly building with a check at the end. Now the building is a sliver and the check is the job. What I observe, in myself first, is that the old proportions survive the new tools: build fast, look at the result, ship on “looks right.” The times I’ve gotten better with these tools are the times I moved the effort to the instrument.

An instrument has the same shape in every domain I’ve worked in. Two inputs: the candidate, and a reference the candidate’s author did not produce. A comparator. A residual, which is a number and not a feeling. A threshold, set before the run. A verdict that can be no.

The reference is the part people skip, because it costs something. For geometry it’s synthetic ground truth. For a classifier it’s held-out data. For an interface it’s a person doing the task while you watch. For a material it’s a coupon on the bench. For a speech model it’s a recorded set of the phrases it has to hear, scored by word error rate. That last one is the instrument I did not build in June. It’s on the compass page now, marked not built, which is the honest state.

Five paths, one instrument

Cheap building’s real upside isn’t speed, it’s breadth. When a candidate costs minutes, you try five. The discipline is what you do with five.

Not: look at all five and pick the one that looks best. Five impressions are still impressions; that’s the same skip, done five times.

Instead: a cheap filter first, seconds per candidate. Does it run, does it pass the smoke case. Most die there. Then the real instrument on the one or two left. The filter says no to the obvious; the instrument says no to the plausible. Spend accordingly.

What stays sharp is judgment about what would prove a thing wrong. That’s the skill that goes soft when every check is by eye.

Three honest sentences

When a model wrote some of the code, I use one of three sentences, each a position relative to the instrument.

I wrote this. I understand it and it’s instrumented.

AI-assisted; I chose the method and validated. I picked the approach, the model filled the implementation, the instrument passed.

AI-assisted; I’d want to study it more. The instrument passed, or there isn’t one yet, and my understanding is pending. Not a confession. A reading that tells the next person where to look.

The third sentence is the one that keeps you from getting dumber. A named gap gets an instrument eventually. An unnamed one doesn’t.

Where this gets tested

This post went up before that phrase set exists, on purpose. It’s a position, and the compass page is where it gets checked. That page keeps a list of instruments, each with what it can say no to; the first entry marked not built is the phrase set from June. That’s the whole practice: every “looks right” becomes a line on that list, and every line becomes a thing that can say no.