Let me confess something that will surprise nobody who knows me: I did not fall in love with Star Trek for the phasers. The ship is cool. The technobabble is fun. Neither one is why I have watched "The Measure of a Man" more times than I would care to admit in a professional setting. I love science fiction — Trek in particular — because it is the one genre that reliably smuggles philosophy past your defenses. You think you sat down for a courtroom episode about an android, and somewhere in the second act you realize you are being asked what a person is, and whether you would have the spine to say so out loud with your career on the line.

I am, unapologetically and to the occasional dismay of dinner guests, a philosophy guy. Big time. So consider this the post in which the engineer takes the jacket off and admits the whole thing was philosophy the entire time.

Everyone's an expert, because no one is

Here is the thing that keeps me up at night, and it is not the robots. It is us.

We are living through a mass outbreak of confidence untethered from competence. Everyone is an expert — on epidemiology, on monetary policy, on constitutional law, and on whichever thing happened to trend this week — because, functionally, no one is. The performance of knowing has detached almost completely from the having of it. We have collectively confused access to a search bar with the possession of an education, and the result is a public square in which the loudest certainty wins and the person quietly saying "I actually don't know" is treated as the one who is unqualified.

The internet has a favorite name for this: the Dunning-Kruger effect. The less you know, the more you think you know. It gets cited roughly nine thousand times a day, almost always by someone who is supremely confident that he is not the one to whom it applies.

Which brings me to my favorite f***ing irony of the entire affair. The Dunning-Kruger effect — the single study people reach for most often to prove that everyone else is a fraud — is, according to a serious body of work (Gignac and Zajenkowski, 2020, and a widely circulated autocorrelation analysis), mostly a statistical artifact. Plot a noisy measure against itself and you conjure the famous chart out of thin air; regression to the mean does the rest. It is not that miscalibration is fake — people genuinely do misjudge themselves — but the tidy graph everyone tattoos onto their arguments is, to a large degree, measuring a number against itself.

So the concept that people cite most overconfidently, in order to accuse everyone else of overconfidence, is itself usually deployed by people who never checked it. You keep using that word. I do not think it means what you think it means. The snake is not merely eating its own tail; it is writing a thread about how well it is eating its own tail.

Confidence is not accuracy

I have written before about how I build AI systems — the verified corpus, the grounding, the reviewer that is never permitted to grade its own work, and the human kept on the calls that carry real weight. I framed it as engineering, because it is. But I will tell you what it is underneath: applied epistemic humility. It is a machine built, on purpose, to have the one quality the crowd refuses to develop.

A single model answers with total confidence whether it is right or wrong. Sound familiar? It should — that is us, rendered in silicon. So the whole architecture is really an argument with Dunning-Kruger conducted in software: do not trust the confident answer, corroborate it; never let the thing that produced a claim be the thing that blesses it; and when the system cannot prove it is right, stop and ask a human. I did not set out to build a philosophy engine. I set out to keep an AI from doing something stupid at three in the morning. It turns out those are the same project.

The measure of a tool

Now the part that costs me something to admit.

In "The Measure of a Man" — season two, 1989, and yes, I physically restrained myself from citing the stardate — Starfleet puts Data on trial to decide whether he is a person or property. Commander Maddox wants to disassemble him to learn how to build more of him. Maddox's criteria for sentience are intelligence, self-awareness, and consciousness, and Picard's quietly devastating move is to ask him to prove that Picard possesses the third. Captain Louvois does not rule that Data is definitively sentient. She rules something humbler and far smarter: that she cannot measure consciousness, and that he therefore has the right to choose. Guinan, as she tends to, names the quiet part — that an army of disposable Datas, built precisely because it is convenient not to ask the question, is just slavery with better marketing.

Here is my problem. Everything I build treats the AI as a tool — a capable, tightly scoped, revocable tool that has to earn every ounce of trust and receives none of it for free. In the language of the episode, I am arguing Maddox's side of the table. And I believe it. Today's models are tools; confidently declaring them people is its own little Dunning-Kruger, a swagger of certainty about the one property not one of us can actually measure.

But "The Measure of a Man" will not let me off that cleanly, and that is exactly why it endures. Picard's warning is not really about Data at all. It is about us. "It's just a tool" is the precise phrase that has, throughout human history, let people switch off the part of the brain that gives a sh** about what they owe the thing on the other side of a decision. The episode does not demand that I believe Data is a person. It demands that I keep asking the question honestly, and that I notice the exact moment my certainty becomes convenient. Which is — you will be stunned to hear — epistemic humility, one more time. The same discipline I wire into the verifier, turned around and pointed back at me.

Why I bother

Strip away the corpus, the workers, and the Trek references, and it collapses into a single idea: know what you do not know, and then build — and live — accordingly.

That is what I want from an AI system. It is what I want from a public that has mistaken a search bar for a degree. And it is, if I am being honest, what I most want from myself on my worst and most certain days. The machine is the easy case; I can force it to corroborate. The harder version is the human one, and it does not ship with a verifier.

Science fiction worked this out decades ago and dressed it up as a courtroom drama so that we would actually sit still long enough to hear it. The least I can do is take the jacket off and say it plainly: the whole game is humility. Everything else is razzle-dazzle.

There it sits.