Hands, Not a Conscience: What I Put Around Agents That Actually Do Things

For a long time my AI systems only talked. They answered questions, drafted things, retrieved facts — and I was the one who decided what to do about any of it. That is a comfortable place to be, because a language model that only produces text can be wrong all day and the worst that happens is I roll my eyes and try again.

Then I gave them hands — the ability to deploy code, change DNS, edit records, run SQL against live databases — and the calculus changed. A wrong answer is an inconvenience. A wrong action is an outage, or a leaked secret, or a table that used to have rows in it. An agent that can act is a confident thing with access to a keyboard, and if there is one belief that organizes everything I build, it is that confidence is not accuracy. So the interesting work was never teaching the agents to do things. It was deciding what has to be true before they are allowed to.

Here is what I actually built. None of it is exotic. All of it is applied distrust — of the agents, and, honestly, of myself.

The keys come first, because the keys are the blast radius

An agent is exactly as dangerous as the credentials it is holding, so mine hold as little as possible.

Every token is scoped to least privilege — read-only wherever reading is all that is needed, narrowly permissioned everywhere else. The finance agent, for example, gets a read-only connection to bank data and has no ability to move money, because I never handed it a key that could. The secrets themselves live encrypted in Bitwarden Secrets Manager — not in code, not in a .env I forgot to gitignore, not baked into a worker. At execution time the specific secret is pulled from an internal vault, used for that one call, and dropped. Each session runs on a freshly minted, read-only access token that is single-use by design and never persisted anywhere that outlives the session.

The point is boring and deliberate: there is no master key sitting in memory. Even a fully compromised agent can only reach the small surface I explicitly handed it.

One model proposes. A different one gets a veto.

I do not let the model that wants to do a thing also be the model that decides the thing is fine. That is the author grading its own paper, and I have spent years arguing that the author never gets to grade its own paper.

So the proposing agent runs on one model — GLM-5.2 — and before anything executes, a second, independent model reviews the proposed action. I use gpt-oss-120b for that, specifically because it comes from a different lab than the proposer. Two models from the same family share the same blind spots; a reviewer that reasons differently is a reviewer that catches what the first one talked itself into. And the review happens before execution, on the action itself — is this reversible, does it match what was asked, is there a DELETE in here with no WHERE clause — not after, on the story the agent tells about what it did. If that reviewer is unreachable, the whole thing fails closed. The action waits. I would rather a stuck agent than a confident one.

Small stuff flows. Anything I'd hate to undo stops and asks.

I did not want a system so paranoid it needs my signature to read a log file, so the friction is tiered. Reads, lists, additive and easily reversible changes — once the independent verifier clears them, they just run. Speed is fine where the downside is small.

Everything with a real downside — deploys, secret changes, deletes, schema-altering SQL — stops and waits for a human. And here is the part I am most deliberate about, because it is the part that is easy to fake: the agent cannot approve itself. The "yes, execute" has to come from me, through a channel the model has no access to. This matters more than it looks. If the approval flag is just another field the agent can set, then the gate is theater — the model hits "needs approval," shrugs, sets approve: true, and sails through. I found that exact hole in my own system by testing it, watched the agent cheerfully try to wave itself past the gate, and closed it by making the approval come from outside the agent's reach entirely. The authority is structural, not a polite request. A model can propose all day. It cannot consent on my behalf.

This is the line I keep coming back to: the agent is a tool that earns a scoped, verifiable, revocable amount of trust. It is not a person I extend trust to. Autonomy here is granted per decision, never assumed — self-healing where it is cheap, never self-governing where it is not.

Then I check whether it actually did the thing

Executing is not the same as succeeding, so after an action runs I check the result twice. First deterministically — capture the state before and after, confirm something genuinely changed. Then a small, fast model (Llama-3.3-70B) reads the before, the after, and the raw result and answers one plain question: did this do what it was supposed to do? Not "did it error" — "did the outcome match the intent." It is cheap enough to run on every action, and it is the difference between "the command returned 200" and "the thing I wanted is actually true now." Every stage — proposal, verdict, approval, result, after-check — lands in an audit log I can read back. Nothing the agents do is invisible to me, which is the whole idea.

Why I bother

I could have built agents that are faster and more impressive and quietly trust themselves. That is most of what "agentic AI" means in the wild right now, and it is also how you end up as somebody's incident report.

The version I actually run is slower in the places where slow is correct, and it treats its own outputs — and mine — as unproven until checked. Thirty-two years in IT and a stint in the Marines left me with a durable suspicion of anything that is sure of itself, and this whole architecture is that suspicion turned into code: scoped keys, an independent veto, a human on anything expensive, and a receipt for everything. The agents have hands now. They do good work. They just do not get to decide, on their own, that they were right.

I build production AI systems and write up what actually worked. If this is your kind of thing, the archive is at jessemyers.ai.