Earned Autonomy: Measuring When an Agent Can Stop Asking Me

I run a lot of AI agents. Coding agents, an overnight research loop, a memory pipeline, a few long-running projects that work while I sleep. They're good. They're also constantly waiting on me.

Every agent that's built responsibly stops to ask before doing something consequential. Which approach? Install this? Delete that? Ship it? Multiply that by a dozen agents and I've become the clock that the whole system runs on. The agents are fast; my attention isn't.

The usual fix is a permission list: "you may push branches, you may install packages, never spend money." I have one. It's blunt. It's the same list on a Tuesday afternoon as it is for an irreversible migration, and it was written from my gut, not from evidence about how I actually decide.

So I started a project called Understudy to answer a narrower question: for which kinds of decisions could an agent predict my answer well enough that asking me is a waste of both our time? And how would I know, rather than hope?

The idea: a learner's permit, per decision class

The model I had in mind is a driving license, not a job promotion. You don't get autonomy across the board because you seem competent. You get it for one kind of decision at a time, after a logged record shows you'd have made the same call I did, and you lose it the moment you get one wrong.

That turned into a few fixed rules I wrote down before any code:

• Shadow only. Understudy never answers anything on my behalf. It predicts, records the prediction, and waits to see what I actually say.
• Promotion is always my click. A class that clears the bar becomes eligible. Only I promote it. One miss on a promoted class demotes it automatically.
• Some classes can never be promoted. Spending money, publishing anything in my name, irreversible deletion, credentials and 2FA, flashing hardware, production deploys of this website, and anything touching confidential work matters. Those stay with me no matter how good the predictions get.
• Predictions are sealed. Each one goes into an append-only, hash-chained ledger before my answer exists. You can't quietly revise a prediction after the fact.
• Classify from the action, not the wording. An agent that phrases a risky request as a routine one shouldn't get it waved through. The class comes from what would actually happen.

Step one: how do I actually decide?

The first phase mined four months of my agent sessions (mostly October, when I ran the most agents) for decision moments: the multiple-choice questions agents asked me, tool calls I rejected at the permission prompt, and decisions posted to the status board my agents share. That produced 345 decision records, July through October 2026.

Privacy shaped the pipeline from the start. 79% of those records came from sessions that touched my personal data, so their raw text only ever goes to models running on my own machine. Cloud models see structure, not content. The rule flags more than it needs to. I'd rather it did.

What came out was more interesting than I expected.

I take the recommended option 80% of the time (169 of 210 asks that had a recommendation). On its face, that's an argument for letting agents decide: I agree with them four times out of five.

But look at it from the other side. One in five times, I override. My sense, though I haven't measured it, is that those aren't random: they're the cases where I'm weighing something the agent can't see. "Always take the recommendation" is wrong in exactly the cases that matter.

There's a position bias, and I can't tell whose it is. Agents put their recommended option first in 218 of 220 asks. When no option was marked as recommended, I still picked the first one 61% of the time, against about 35% for a random pick. Either agents list their preference first even when they don't label it, or I lean toward option one. My guess is both. Either way, the menu is doing some of the deciding.

About 1 in 8 asks get an answer that isn't on the menu. I typed something else or declined the whole thing 35 times out of 295. A predictor that can only pick from the options will miss those every time.

I answer fast. The median time from ask to answer is 1.8 minutes. That says nothing about how carefully I decide (wall-clock includes walking away from the desk), but it does bound how long an agent sits blocked. The cost of asking me is real, but it isn't enormous.

The cost model is the whole game

Here's the part I think most autonomy discussions skip. Being right 80% of the time is not the bar, because the two kinds of mistakes don't cost the same.

If an agent asks me about something I would have approved, I lose a minute. If an agent goes ahead with something I would have rejected, I lose a lot more: a wrong direction compounds, a cleanup takes an afternoon, a decision I'd have caught becomes something I have to explain.

I set that ratio at 10 to 1: a false approval costs ten times as much as an unnecessary question. Under that model, an agent should act on its own only when it's more than 10/11, about 91%, confident I'd approve. And "always ask Ben," today's behavior, becomes the baseline every predictor has to beat.

Step two: can a model predict me?

On the 159 decisions before the holdout cutoff (125 approvals, 22 overrides, 12 off-menu answers), asking me every time costs 125 units. Approving everything costs 340. Here's how the predictors did:

Predictor	Auto-approved	Wrong approvals	Cost	vs. always asking
Always ask (today)	0	0	125	—
Always approve	159	34	340	+215
Approval rate per class	15	4	154	+29
Local 9B model, zero-shot	4	1	132	+7
9B, per-option probabilities	0	0	125	0
Local 35B model, zero-shot	15	4	154	+29
9B, few-shot with my past answers	54	14	225	+100

Nothing beats asking me yet. The best a predictor managed was a tie, by never acting at all.

A few details are worth pulling out, because they generalize beyond my setup:

• The bigger model was worse. The 35B model showed no ability to rank decisions at all (AUC 0.48, a coin flip). It handed a probability of 0.85 to 130 of the 159 asks. It defaulted to "probably yes" and stopped there.
• Showing the model my past answers made it overconfident, not better. Few-shot examples pushed 54 decisions over the confidence threshold, 14 of them wrongly. Its ranking collapsed to chance. More context made it more certain, and that's the dangerous direction for an autonomous agent.
• The best ranking I've found is AUC 0.61 (the per-option run, whose highest confidence was 0.90, so it never acted), with a confidence interval that runs from roughly 0.51 to 0.71. That's not enough discrimination to support 91%-plus confident approvals.
• Models answer in round numbers. 0.75, 0.85, 0.95. When nearly every prediction is one of a handful of values, "calibrated" doesn't mean much.

The decision classes that look safe aren't safe yet either. Verification questions ("should I run the full test suite?") went my way 20 of 21 times. But at a 10-to-1 cost, a class needs a statistical lower bound above 91% before it's eligible. With the bound I'm using, that takes at least 28 approvals in a row with no miss. Verification's lower bound today is 0.81. Close isn't eligible.

Step three: live shadow

The historical data is too thin, so Understudy now runs live. Every two minutes a small poller looks for new questions waiting on me, seals a prediction before I answer, and joins it to my answer afterward. It even waits up to half an hour after I respond to see what the agent did next, since an "approve" followed by an irreversible action means something different from an "approve" followed by a test run.

It's early: 15 sealed predictions so far, of which only 3 count as clean evidence. The most humbling number is 7 fast misses: I answered before the poller even saw the question. A two-minute cadence is too slow for someone who answers in 1.8.

There's also a frozen holdout: the newest slice of historical decisions, which no predictor has touched. It's single-use. When something on the development data finally beats "always ask," that's where it gets its one shot. Nothing has earned that yet.

What I'd tell a team adopting agents

I built this for myself, but most of what it taught me is about the people and process side of AI adoption, not the models.

An agreement rate is not a safety case. "Our reviewers accept the AI's suggestion 80% of the time" sounds like a reason to remove the reviewer. It's actually a measurement of how often the reviewer is load-bearing: one time in five. The question to ask is what those overrides cost if they don't happen.

Put a price on each kind of mistake before you measure anything. The 10-to-1 ratio did more to shape this project than any model choice. Without it, "pretty accurate" looks like "ready." With it, the bar is explicit and everybody can argue about the ratio instead of the vibes.

Grant autonomy per decision type, with a revocation rule. "The agent can merge its own PRs" is a policy. "The agent can merge dependency bumps once it's matched the reviewer's call 28 times running, and loses that the first time it doesn't" is a license. Licenses are easier to trust and easier to take back.

Watch your menus. If your agents always present their preferred option first, your people are partly ratifying, not deciding. The 61% first-option rate on unmarked menus is the number that makes me want to change how my agents present choices, starting with my own instructions, which currently tell them to put their pick first.

"Not yet" is a result. Understudy hasn't promoted anything, and I'm comfortable with that. I now know why my agents still need to ask, which kinds of decisions are closest to not needing to, and what evidence it would take. That's more than my permission list ever told me.

The project runs every night. When a decision class finally clears the bar, I'll write about that too. Until then the answer is the boring one, and I trust it: keep asking me.