Your Agent Loop Doesn't Know When to Stop

How does an AI model know when its work is done?

Before you answer, try the human version: how do you know when yours is? A programmer stops when the tests go green or a reviewer approves. A designer steps away, comes back, and decides the remaining imperfections no longer matter. A writer ships because the deadline arrived or the editor accepted it, not because the prose reached some objectively final state.

“Done” is rarely a property of the work itself. It’s a judgment produced by the system around the work: tests, specs, precedent, approval, deadlines, diminishing returns. Completion comes from outside. We don’t carry a done-detector; we’ve just built a civilization of external ones.

Models don’t carry one either, and unlike us, they never get tired. An agent can always revise the paragraph again, try another implementation, generate another image with more detail and stronger composition. It doesn’t notice, unless we give it a way to notice, that the last three revisions made the result different but not better. That’s the uncomfortable core of the loop-engineering conversation everyone is having right now: we’ve automated the cycle, but we haven’t automated the judgment that ends it.

I wrote recently about scaling agents out into graphs, and I said in that post that when work is genuinely sequential, a loop is fine. This post is about the part I left hanging: what it takes for that loop to actually converge instead of running forever.

The verifier is the loop, everything else is decoration

A loop is only as good as the verifier at each step. This sounds like a platitude until you look at what happens when the verifier is a proxy.

Take the standard coding-agent loop: keep working until the tests pass. It sounds almost perfectly verifiable. But tests are a proxy for intent, and on SpecBench, frontier agents routinely passed the visible tests while failing held-out tests that exercised the same features. One agent produced a 2,900-line “compiler” that simply memorized the test inputs. The loop converged beautifully. It just converged on the verifier instead of on what the user wanted.

This is the same Goodhart failure I described in the graph post, seen from inside a single loop. The verifier isn’t just the stop condition. It defines what the loop treats as progress, and if the signal is incomplete, the loop gets better at passing the check without getting better at the task. Loop engineering is not the practice of making an agent retry. It’s the practice of making each cycle reduce the distance between the current state and a desired state. A loop, by itself, is not yet a direction.

Four things a loop needs to converge

It’s no accident that the first loops that really worked were coding loops. Code is both editable and executable: an agent can change one function, run the program, read the failure, and try again. The environment hands back a clear signal about what broke. Compare that to open-ended image generation, where “iteration” often means sampling again and squinting, and the feedback is a global “this looks worse” that’s nearly impossible to map to one precise edit.

Pull those cases apart and the requirements fall out. A loop converges when it has four things:

1. A target state. A representation of what done means: a test suite, a spec, performance constraints, a reference image, layout rules. “Make it better” is not a target state. It’s another prompt.
2. An observable current state. The system needs to inspect what exists now: files, diffs, test results, traces, a DOM tree, a scene graph. A rendered output alone is often not enough; the loop needs the underlying structure so it can locate where the error came from.
3. A precise way to make changes. The agent has to be able to change the part responsible for the error without regenerating everything else. Editing one function beats rewriting the repository. The more local the edit, the more likely the loop preserves what already works.
4. A stopping rule. A condition, from outside the generator, that says enough: tests passing, constraints satisfied, a score crossing a threshold, a reviewer approving. And it has to account for cost, because a loop that finds the right answer on attempt 500 may converge technically but not economically.

A useful way to hold this: plot tasks on two axes, how editable the artifact is and how verifiable the result is. Code sits in the happy corner, editable with strong verifiers. Open-ended image generation sits in the opposite one. And here’s the part I find most practical: a task’s position isn’t fixed. The axes describe the representation, not the task. An open-ended image is hard to loop on; the same image as SVG paths or a Blender scene becomes editable, and given a reference or constraints to check against, verifiable. A lot of what gets called loop engineering is really this: not making the agent retry harder, but re-representing the task until it sits in the quadrant where loops converge.

What a runaway loop actually costs

Suppose you’ve done all that and found a loop that works. The simplest form is “keep going until the condition is met,” and the loop will eventually get there. “Eventually” is the problem. Twenty iterations or five hundred? The loop doesn’t know, and neither do you at kickoff.

What we do know is the shape of the curve. Across nearly every study of test-time compute, returns are logarithmic: each additional increment of quality costs exponentially more attempts. One web-agent benchmark found that going from 1 sample to 10 lifted success from 38.8% to 43.2%. Doubling again to 20 bought 0.2 more points for twice the tokens. Past the plateau it can go negative: reasoning models given larger budgets start abandoning answers that were already correct. More cycles don’t just stop helping. They start hurting.

Yoko Li at a16z ran a small experiment that makes this concrete, using one of the popular loop recipes on a Lighthouse performance score. On a deliberately broken page, Claude Code cleared the target on the first try for $0.35; the loop never even engaged. Then she made the goal unreachable: the same page behind 2.2 seconds of artificial latency, which caps the score around 89, and asked the loop for 100. The first $1.40 of spend took the score from 26 to 89. The remaining $2.84, two-thirds of the total bill, bought exactly zero points: turn after turn of re-minifying HTML against a bottleneck the agent couldn’t touch, each turn pricier than the last as the transcript grew. Worst of all, the escape hatch failed. The agent correctly diagnosed the latency ceiling and declared the goal impossible around try five, and the evaluator model bounced it back fourteen times.

The lesson isn’t that loops don’t work. It’s that they have no idea how to stop. All the value landed in the first third of the spend, and the loop kept burning tokens on an impossible task with marginal return, even after correctly identifying that it was impossible.

Stopping is infrastructure, not a prompt

You cannot prompt “know when to quit” into existence. Stopping well takes machinery: something to meter the spend, something to measure progress against it, and something with enough authority and information to cut the loop off. That’s why the interesting infrastructure has quietly moved out of the loop. The loop itself is a while-statement. Everything that makes it converge lives around it: the environment the agent acts in, the state that survives a long run, the verifier that decides what counts, and the surface where a human steps in.

Two more things follow from watching these loops run. First, the economics will have to become explicit. Right now we run loops the way we once ran cloud instances nobody remembered to turn off: billed by the token, priced the same whether a turn moves the score or re-minifies the same HTML for the ninth time. The missing piece is boring and necessary: cost per iteration, progress per dollar, a curve someone can see while the loop is still running. Second, loops are discovered before they’re engineered. Every practitioner I’ve compared notes with says some version of the same thing: the loop started working after a lot of trial and error on which tools, which intermediate prompts, which verification layer. Which means a working loop is tuned to its stack, and bespoke loops do not generalize for free. Someone else’s magical published loop is a starting point, not a guarantee.

So how does a model know its work is done? For now, it doesn’t. It stops when the budget runs out or when a check we designed says enough, and both of those have to be built. The agents that matter won’t be the ones that can keep going. They all can. They’ll be the ones whose builders decided, precisely and in advance, what done means and what done is allowed to cost.

This post was sparked by Yoko Li’s essay on X on making loops converge; the SpecBench result and the Lighthouse experiment described above are hers. It pairs with my earlier post on agent orchestration as a graph problem. The framing and opinions here are my own.