Knowing When to Stop
The failure that surprises leaders most isn't a wrong answer — it's an agent that never finishes: circling, re-trying, burning time and money on a task it can't recognize as done. A stopping rule isn't a detail; it's a safety system.
Because an agent runs in a loop, it needs two things to end well: a clear definition of "done" so it knows when to stop, and a hard budget — a cap on steps or cost — so it can't run forever even if it gets confused. Without them, an agent that hits a snag can repeat the same failing action indefinitely, or wander in circles, quietly spending money the whole time. Real incidents have involved single runs consuming hundreds of dollars because nothing told them to quit.
The budget is a floor, not a fix: it caps the damage, but a healthy agent should usually finish or escalate long before hitting it. As a leader, ask two questions — "how does it know when it's done?" and "what stops it if it doesn't?" A team without good answers has built something that can run away from them.
The $300 run
Finance flags that one agent run last week cost $300 — it ran for an hour before someone killed it. Your team is embarrassed. What two safeguards were missing, and what do you require?
Two things were missing: a hard budget (a cap on steps or cost that would have stopped it automatically at, say, $5) and a real stopping rule (a clear definition of done, plus no-progress detection so it escalates instead of circling). Require both before it runs unsupervised again. The budget guarantees no single run can ever surprise finance; the stopping rule means it rarely gets near the budget because it finishes or hands off. An agent without these can always run away — the $300 was luck it wasn't $3,000.
Vectorly: the loop that wouldn't quit
Vectorly's research agent occasionally ran 40+ steps and never finished — re-running the same search, re-reading the same page, circling. A few runaway runs racked up real cost, and one ran twenty minutes before a human intervened. The cause wasn't a weak model; it was the absence of a stopping rule. The agent had no crisp definition of "done" and couldn't tell it was repeating itself.
They added a hard step-and-cost budget as a backstop, then the real fix: an explicit definition of done, results made legible so the agent could see its last action's outcome, and detection of repeated no-progress states so it changed course instead of looping. Average steps fell from 14 to 6 and the never-terminating runs vanished. Simply raising the step cap, which they'd tried first, had only made the expensive runs more expensive — the fix was teaching it to know when to stop.
Tessera caps every request at a step and cost budget and defines "done" as resolved-or-escalated. When the agent starts re-fetching the same record, a no-progress check routes it to a human instead of letting it spin. No run can surprise the finance team.
Quiz · Chapter 5
- An agent needs to end well because:
- The two safeguards are:
- A step/cost budget is:
- Raising the max-step cap on a looping agent: