The Agent Loop & Stopping

The loop is easy to write and easy to get badly wrong. Agents fail in two opposite ways: they stop too early with half an answer, or they never stop and burn through your budget. Designing when to stop is most of designing the agent.

The model signals it's finished by replying with text and no tool calls. That's the natural end, but relying on it alone is how agents run away, because a confused model will keep calling tools. A real loop has three ways to stop, in this order:

Natural stop: the model gave a final answer. Stuck detection: the loop isn't making progress, so stop early. Step limit: a hard max_steps cap that guarantees the loop ends.

A runaway, in numbers

Give an agent with no cap a task it can't complete, such as a tool that always errors, and it won't give up gracefully. It retries, rephrases, tries other tools and keeps going. In one test, an uncapped agent on an impossible task ran more than 50 steps before being stopped by hand, spending about $4 on a request that should have failed in 3 steps for a fraction of a cent. At real traffic volumes, a missing cap is both an outage and an invoice. Every production loop needs one.

Stuck detection

The cap guarantees an end, but you still pay for every step before it. It's better to notice the agent spinning and stop sooner. The simplest signal is a repeated action: the same tool with the same arguments it already tried. If it has just called get_weather("Paris") a second time, it's looping. Tell it so ("You already tried that and got X; try something different or stop"), and if that doesn't help, end the run. In practice this catches most runaway runs well before the cap.

Decide what a failed run returns

On a natural stop, return the model's answer. On a stuck or capped run, returning the model's last confused message is a poor experience. Define a proper result instead: "I couldn't finish this. Here's what I found and where I got stuck." An agent that fails honestly is far more useful than one that guesses confidently after eight wasted steps.

Starter code

# A stand-in "model" that stubbornly retries the same failing tool.
def fake_model(messages):
    # Always wants to call broken_tool with the same args => a loop.
    return {"tool":"broken_tool", "args":{"x":1}}

def broken_tool(args):
    return "ERROR: service unavailable"

def loop(max_steps=20, detect_stuck=False):
    messages, seen, steps = [], set(), 0
    for steps in range(1, max_steps+1):
        act = fake_model(messages)
        if act is None:
            return steps, "natural"
        key = (act["tool"], tuple(sorted(act["args"].items())))
        if detect_stuck and key in seen:
            # nudge once already given? here we just stop on 2nd repeat
            return steps, "stuck"
        seen.add(key)
        result = broken_tool(act["args"])
        messages.append(result)
    return steps, "capped"

# TODO: run loop() both ways and print steps + reason for each.

print(loop(max_steps=20, detect_stuck=False))   # (20, 'capped')
print(loop(max_steps=20, detect_stuck=True))    # (2,  'stuck')