Chapter 11

Advanced Patterns

Once the basics hold, a few patterns take agents further: self-review, retrieving tools from a large catalog, agents used as tools, and code execution. Each helps in a specific situation and each has a cost, so the skill is knowing when each one pays off and measuring that it did.

Case study

Sentinel adds a second look

Sentinel's coding agent produced changes that looked fine but shipped small errors, a wrong edge case or a missing import, about one time in four. A bigger model barely helped, because the problem was overconfidence. They added a check before the agent could declare itself done: run the tests, or have a critic pass reread the change against the requirement.

With that check-then-fix step, the agent caught and repaired many of its own mistakes, and the error rate fell from 25% to 9%. It cost extra steps, which the team measured and decided were worth it.

Self-review

Self-review, often called reflection, adds a step where the agent critiques its own draft ("Does this answer the question? Is every claim supported? What's missing?") and then revises. On tasks that are hard to get right in one pass, like complex reasoning, code or structured output, it helps measurably. On one set of coding tasks, a single review pass raised correctness from about 68% to 82%. It costs at least one extra model call on every task, including ones that were already right, and too much review can even talk a correct answer into a wrong one. Use it where you've measured a gain worth the extra step.

Retrieving tools

Chapter 2 assumed a handful of tools. Some systems have hundreds, and you can't put 200 tool definitions in every prompt: it bloats the context, slows the model and makes it choose worse. Keep the tools in a searchable store and give the model only the few relevant to each request. An agent that picks badly among 60 always-loaded tools often picks well among the 5 retrieved for the task. This is where agent work and search work meet.

Agents as tools

You can expose a whole agent as a tool for another agent. The parent calls research_agent(query) like any other tool, and the child runs its own loop with its own clean context. It's the orchestrator pattern from Chapter 6 in another form, with the same benefit (focus) and the same cost (more tokens).

Code execution

A sandboxed code interpreter can be worth more than a dozen custom tools. Instead of you predicting every operation and building a tool for it, the agent writes and runs code for whatever calculation or data work it needs. It's also a serious security boundary, since the agent is running arbitrary code, so it must be sandboxed, limited in resources and cut off from the network.

Try it · ~25 minFree · no API key

Measure what self-review costs and gains

Score a task set with and without a review pass, measuring both the gain in correctness and the extra steps. Then write a one-line rule for when to turn review on.

Starter code

# Per task: (correct_without, correct_with_reflection, base_steps)
TASKS = [(True,True,3),(False,True,3),(True,True,2),(False,True,4),
         (True,True,3),(False,False,4),(True,True,2),(True,True,3)]

def score(reflect):
    correct = sum((cw if reflect else c0) for c0,cw,_ in TASKS)/len(TASKS)
    steps   = sum(s + (1 if reflect else 0) for _,_,s in TASKS)/len(TASKS)
    return correct, steps

# TODO: print both configs; compute how many tasks reflection fixed and the
# extra steps spent across ALL tasks to fix them.

Worked solution

c0,s0 = score(False); c1,s1 = score(True)
fixed = sum((not a) and b for a,b,_ in TASKS)
print(f"no-reflect {c0:.0%} @ {s0:.1f} steps")
print(f"reflect    {c1:.0%} @ {s1:.1f} steps  (+1 step on ALL {len(TASKS)} tasks, fixed {fixed})")
# => no-reflect 62% @ 3.0 steps
#    reflect    88% @ 4.0 steps  (+1 step on ALL 8 tasks, fixed 3)

Review raises correctness from 62% to 88% and adds a step to every task: 8 extra steps to fix 3 tasks. Whether that's worth it depends on what a wrong answer costs. For high-stakes tasks it's cheap; for a busy, low-stakes endpoint, a third more steps on every request might not be worth 3 fixes. To go further, only review when a cheap signal suggests the draft is risky, such as length or low confidence, and measure again. You may keep most of the gain for a fraction of the cost.

Running case · Ridgeline

Before closing a ticket, Ridgeline's agent checks that the resolution matches what the customer asked and that every step was completed. That check catches premature "resolved" replies and raises real completion by a few points.

If you're starting out

You don't need to have built all of these to talk about them well. Interviewers want to hear that you know which problem each pattern solves and what it costs. For code execution in particular, mention sandboxing right away; it shows you think about safety.

If you run the team

Each of these patterns comes up in vendor pitches and conference talks. When your team wants to adopt one, ask what failure it fixes in your system and what it did to success and cost on the team's task set. If the agent will run code, ask how it's sandboxed and what it can reach on the network.

Back Continue