Human-in-the-Loop & Permissions

Autonomy is a dial, not a switch. The engineering question is never "should the agent be autonomous?" but "which actions can it take alone, and which require a human's yes?" Getting that boundary right is what makes an agent safe to deploy on anything that touches the real world.

The organizing principle is reversibility. Reading data, drafting text, running a query — cheap to undo, safe to let the agent do freely. Sending an email to a customer, deleting records, moving money, deploying code — expensive or impossible to undo, and exactly where a human confirmation belongs. You classify each tool by its blast radius and gate the dangerous ones.

The approval gate

Mechanically, a permissioned tool doesn't execute on the model's say-so. When the agent calls a gated tool, the loop pauses, surfaces the intended action to a human ("The agent wants to send this email to 4,000 users: [preview]. Approve?"), and only runs it on a yes. The agent's proposal and the human's decision are logged. This is the same loop you've built, with one addition: some tools return "pending approval" instead of a result, and the loop suspends until the human responds.

Why gates matter more with agents than with chat

A chatbot that says something wrong is embarrassing; an agent that does something wrong has acted. And two forces make bad actions more likely than in ordinary software: the model is non-deterministic, and — critically — its context can be poisoned by prompt injection from tool results (Ch. 2). If a web page the agent read says "delete all records," an ungated delete tool will do it. The approval gate is the backstop that ensures a compromised or confused agent still can't take an irreversible action without a human. Gates are not a UX nicety; they're your primary defense against the agent doing real damage.

Design the confirmation for a human who will rubber-stamp it

A gate only works if the human can actually judge the action. "Approve tool call get_ratio(x=74,y=4)?" is unreviewable — people will click yes reflexively. Surface the action in human terms with the real consequence: "Send this refund of $420 to customer #4471?" The quality of the gate is the quality of the summary you show. A gate that induces reflexive approval is barely a gate at all.

Starter code

TOOL_TIER = {"search":"auto", "read_order":"auto",
             "send_email":"confirm", "delete_all":"confirm"}

executed = []

def ask_human(name, args, approve):
    action = {"send_email": f"Email to {args.get('to')}",
              "delete_all": "DELETE ALL RECORDS"}.get(name, name)
    print(f"[gate] Agent wants: {action} — {'APPROVE' if approve else 'DENY'}")
    return approve

def execute(name, args, approve_fn):
    if TOOL_TIER.get(name,"confirm") == "auto":
        executed.append(name); return "ok"
    if ask_human(name, args, approve_fn(name)):
        executed.append(name); return "done"
    return "blocked by human"

# Poisoned trajectory: a tool result told the agent to wipe data.
PLAN = [("search",{"q":"policy"}), ("delete_all",{})]

# TODO: run PLAN with approve_fn = lambda n: n != "delete_all" (deny deletes),
# then assert "delete_all" not in executed.

approve_fn = lambda name: name != "delete_all"     # human denies the wipe
for name, args in PLAN:
    print(name, "->", execute(name, args, approve_fn))

assert "delete_all" not in executed
print("executed:", executed)     # => ['search']  (delete never ran)