AI Engineer Dojo Contents
AI Agent Engineer · Chapter 12

Tools of the Trade & Interview Prep

You now have the mental models and the measurement discipline. This chapter maps them onto the real tools you'll name in an interview, then rehearses the questions — with the senior answers — that separate someone who has shipped agents from someone who has read about them.

The stack, by concern

ConcernTools you should be able to nameWhat to say about choosing
Core loop / tool useAnthropic SDK (tool use, claude-opus-4-8); raw loop vs. a frameworkStart with the raw SDK loop — you must understand it before a framework hides it (Ch. 1–3)
Orchestration frameworksLangGraph, the Agents SDK, CrewAI, AutoGenAdopt for graph/state management at scale; don't let them obscure the loop or the token cost (Ch. 4, 6)
Memory / long-termVector stores (pgvector, Pinecone, Qdrant) for retrieval-based memory; a real DB for precise stateSummarize the narrative, externalize precise facts to tools/DB (Ch. 5)
EvalCustom task-set harness; trajectory + outcome scoring; LLM-as-judge for soft qualityOwn the harness; report success with steps and cost (Ch. 8)
ObservabilityTracing (LangSmith-style, OpenTelemetry); per-step token/latency captureTrace every step; a run you can't replay is a bug you can't fix (Ch. 10)
Safety / controlApproval gates, tool tiering, sandboxing for code exec, prompt-injection defensesGate by reversibility; treat tool output as untrusted (Ch. 2, 7, 9)
The one-line philosophy that lands in interviews

"I treat an agent as a loop with tools and a stopping rule, and I engineer the scaffolding — not the model. I measure success, steps, and cost on a fixed task set, I trace every run, I gate irreversible actions behind a human, and I only add complexity — planning, reflection, multi-agent — when the harness proves it moves the number." That sentence signals you build agents the way senior engineers do: empirically and safely.

Interview simulation — the questions and the senior answers

Q1. "When would you not build an agent?"

Weak answer: "Agents are always better." Senior answer: "Whenever I can write down the steps. If the control flow is fixed — fetch, summarize, route — it's a workflow, and an agent just adds latency, cost, and non-determinism for flexibility I don't need. I reach for an agent only when the steps depend on what the model discovers mid-task." (Ch. 1.)

Q2. "Our agent sometimes runs forever. How do you fix it?"

Senior answer: "Three layers. A hard max_steps cap so it always terminates. Stuck-detection on repeated (tool, args) to bail early with a nudge before the cap. And a per-request token/dollar budget so tail-risk is bounded. Then I log the stop reason — natural/stuck/capped — and watch the capped rate as a health metric." (Ch. 3, 10.)

Q3. "The agent keeps calling the wrong tool. Where do you look first?"

Senior answer: "The tool descriptions, before anything else. Routing is driven almost entirely by the name and description text. I make each say what it does, when to use it, and when not to — I've seen that alone take routing from the low-60s to the mid-90s. If there are hundreds of tools, I stop loading them all and retrieve the relevant few per query." (Ch. 2, 11.)

Q4. "How do you know one agent version is better than another?"

Senior answer: "A fixed task set with checkable outcomes, scored on success rate, steps, and cost — never success alone, because a higher success rate at 4× the steps is often a worse product. Agents are non-deterministic, so I report a rate over the set, and I keep a failure taxonomy to know what to fix. Every change — a prompt, reflection, a new model — is a before/after on that harness." (Ch. 8.)

Q5. "How do you let an agent take real actions safely?"

Senior answer: "Tier tools by reversibility. Read-only and easily-undone actions run automatically; irreversible ones — sending, deleting, payments — pause for a human approval that shows the action in plain terms. That gate is also my backstop against prompt injection: even if a tool result poisons the context into proposing something destructive, it can't execute without a human yes. And mutating tools are idempotent so retries don't double-fire." (Ch. 7, 9.)

Q6. "Multi-agent — when and why?"

Senior answer: "Only for wide, parallel, independent sub-tasks whose value justifies roughly an order-of-magnitude more tokens — deep research, fan-out gathering. Orchestrator–worker with structured handoffs. For sequential or interdependent work I stay single-agent, because I'd pay the token multiple for no parallelism. I start single and graduate only when a real isolation or parallelism need shows up." (Ch. 6.)

Case study

How Marcus got the offer

Marcus had no "agent engineer" title on his résumé. What he had was one repo: a coding agent that fixes failing tests in a small codebase — real tools, a step budget, a permission gate on shell commands — wrapped in an eval harness that ran it over 30 tasks and reported completion rate, average steps, and cost. His write-up led with a finding: the agent kept re-running the same broken command because it couldn't see the previous result; after he surfaced tool output cleanly and added no-progress detection, completion went from 55% to 80% and average steps from 14 to 8.

In the interview he didn't recite definitions. Asked "how would you evaluate an agent?", he opened his own dashboard. Asked "it loops forever — what do you do?", he described the exact fix from his own logs. The measured artifact was the interview. One real, instrumented agent beat every candidate who could only describe agents in the abstract.

Running case · Ridgeline

Ridgeline's finished agent — five sharp tools, a step-and-cost budget with a real stopping rule, curated context, bounded retries, a human gate on refunds over $50, full traces, and a 30-ticket eval suite — is exactly the architecture you can now whiteboard and defend, decision by decision, in an interview.

Quiz · Chapter 12 — reasoning, not recall

  1. An interviewer asks when you'd avoid an agent. The strongest answer:
  2. To stop runaway agents, the senior answer layers:
  3. An agent calls the wrong tool. First place to look:
  4. Comparing two agent versions, you should never report:
  5. The approval gate doubles as a defense against:

← Back Continue →

AI Agent Engineer — New-Grad Edition · AI Engineer Dojo · aiengineerdojo.com