Tools, Interviews & Hiring
This chapter maps the book onto the tools you'll name in interviews, goes through the questions you're likely to get, and closes with notes on landing a first agent job and on hiring someone to build agents for you.
How Marcus got the offer
Marcus had never held the title "agent engineer." He had one repo: a coding agent that fixes failing tests in a small codebase, with real tools, a step budget and an approval step on shell commands, plus an evaluation harness that ran it on 30 tasks and reported completion, average steps and cost. His write-up opened with a finding: the agent kept rerunning a broken command because it couldn't see the previous result. After he made tool output readable and added no-progress detection, completion rose from 55% to 80% and average steps fell from 14 to 8.
In the interview, asked how he'd evaluate an agent, he opened his dashboard. Asked what he'd do about an agent that loops forever, he described the fix from his own logs.
Tools, by concern
| Concern | Tools to be able to name | How to choose |
|---|---|---|
| Core loop | A model provider's SDK with tool use; your own loop or a framework | Write the loop yourself first so a framework doesn't hide it (Ch 1–3) |
| Orchestration | LangGraph, the OpenAI Agents SDK, CrewAI, AutoGen | Useful for state and graphs at scale; watch the token cost (Ch 4, 6) |
| Memory | Vector stores such as pgvector, Pinecone or Qdrant; a database for exact state | Summarize the story, store the facts (Ch 5) |
| Evaluation | Your own task-set harness; outcome and path scoring; a model judge for soft quality | Report success with steps and cost (Ch 8) |
| Tracing | LangSmith, OpenTelemetry-based tracing, per-step token and time capture | Trace every step (Ch 10) |
| Safety | Approval steps, tool tiers, sandboxes, injection defenses | Gate by reversibility; distrust tool output (Ch 2, 7, 9) |
"I treat an agent as a loop with tools and a stopping rule, and I engineer everything around the model. I measure success, steps and cost on a fixed task set, trace every run, gate irreversible actions behind a person, and only add planning, review or more agents when the numbers show it helps."
Questions you'll hear
"When would you not build an agent?" When you can write down the steps. A fixed sequence is a workflow, and an agent only adds latency, cost and unpredictability.
"Our agent sometimes runs forever. How do you fix it?" A hard step limit, stuck detection on repeated actions with a nudge, and a per-request cost budget. Log why each run stopped and watch how often runs hit the limit.
"The agent keeps calling the wrong tool. Where do you look first?" The tool descriptions. Make each say what it does, when to use it and when not to. With hundreds of tools, retrieve the relevant few for each request.
"How do you know one version is better?" A fixed task set with checkable outcomes, scored on success, steps and cost, run several times because agents vary, with a breakdown of failure reasons.
"How do you let an agent take real actions safely?" Sort tools by reversibility; irreversible ones wait for a plainly worded human approval, which also protects against injection. Make actions that change things idempotent so retries don't repeat them.
"When would you use several agents?" Only for independent, parallel work valuable enough to justify roughly ten times the tokens, with structured handoffs. Otherwise start with one.
Ridgeline's finished agent has five clear tools, a step and cost budget with a real stopping rule, managed context, limited retries, an approval step on refunds over $50, full traces and a 30-ticket evaluation suite. You can now draw it on a whiteboard and explain each decision.
Build one real project. Marcus's repo is a good template. Pick a narrow task, give the agent three to five tools, add a step budget and an approval step on anything risky, and build a task set of 20 to 30 checkable cases. Report success, steps and cost, and write up one problem you found and fixed.
Search broadly. Titles vary: Agent Engineer, AI Engineer, Applied AI Engineer, LLM Engineer, AI Automation Engineer. Many roles that are mostly agent work don't say so in the title.
Lead your resume with results. For example: "Built a tool-using coding agent with step budgets and approval gates; raised task completion from 55% to 80% and cut average steps from 14 to 8 on a 30-task evaluation suite."
A two-week plan. Read Chapters 1 to 6 and do the labs in the first week. Read 7 to 12, do the labs and build the project in the second. Practice the questions above out loud, then apply.
Where agents fit. Start with tasks where the steps vary and mistakes are cheap to undo, such as research, drafting and triage. Add actions that move money or contact customers only after the agent has a track record on your task set and those actions sit behind approval.
Who to hire. Look for engineers with strong backend habits (retries, idempotency, tracing, rate limits) who also measure. Ask candidates when they wouldn't build an agent, how they'd stop one that loops forever, how they'd compare two versions, and how they'd let it take real actions safely. Then ask them to show something they've built and walk you through a trace.
A one-page scorecard. Rate each from 1 to 3. Do they know when a workflow is enough? Do they design stopping rules and budgets? Do they measure success together with steps and cost? Do they gate irreversible actions and distrust tool output? Can they debug from traces? Have they built something real?
Quiz · Chapter 12
- An interviewer asks when you'd avoid building an agent. Strongest answer?
- To stop runaway agents, which layers belong together?
- An agent calls the wrong tool. Where do you look first?
- Comparing two agent versions, what should you never report on its own?
- What else does the approval step protect against?