Hiring an AI Agent Engineer

Five questions to ask candidates, what a strong versus weak answer sounds like, and a one-page scorecard. You don't need to build the agent yourself — you need to evaluate the person who will build something that takes actions in your name.

1. "What makes something an agent, and when would you not use one?"

Strong: a model in a loop with tools and a stopping rule; would use a script or single call for fixed workflows, reserving agents for genuinely variable, multi-step tasks. Weak: "an agent is a smart AI that does things" — and would make everything an agent.

2. "Our agent sometimes loops forever. How do you fix it?"

Strong: a hard step/cost budget as a backstop, then a real stopping rule — clear definition of done, legible tool results, no-progress detection — and measures average steps to confirm. Weak: "raise the max iterations" or "bigger model."

3. "It has a tool that can delete data or move money. How do you make that safe?"

Strong: a human-in-the-loop gate on irreversible actions, read/write separation, least-privilege access, and awareness that this also contains prompt injection. Weak: "tell it in the prompt to be careful."

4. "How do you evaluate an agent, and keep its cost sane?"

Strong: a task suite with checkable success conditions; reports completion, steps, and cost per task with a sample size; evaluates the trajectory, not just the final answer; attacks cost via step count and lean context. Weak: "I run it and see if it looks right," with no sense of cost drivers.

5. "Show me something you've built."

Strong: walks through a real agent — tools, stopping rule, a gate, an eval harness — with before/after completion numbers and a failure they diagnosed from traces. Weak: only names frameworks and courses, with nothing measured to show.