Agents & launch

Chapter 8 · Phase 7 · practice in Week 13 and Week 14

What you will know after this chapter

What an LLM agent is vs a simple chatbot · the plan–act–observe loop in detail · how tool calling enables real-world actions · when to require human-in-the-loop gates · multi-step evaluation strategies · the production launch checklist

1. What is an LLM agent?

LLM Agent

A system where an LLM acts as a controller: it receives a goal, decides which tools to use, executes those tools, observes the results, and loops until the goal is achieved. Unlike a chatbot, an agent can take multi-step actions with real-world effects — writing code, querying databases, sending emails, browsing the web.

Chatbot vs agent

Chatbot

Single LLM call per user message
One response, no action
Deterministic control flow
User drives each step
No external state changes
Low risk per call

Agent

Multiple LLM calls to complete a goal
Takes real-world actions via tools
LLM decides the next step
Can operate autonomously for many steps
Can write to databases, send messages, run code
Higher risk — mistakes compound across steps

2. The agent loop

ReAct agent loop (Reason + Act)

Receive goal
"Research and summarize all P0 incidents from last quarter"

↓

LLM: Plan
"I need to: 1) query incident DB, 2) filter P0s, 3) retrieve details, 4) summarize"

↓

Act: call tool
query_incidents(severity="P0", since="90d")

↓

Observe result
Returns 12 incidents with IDs

↓

LLM: Reason
"I have the list. Now fetch details for each to extract root causes."

↓

... loops until task complete or step limit reached ...

↓

Final answer

3. Tool calling in agents

Tools are the agent's arms and legs. Each tool is a function your application exposes. The LLM decides which tool to call with which arguments; your code executes it and returns the result.

Common agent tool types

Category	Examples	Risk level
Read-only	Search docs, query DB (SELECT), call read APIs	Low
Compute	Run Python, execute SQL, call calculators	Medium (sandbox required)
Write to DB	INSERT, UPDATE, DELETE	High (require confirmation)
External actions	Send email, post to Slack, create ticket	High (irreversible)
Code execution	Run arbitrary Python/shell	Critical (sandboxed only)

Tool design principle

Every tool the agent can call should be designed with the principle of least privilege. If the agent only needs to read incidents, do not give it a tool that can delete them. Minimize the blast radius of any single bad tool call.

4. Human-in-the-loop (HITL) gates

Not every agent action should run automatically. For high-stakes or irreversible actions, require human approval before execution.

✓

Always gate

Sending communications (emails, Slack messages), modifying production data, running shell commands, making purchases. These are irreversible or high-visibility.

Gate when confidence is low

If the agent expresses uncertainty ("I'm not sure if X or Y is correct here"), pause and ask the user. Define a confidence threshold below which the agent must ask.

✗

Never gate

Read-only queries, search operations, formatting operations. These can run freely — they have no side effects.

How the gate decision works

Before executing any tool call, the system checks two conditions: is this tool on the high-risk list (send email, delete record, post to Slack, run shell commands), OR did the agent express low confidence below a set threshold (typically 70%)? If either is true, the action is paused and a confirmation UI is shown to the user before anything runs. If the user declines, the action is cancelled and the agent receives that feedback so it can try a different approach. If neither condition is met, the action runs automatically without interruption.

The key PM decision is defining what goes on the high-risk list and setting the confidence threshold. Too many gates and the agent becomes annoying and slow. Too few and users lose trust the first time it does something they didn't expect.

5. Evaluating agents — multi-step evals

Agent evaluation is harder than single-call evals because the output is a sequence of steps, not just a final answer. You need to evaluate both the process and the result.

Task completion rate

Did the agent successfully complete the assigned task? Score 0/1. Run on a test set of 20–50 tasks with known correct outcomes.

Step efficiency

How many tool calls did the agent make? More is not always better — an agent that calls 10 tools when 3 suffice is wasteful and slow. Compare to a reference trajectory.

Hallucinated tool calls

Did the agent attempt to call a tool that does not exist? Or call a real tool with fabricated arguments? Log every tool call and validate schema compliance.

Loop detection

Agents can get stuck in loops — calling the same tool repeatedly because the result does not make sense to them. Detect repeated identical calls and abort after N repeats.

How the step limit works in practice

Every agent run has a hard cap on iterations — typically 10–20 steps. On each step, the model either calls a tool (which your system executes and returns results for) or produces a final text answer (which ends the loop). If the agent reaches the step limit without finishing, the run is terminated and returned with a "max steps reached" status. This prevents runaway agents from burning through API budget indefinitely. As a PM, the step limit is both a safety valve and a diagnostic: if your agents are frequently hitting the ceiling, the goal decomposition is too complex and should be broken into smaller sub-tasks.

6. Production launch checklist

Shipping an LLM feature to production requires infrastructure beyond the prototype. Use this checklist before every launch.

📊

Observability

Full request/response logging, latency metrics (p50/p95/p99), error rate dashboard, cost per user/session dashboard. Set alert thresholds before launch, not after.

🔒

Safety and content filtering

Implement input and output classifiers for harmful content. Define explicit policy for what the system will not do. Test with adversarial inputs before launch.

⚡

Rate limiting

Per-user and per-org token quotas. API provider rate limit handling (exponential backoff). Queue for burst traffic. Never let one user exhaust your API quota.

🔄

Rollback plan

Pin the model version and prompt version. Document how to roll back to the previous version in < 10 minutes. Test the rollback procedure before launch day.

📋

Eval baseline

Run your eval suite against production before and after launch. Record the baseline scores. If post-launch scores drop, you have a regressions signal.

📣

User-facing disclosure

Users must know they are interacting with AI. Label the feature clearly. For regulated domains (healthcare, legal, finance), include appropriate disclaimers.

🌊

Gradual rollout

Do not flip to 100% on day one. Start at 1% → 5% → 20% → 100% with a hold period at each stage. Monitor all metrics at each stage before expanding.

You made it through the course

You now have the vocabulary, mental models, and practical skills to contribute meaningfully to every AI feature your team builds. Foundations → Prompting → SQL/data → Visualization → RAG → Chatbots → Prototypes → Agents & launch. Go build something.

← Prototypes

Practice → Week 13 Back to home

Test your understanding

What is an 'agent loop' and what are its three key components?v

(1) Observe — receive current state (user input, tool output, memory). (2) Think — the LLM reasons about what to do next. (3) Act — execute a tool call, generate output, or terminate. The loop repeats until a stopping condition is met.

What is a human-in-the-loop gate and when should you add one?v

A HITL gate pauses the agent and asks a human to approve before continuing. Add one when the next action is: (1) irreversible (send email, write to database), (2) high-stakes (medical, legal, financial), or (3) low-confidence (model expresses uncertainty).

Name two failure modes specific to multi-step agent tasks.v

(1) Action compounding: 95% accuracy per step means 60% end-to-end success over 10 steps. Small errors compound. (2) Unrecoverable state: the agent takes an irreversible action mid-task (deletes a file, sends a message) then fails later, leaving the system in a broken partial state.

What is the difference between a tool-calling agent and a simple chain?v

A chain is a fixed sequence of LLM calls defined at build time. A tool-calling agent decides at runtime which tool to call and when to stop, based on the model's output. Agents are more flexible but less predictable and harder to debug.

What would you include in a production launch checklist for an AI agent feature?v

(1) Offline eval suite passing threshold. (2) Human review of 50 random end-to-end task completions. (3) Irreversible actions behind HITL or confirmation. (4) Minimum-permission tool access. (5) Per-user rate limits. (6) Cost-per-task monitoring with alerting. (7) Full tool-call logging for audit. (8) Rollback plan if quality degrades.

Why do multi-step evals matter more for agents than for simple Q&A features?v

A Q&A eval tests a single input-output pair. An agent eval must verify a sequence of tool calls achieves the correct end state. A correct final answer reached via a wrong intermediate path is a fragile agent that will fail on real-world variants.