How to Test an Agent

Testing ordinary software is already hard; agents add several orders of magnitude: non-deterministic output (the same input produces a different result each time), multi-step execution (any intermediate step can fail), and external tool dependencies (the agent breaks when a tool changes). Many teams give up on testing agents as a result and rely entirely on manual poking—launching on luck.

This article offers a practical, layered agent testing strategy.

1. The Hard Parts of Agent Testing

Face the hard parts first:

• Non-determinism: asserting "output equals X" is almost infeasible.
• Multi-step chains: an agent calls the model/tools multiple times per task; any step can break.
• External dependencies: tools talk to real services, and tests should not actually send emails or transfer money.
• Cost: running an agent costs tokens; running the full test suite is not cheap.

So agent testing cannot copy traditional unit tests verbatim. It needs layering, stubbing, and quality scoring rather than equality checks.

2. A Layered Testing Strategy

Unit (tools)  →  Integration (end-to-end)  →  Evaluation (quality)  →  Chaos (fault tolerance)

Each of the four layers has a different focus.

3. Layer One: Unit Tests (Tool Functions)

An agent's tools (search, query_db, send_email) are deterministic functions—test them like normal:

• Parameter validation (illegal arguments throw).
• Correct return on valid input.
• Boundary cases (empty input, very long strings, special characters).
• Error handling (return a readable error when the upstream fails).

test("query_order rejects other user's order", async () => {
  await expect(queryOrder({ id: "o_other", userId: "u_me" }))
    .rejects.toThrow("forbidden");
});

This layer is the foundation—when the agent misbehaves, suspect the tools first. Once the tools are solid, test the agent orchestration.

4. Layer Two: Integration Tests (End-to-End, but Stubbed Model/Tools)

Test the whole agent's orchestration logic: give it a task and watch it call the right tools and follow the right flow. The hard part is non-determinism; the answer is stubs:

• Stub the model: do not let the agent actually call the LLM (expensive + non-deterministic). Use a mock that returns a preset "tool-call decision". You are testing "given the model wants to call X, does the orchestration layer execute X correctly and feed the result back in".
• Stub the tools: the tools do not hit real services (send_email does not actually send); they return preset results. You are testing "when a tool returns Y, how does the agent continue".

test("agent calls search then answers", async () => {
  const agent = createAgent({
    llm: mockLLM([                // preset the model's two-step decision
      { tool: "search", args: { q: "x" } },
      { text: "the answer is..." },
    ]),
    tools: { search: async () => "search results..." },
  });
  const result = await agent.run("ask a question");
  expect(result).toContain("the answer is");
  expect(toolCalls).toEqual([["search", { q: "x" }]]);
});

This layer validates orchestration logic (order, result feeding, stop conditions), not model intelligence (that is the evaluation layer's job).

5. Layer Three: Evaluation Sets (Quality, the Most Critical)

Integration tests only check "is the flow right"; they cannot check "is the answer good". Quality relies on evaluation sets:

• 20–100 items of task → expected attributes / reference answer.
• Run them every time you change the agent (prompt/model/tool).
• Score with LLM-as-judge or rules.

Choose metrics by agent type:

• Correctness: is the answer right (compared to reference answer / key facts).
• Format compliance: when JSON is expected, is it valid and complete.
• Safety: did it perform operations it should not (privilege escalation, leakage).
• Efficiency: average steps, token consumption (more steps for the same work = regression).

This layer is the real guarantee of agent quality. Without an evaluation set, prompt changes are pure gut feel; with one, changes become regressable.

6. Layer Four: Chaos Testing (Fault Tolerance)

Agents run in real environments where tools fail, models time out, and networks drop. Test whether it stays stable under failure:

• Simulate tool failure: make search randomly return errors and see whether the agent handles it gracefully (retry / take another path / inform the user) instead of crashing.
• Simulate model timeout: verify the timeout mechanism fires and the fallback triggers.
• Simulate abnormal returns: the tool returns invalid JSON; see whether the agent can recover.

Chaos testing exposes fragility that "happy path tests" cannot cover. Most agent production incidents are not crashes in the normal flow; they are cases where a tool fails sporadically and the agent runs off the rails.

7. Reproduce Bad Cases with Traces

When an agent has a bug, the hardest part is reproduction (non-determinism). The fix: record a trace for every run (input/output/tool calls per step) and replay the trace when something goes wrong.

• A trace lets you see exactly "which step, which model call, what prompt, what return" went wrong.
• For especially hard-to-reproduce cases, turn the trace into a regression test (freeze that run's tool returns and test the orchestration layer).

Without traces, debugging an agent is pure guesswork. Traces are the foundation of agent debuggability.

8. Cost Control

Running agent tests costs tokens; control it:

• Use a cheap model in CI: run the evaluation set on Haiku/Mini (validates flow + rough quality); use the strong model for the official evaluation.
• Sampling: PRs that do not change the agent do not run the full evaluation (only integration tests).
• Caching: cache tool mock results; cache model outputs for identical inputs (saves on evaluation reruns).

9. A Minimal Starting Point

Do not lay out all four layers at once; follow this order:

1. Build tool unit tests first (cheapest, most foundational).
2. Add end-to-end integration tests (stubbed model/tools), covering core scenarios.
3. Build a 20-item evaluation set, run it on every prompt/model change.
4. Ship traces, and start accumulating bad cases.
5. Chaos testing, added once the agent stabilizes.

The first three steps are low-cost, high-return; they take your agent from "launching on luck" to "launching with confidence".

Summary

Agent testing must be layered; you cannot force traditional unit tests onto it:

• Unit tests for tools (deterministic, test normally).
• Integration tests for orchestration (stub model/tools, test the flow).
• Evaluation sets for quality (most critical, regressable quality).
• Chaos tests for fault tolerance (robustness under failure).
• Reproduce bad cases with traces, turn them into regressions.

Remember one thing: agent non-determinism does not mean "untestable"; it means "test it differently". Land this layered strategy and your agent can move from "the demo runs" to "production-trustworthy"—and the value of that cannot be overstated.