Generating Tests with AI: A Practical Guide

Every team has test debt. The reasons are highly consistent: writing tests is slower than writing code, deadlines loom overhead, the moment a refactor comes the tests break first, and backfilling tests is always "the next thing." The result is that core logic runs naked, and only when something breaks in production do you remember to backfill a regression test.

AI test generation looks like a silver bullet for this problem—give it a file, and in ten seconds it spits out 80% coverage. But in real use you'll hit two kinds of pitfalls: first, the tests it generates assert nothing—they pass, but it's as if they weren't tested; second, it churns out a pile of fake coverage, with 100% line coverage that can't catch a single bug, while also making CI slower and refactors harder.

AI test generation isn't "one-click debt repayment"; it's "automating the manual labor of backfilling tests." Manual labor can be automated; design work can't. This article explains how to nail the automatable part and leave the non-automatable part to humans.

1. First, separate them: which tests AI writes well, and which it doesn't

Before letting AI write tests, first judge which category the code under test falls into.

What AI writes well is essentially "deterministic input/output, clean dependencies":

• Pure functions: formatters, parsers, validators, money calculations, date handling. Clear signatures, no side effects, obvious assertion points—AI almost never face-plants.
• Boundary conditions: null / undefined / empty arrays / negatives / ultra-long strings / Unicode / timezone edges. These are AI's strengths—corners humans are too lazy to write, it covers one by one.
• Parameterized (table-driven) tests: given a set of (input, expected), generate a list of it.each cases.
• Regression tests: give it a bug report and reproduction steps, and it can freeze the failing case into a test. This is one of the highest-ROI uses.
• Snapshot tests: serialized output, component render trees, a CLI's stdout.

What AI writes poorly is essentially "needs to understand the whole picture or business intent":

• Architectural tests: module boundaries, dependency direction, layering constraints. These require it to first understand the whole system, but it usually only sees the file you pasted.
• Business-semantic tests: "is this discount-stacking rule correct," "is this checkout flow consistent with what product wants"—AI doesn't know what "correct" looks like; it can only describe the code's current state in terms of the code's current state, which is self-proving.
• Exploratory / performance tests: where things might be slow, where concurrency conflicts might arise, where there's a race condition—these require humans to think; AI can only mechanically cover explicit branches.
• Test design itself: what should be tested, what shouldn't, which paths are high-risk—this is the most valuable part of testing, and the least outsourceable.

In one sentence: AI is good at filling in "breadth," not good at filling in "judgment." Hand it the breadth; keep the judgment for yourself.

2. Workflow: not one-shot generation, but a "generate→run→fix→accept" loop

The most common misuse of AI for writing tests: paste a chunk of code, ask it to "write tests," then paste the output into a file. Tests produced this way most likely won't run, or they run but test the wrong thing.

The right approach is a closed loop, with only one core principle: the generated tests must run; if they don't, have it fix them itself until they do.

1. Analyze the code under test: first nail down the signature, types, and dependencies. The key is what's an external dependency (db, HTTP, file system, time, random numbers)—these must be mocked; and what's an internal collaborator—these shouldn't be mocked.
2. Generate: have AI produce test cases based on the signature and dependency list, including the normal path, boundaries, and exceptions.
3. Execute and validate: immediately run vitest run / jest, and feed the result back to AI.
4. Fix failures: test failures come in two kinds—the mock is misconfigured (fix the test), or a real bug is exposed (fix the implementation, and you've struck gold). AI fixes itself based on the error message; don't relay it yourself.
5. Human acceptance: the last gate is you taking a look—what behavior is this test actually testing, and would it still pass with a different wrong implementation?

Agent tools like Claude Code / Cursor are naturally suited to this loop, because they can run commands themselves, read errors, and iterate. A prompt you can use directly looks like this:

Generate Vitest unit tests for calculateDiscount in src/pricing/calculate.ts.
Requirements:
1. First read the function signature and return type, identify all external dependencies, and mock them correctly.
2. Cover: normal discount, threshold boundaries, negative/0/empty inputs, stacking-rule caps.
3. Each assertion must verify a concrete business expectation; empty assertions like toBeDefined / toBeTruthy are forbidden.
4. After generating, run `pnpm vitest run src/pricing/calculate.test.ts`,
   fix any failures yourself until everything passes. Paste the final coverage to me.

The key is the last item: let it self-validate. Agents can read errors and self-correct, an order of magnitude faster than you pasting back and forth. If the tool you're using can't run commands (Copilot-style completion), then you must run the tests it generates manually—don't get lazy. Tests that don't run are worse than no tests: they rot, give false confidence, and block CI.

A ruthless way to judge whether a generated test has value: mutation testing. Tools like Stryker intentionally introduce bugs into your implementation (change > to >=, + to -), then see whether the tests fail. If the code is broken and the test still passes, the test is effectively useless. AI-generated tests are especially worth running through mutation—it instantly exposes the junk tests "written for coverage."

3. Quality control: don't let coverage become a trap

Coverage is the metric most easily gamed by AI. The coverage tool would give this test a perfect score:

// Looks tested, actually tests nothing
describe("formatPrice", () => {
  it("works", () => {
    const result = formatPrice(99);
    expect(result).toBeDefined();
    expect(typeof result).toBe("string");
  });
});

100% line coverage, but the day formatPrice changes ¥99 to ¥0, this test still passes. It asserts no business expectation; it only proves "the function returned something."

A good test has only one criterion: can it catch a reasonable wrong implementation. Based on that, here are a few hard rules:

• Assertions should land on business expectations, not on "it returned." expect(formatPrice(99)).toBe("¥99.00") is a good assertion; expect(result).toBeTruthy() is a junk assertion. AI loves generating the latter—delete them all during review.
• Test behavior, not implementation. Don't test internal methods to cover a branch of a private function, and don't mock internal calls to death to scrape together branch coverage. If a refactor makes all the tests fail, those tests are a liability, not an asset.
• Reject duplicate tests. If three cases for the same branch are all testing the same thing, keep only the most representative one and delete the rest. AI generates a lot of homogeneous cases to "look diligent."
• Coverage is a byproduct, not the goal. Good tests naturally bring reasonable coverage; tests written for coverage yield a pile of line-counting garbage. Using coverage as a gate is fine (to prevent obvious omissions); using it as a KPI will backfire.

A practical principle: reviewing AI-generated tests is more important than reviewing the implementations it generates. Because a wrong implementation gets caught by tests (if you wrote the tests correctly), but a wrong test catches nothing—it sails through green all the way into main.

4. Mocks and dependencies: where AI most easily face-plants

When AI test generation face-plants, eight times out of ten it's on mocks. Typical symptoms:

• Mocks too deep: mocking all internal modules and internal functions, leaving the code under test as a single layer of if-else—effectively testing the mocks, not the code.
• Mocks too shallow: the database that should be mocked isn't, and the test actually connects to the dev database—sometimes fast, sometimes slow, and it pollutes the data.
• Forgot to reset: mock state from the previous test leaks into the next, producing mystical "passes alone, fails together" failures.
• Mocked the wrong layer: what should be mocked at the repository layer gets mocked at the fetch layer, so when you swap HTTP libraries all the tests are dead.

The correct mock principle: mock only at boundaries—database clients, HTTP clients, external APIs, time, random numbers, file systems. Use real implementations for internal collaborators. This way what you're testing is "real code + fake outside world," not "fake code + fake world."

import { describe, it, expect, vi, beforeEach } from "vitest";

// Correct: mock at the boundary (sending email); use real code for internal logic
vi.mock("../email/client", () => ({
  sendEmail: vi.fn().mockResolvedValue({ ok: true }),
}));

import { sendEmail } from "../email/client";
import { checkout } from "./checkout";

beforeEach(() => vi.clearAllMocks()); // Key: reset per case

describe("checkout", () => {
  it("sends a confirmation email after a successful order", async () => {
    const order = await checkout({ userId: "u1", items: ["sku1"] });

    expect(order.status).toBe("confirmed");
    expect(sendEmail).toHaveBeenCalledOnce();
    expect(sendEmail).toHaveBeenCalledWith(
      expect.objectContaining({ to: expect.any(String), subject: "Order Confirmation" }),
    );
  });
});

Two details not to miss: one is beforeEach(() => vi.clearAllMocks())—omit it and you get cross-case pollution; the other is don't mock what you don't own—don't mock a third-party library's internal methods; wrap it in your own adapter and mock that adapter layer, otherwise when the library upgrades all your tests die.

And two more hidden dependencies: time and randomness. Date.now(), new Date(), Math.random(), crypto.randomUUID()—as long as the code under test touches them, the test will "pass today, fail tomorrow." In Vitest use vi.useFakeTimers() to lock time, and use injection or vi.spyOn to fix ID generation. AI often misses these; you have to explicitly remind it.

5. Tool selection: Claude Code, Cursor, Codium, in-house

Different tools suit different scenarios; don't expect one to rule them all.

• Claude Code / Cursor (agent mode): suited to the "generate + self-validate" loop. They can run tests themselves, read errors, and iterate until passing. Primary scenario: backfilling tests for a module from scratch. Give it the file path + test requirements + let it run to green itself, and you free up your hands for other things. The key is to give clear constraints (which framework, which layer to mock, which assertions are forbidden), otherwise it writes by its default habits.
• CodiumAI / Qodo Cover: tools specialized in test generation, with a strength in scenario enumeration—it proactively asks "did you test this exception branch," "did you consider this null input," which is especially useful for "what did I miss." Good for backfilling behavior tests on legacy code, though the mocks it generates are sometimes verbose and need trimming.
• Diffblue (Coverage Advisor, mainly Java): analyzes at the bytecode level, suited to batch backfilling tests in enterprise Java repos; not useful for TS projects, but the approach is worth borrowing—it emphasizes "generating based on real execution paths."
• In-house agent: most cost-effective when the team has unified testing conventions (fixed mock factories, fixed fixtures, fixed directory structure). Use the Claude API + tool use to run tests yourself, stuffing project norms, mock templates, and forbidden items into the system prompt; the generated tests directly match team norms with almost no manual adjustment. High upfront investment; highest payoff at scale.

Selection really comes down to one sentence: use an agent for a closed loop, use Codium for scenario enumeration, build in-house for unified norms at scale. For most teams the first two are enough; in-house is a concern after you scale.

6. The part AI can't replace: test design

Having covered so much of what AI can do, let's draw the boundary clearly—AI can supplement coverage, but it can't replace "test design" itself.

The truly valuable part of testing is never those hundreds of lines of it(...). It's these:

• What to test, what not to test: resources are limited; which paths yield the highest return? AI doesn't know which things are core assets and which are one-off scripts.
• Where the risk is: if this function is wrong, does it cost the company money or just misalign the UI? Priority is a business judgment; AI can't see it.
• What "correct" looks like: the right/wrong of business rules lives in the product docs and in users' heads, not in the code. AI can only restate the code's current state.
• Design feedback: when writing a function you discover "this thing is impossible to test," which often signals a design problem—dependencies too heavy, responsibilities too tangled, side effects too deep. This "test-driven design" insight is the biggest byproduct of humans writing tests; if AI writes them for you, the signal is lost.

So the healthiest use is: AI is responsible for spreading out already-designed tests; humans are responsible for designing tests. You decide which behaviors to test, which boundaries matter, which risks are unacceptable; AI is responsible for quickly turning those decisions into executable test code, and for helping you notice "oh, you forgot to test this branch." It's a tireless executor, not a test architect.

Summary

• Categorize before you act: pure functions/boundaries/parameterized/regression suit AI; architectural/business-semantic/exploratory/test design don't. Give AI the breadth; keep the judgment for yourself.
• Must be a closed loop: generated tests must run; let the agent execute, read errors, and fix until passing. Tests that don't run are worse than no tests.
• Strict quality control: reject empty assertions, reject duplicates, reject testing for coverage's sake. Review AI's tests as strictly as you'd review its code, because wrong tests catch nothing.
• Mock only at boundaries: mock external dependencies like database/HTTP/time/random; use real implementations for internal collaborators. Don't forget reset and fake timers.
• Coverage is a result, not a goal: it's the natural byproduct of good tests. Use it as a gate, but never as a KPI.

AI can turn test debt from "never payable" into "payable," but what it pays off is manual labor debt, not brainpower debt. The decision of what to test always has to be yours.