How to review AI-generated tests before you merge them

Ask a coding assistant to "add tests for this" and you'll get tests. Lots of them, neatly named, all green on the first run. The pull request shows coverage going up, and it gets approved with a quick glance, because reviewing tests is nobody's favorite part of code review.

That quick glance is the problem. A test that passes tells you very little on its own. What matters is whether it would fail if the code were wrong, and AI-written tests fail that check in a few recognizable ways. Here's what to look for and how to review them without reading every line twice.

Why AI tests need a different review

When a coding agent writes both the code and the tests, the tests are usually derived from the code. The model reads the function, works out what it returns, and asserts exactly that. If the function has a bug, the test faithfully locks the bug in.

A human writing tests from a spec, or from what the feature is supposed to do, would at least have a chance of catching the mismatch. Tests generated from the implementation mostly can't. They confirm the code does what it does. That's a big part of why AI coding tools strain existing test suites, and why coverage numbers get less trustworthy as more code is generated.

The patterns to look for

Assertions copied from the output. The expected value is a long, oddly specific literal that nobody would have worked out by hand: a full JSON blob, a float to eight decimal places, a formatted date string. Ask where the number came from. If the honest answer is "the code produced it," the test isn't checking anything.

Everything mocked. The test mocks the database, the HTTP client, the clock, and the two helpers the function calls. What remains under test is a few lines of glue. Sometimes that's the right call. Often it means the test verifies that mocks return what they were told to return.

Weak assertions. toBeDefined(), not.toThrow(), toBeTruthy(), or a check that an array has a length greater than zero. These pass for almost any output, including wrong output. One or two are fine. A test file made of them is decoration.

Only the happy path. Five tests that all pass valid input in slightly different shapes, and none that pass an empty list, a null, a negative number, or a user without permission. The edge cases are where the bugs live and where generated tests are thinnest.

Tests changed to match the code. This one is the most important to catch. An agent makes a change, an existing test fails, and the agent "fixes" it by updating the expected value, loosening the assertion, or adding a skip. The diff looks like routine maintenance. Any time a PR modifies an existing assertion, someone should be able to say why the old expectation was wrong.

Sleeps and retries. In end-to-end tests especially, a fixed wait or a retry loop added to get a test green is a timing bug waiting to come back. Our flaky tests guide covers the better options.

A review routine that actually scales

You can't read 400 lines of generated tests with full attention on every PR. You don't need to. A few habits catch most of it.

1. Read the test names first, alone. Do they describe behavior the feature should have ("rejects expired coupons") or implementation details ("calls validateCoupon")? Behavior names tend to come with behavior tests
2. List the cases that are missing. Before reading the bodies, write down the three edge cases you'd expect. Then check whether they're there
3. Break the code on purpose. Flip a comparison, remove a condition, return early. If nothing goes red, the tests don't cover that line in any meaningful sense
4. Diff the existing tests separately. New test files and edits to old ones deserve different levels of suspicion. Review the edits more carefully
5. Ask for the spec. If there's a ticket or acceptance criteria, the tests should map to it. If they can't, the tests were written from the code

Step three is manual mutation testing, and tools can do it for you. Stryker (JavaScript, TypeScript, C#), PIT (Java), and mutmut (Python) make small changes to your code and report which ones your tests failed to notice. They're slow on big codebases, so run them on changed files rather than everything. A surviving mutant in new code is a specific, actionable review comment.

Give the model better input

Review catches problems after the fact. A lot of them can be prevented by changing what the agent works from.

Point it at the requirement, not just the code. "Write tests for this function" invites tests derived from the function. "Write tests that verify these acceptance criteria, including the failure cases" gives it something independent to check against. That's the core idea behind spec-driven development, and it works just as well at the scale of a single PR.

It also helps to have tests written in a separate step, or by a separate agent, from the code. When the same context writes both, it shares the same blind spots. Splitting them is the same reason you wouldn't have one person write and approve their own change.

And add a rule to your agent instructions file: never modify an existing assertion or skip a test without flagging it in the PR description. Agents follow that kind of instruction reasonably well, and it turns the most dangerous pattern into a visible one.

Unit tests aren't the whole picture

Everything above is about tests that sit next to the code. They're necessary, and they're also the layer most exposed to the "tests derived from code" problem. The layer that checks the product from the outside, against what a user expects to happen, is harder to fool, because it never saw the implementation. That's the layer that tends to get thinner when teams lean on generated unit tests, and it's the one worth protecting. We wrote more about that balance in the verification tax.

qtrl works at that outside layer. Test cases are written from requirements, in plain language, and run against the real app in a real browser, so they check what the product does rather than how the code is built. When qtrl suggests new tests from coverage gaps, they go through review and approval before they run, the same discipline this post recommends for generated unit tests.

If your team is shipping more AI-written code than it can comfortably review, that's a good place to add confidence without adding reading. See how it works.