AI code carries more defects. What that means for tests

The debate about whether AI-assisted code is worse than human-written code has been running on vibes for about three years. It's starting to run on data instead, and the data is more specific than either camp was expecting.

A large-scale empirical study of AI-generated code in the wild, Debt Behind the AI Boom, tracked AI-introduced issues across public repositories and found the count of surviving ones passing 110,000 by February 2026. Surviving is the operative word. These aren't issues caught in review. They're issues that made it in and stayed in.

Alongside it, analyses of AI-assisted pull requests have reported meaningfully higher issue density than comparable human-written changes, on the order of 1.7 times. Take any single number with the caution that methodology deserves. The direction is consistent enough across sources to plan around.

The number that explains the other numbers

The finding worth sitting with isn't the defect rate. It's the acceptance rate: developers take AI suggestions with little or no modification a large share of the time, including suggestions with subtle problems in them.

That's not carelessness. Reviewing a plausible-looking block of code that someone else wrote is hard, and it's harder when the author can't explain their reasoning because the author is a model. The suggestion arrives with the confident formatting of code that works. Reading it closely enough to disagree costs more attention than most people have available at 4pm on a Thursday.

So the defect rate isn't really telling you models write bad code. It's telling you the human checkpoint that used to sit between "code exists" and "code is merged" got thinner.

Why the bugs are a different shape

Here's what matters for test strategy. AI-introduced defects skew differently from the ones your suite was built to catch.

Models are good at syntax and at common patterns. They rarely produce code that fails to compile or falls over on the happy path, which is exactly what unit tests and smoke tests are best at catching. What they produce instead is code that runs correctly and does something subtly wrong: an off-by-one at a boundary, a null check that handles the wrong null, an assumption about ordering that holds in dev and breaks under concurrency, error handling that swallows the case it was supposed to surface.

There's a second category that's worse because it's invisible to every functional test you own: code that works and doesn't belong. The fourth slightly different date formatter. A retry loop that duplicates one three files over. Structural debt passes every test in your repo by definition, and it accumulates at whatever speed you generate code.

What to change in the suite

Weight boundary and edge cases much harder. If your test authoring habit is one happy path plus one obvious failure, that pattern is now poorly matched to where the bugs are. Empty collections, maximum lengths, zero, negative numbers, unicode, timezone boundaries, concurrent writes. Property-based testing earns its keep here more than it used to, because it generates the inputs nobody thought to write down.

Test behavior, not structure. Tests coupled to internal implementation break on every refactor, and under AI-assisted development the code gets reshaped constantly. A suite that fails whenever a function moves teaches people to update tests reflexively, which is precisely the habit that lets a real regression through. The argument is laid out at more length in why AI coding tools broke your test suite.

Don't let the same model write the code and the test. A model that misunderstood the requirement will write a test asserting its misunderstanding, and you get a green build certifying the wrong behavior. Tests derived from the requirement, by a different process or at minimum a separate pass, are the ones that catch intent drift.

Add the checks tests can't make. Duplication detection, complexity thresholds, dependency and license scanning, and security static analysis catch the structural category that no functional test will ever see. On the security side specifically, the patterns are in catching security bugs in AI-generated code.

The coverage percentage is lying to you now

Line coverage was always a rough proxy. It's worse now, because generating tests until coverage hits 85% is something a model will do cheerfully and quickly, and the resulting tests will exercise every line while asserting almost nothing worth asserting.

Coverage tells you which lines executed. It has never told you whether anything was checked while they did. If your team is generating tests with AI assistance, and most are, the gap between those two things is now wide enough that reporting on coverage alone is actively misleading. Mutation testing is the honest measurement, and it's worth the runtime on your core domain logic even if you can't afford it everywhere.

Not an argument against the tools

None of this says stop using AI coding assistants. The productivity gain on the generation side is real, and teams that give it up to avoid the defect rate are trading a manageable problem for a competitive one.

It says the assumptions your test strategy was built on shifted. The checkpoint that used to catch this stuff was a human reading their own code carefully enough to notice it was wrong. That checkpoint is thinner now. Something has to do that work, and the only thing that scales with generation volume is automated verification that people actually trust.

qtrl focuses on the layer where these defects surface: real user journeys in real browsers, asserting on behavior rather than implementation, so a refactor doesn't produce a red suite and a subtly wrong checkout flow does.

Coverage is tracked as journeys and cases rather than a line percentage, which keeps the reporting honest when a lot of the code and a lot of the tests are being generated. You get a view of what's actually verified, not what happened to execute. See how it works.