How to measure test coverage beyond code coverage

"What's our test coverage?" usually gets answered with a single percentage from the CI dashboard. 78%, say. Everyone nods, and nobody in the room could tell you whether checkout with a saved card and an expired coupon has ever been tested.

Code coverage is a real and useful number. It just answers a narrower question than the one people are asking. Here's how to measure coverage in a way that tells you what's actually tested, and where the gaps are that no percentage will show you.

What code coverage tells you, and what it doesn't

Line and branch coverage tell you which code ran while the tests were running. That's valuable in one direction: code that never runs under test is definitely untested. A file sitting at 0% is a clear finding.

The other direction doesn't hold. Code that ran isn't necessarily checked. A test can execute a function, ignore the result, and still add to the number. And code coverage can't see anything that isn't code: a requirement nobody implemented, a flow that spans three services, a browser the app breaks in.

Google's testing team published a sensible take on this in Code Coverage Best Practices. Their general guideline treats 60% as acceptable, 75% as commendable and 90% as exemplary, and they're explicit that teams should pick a target that fits their own risk rather than chase a mandated number. Keep measuring it. Just don't let it stand in for the whole picture.

The dimensions that matter

A more useful coverage picture looks at a handful of dimensions, each answering a different question.

Requirements coverage. For each requirement, user story, or acceptance criterion, is there at least one test that checks it, and did that test pass on the latest build? This is the closest thing to the question stakeholders actually mean. It needs traceability, a link from each test case back to what it verifies, and that link is the part most teams don't have.

Risk coverage. Not all requirements are equal. Payment flows, auth, data deletion, and anything with legal exposure deserve more than one happy-path test. Weight the areas by risk and check that your deepest testing is where the risk is highest. It's common to find the opposite: lots of tests on stable, low-stakes screens because they were easy to write.

Journey coverage. The paths real users take through the product, end to end. Your analytics already know the top journeys. Compare that list against your end-to-end tests. The gaps are usually the multi-step flows that cross team boundaries, because no single team owns them.

Environment and configuration coverage. Which browsers, devices, locales, plans, and feature-flag combinations have been tested? You won't cover every combination, and you shouldn't try. But you should know which ones you're deliberately skipping.

Data coverage. Empty accounts, huge accounts, accounts migrated from an old schema, unicode in names. Plenty of bugs only show up with specific data shapes, which is why test data deserves its own line on the coverage report.

How to actually measure it

1. Start with a list. Requirements from your tracker, top journeys from analytics, supported browsers from your support policy. You can't measure coverage of things you haven't written down
2. Link tests to what they cover. Each test case points to the requirement or journey it verifies. Tags or a field in your test management tool are enough. The goal is being able to query it
3. Build the matrix. Rows are requirements or journeys, columns are their tests and latest results. Empty rows are gaps. Rows where every test is red or skipped are gaps too, just better hidden
4. Weight by risk. A gap in password reset matters more than a gap in the footer. Sort the empty rows by risk and work down the list
5. Review it every release. New requirements show up as new empty rows. If nobody looks, the matrix slowly becomes another dashboard nobody opens

If you already write test cases from requirements, most of step two happens naturally. The link is there at the moment the test is written, which is much cheaper than reconstructing it later.

Coverage of what, when?

One more thing that single percentages hide: time. A requirement that was tested three months ago and hasn't been touched since is covered on paper. Whether it still works is a different question. Report coverage against the latest run, not against whether a test exists. "Tested" should mean tested on this build.

That's also where coverage ties back to release decisions. A release report that says "all critical requirements have a passing test on this build, and these two medium-risk ones don't" gives a product owner something they can actually decide on. 78% doesn't.

Watch out for Goodhart

Any coverage number that becomes a target will get gamed, usually without anyone meaning to. Code coverage targets produce tests that execute code without checking it. Requirements coverage targets produce one shallow test per requirement to turn the row green. The fix is the same in both cases: treat coverage as a map for finding gaps, and pair it with a quality check on the tests themselves, like mutation testing for unit tests or a review of what each test actually asserts. We covered that in more detail in how to review AI-generated tests.

qtrl keeps test cases, plans, and runs in one place with full traceability, so the requirements matrix above is something you query rather than build in a spreadsheet. Every run is recorded per environment, which means "covered" reflects the latest build, not the last time someone remembered to test it.

qtrl also suggests new tests based on coverage gaps, and you review and approve each one before it runs. Try it free.