DORA metrics for QA: the numbers that actually predict quality

Ask an engineering leader how good their test suite is and you'll get a number back fast: test count, coverage percentage, pass rate on the last run. Ask whether the suite is actually helping the team ship, and the answers get a lot vaguer. Those two things aren't the same question, and most QA reporting only answers the first one.

DORA metrics answer the second question. They don't measure your test suite directly. They measure four things about how software moves through your pipeline: how often you deploy, how long a change takes to reach production, how often a deployment causes a failure, and how fast you recover when it does. None of those are QA metrics on their face. But two of them move directly with what your test suite is doing, and the other two move because of what testing does upstream. Understanding which is which is the difference between a QA team that can point to impact and one that can only point to activity.

What DORA actually measures

DORA (DevOps Research and Assessment) is a research program, not a product or a dashboard. It started as academic research into what separates high-performing software teams from everyone else, and it now runs under Google Cloud, publishing findings at dora.dev and in the annual Accelerate State of DevOps report. The four metrics most teams know as "the DORA metrics" are:

Worth knowing: DORA's own framework has moved past exactly four metrics in its more recent research, splitting failure recovery into planned and unplanned categories. The classic four are still what almost every engineering org tracks and what most dashboards report, so that's the frame this post uses. If you want the current, fuller model, the dora.dev guide linked above covers it.

Which of these QA actually moves

Deployment frequency and time to restore are mostly outside QA's direct control. A team can write perfect tests and still deploy once a month if the release process is manual and painful. A team can catch every regression before production and still take an hour to recover from an outage, because recovery is about rollback tooling and monitoring, not test coverage.

Lead time for changes and change failure rate are different. QA has a direct hand in both, and it's worth being precise about the mechanism instead of gesturing at "quality."

Change failure rate is the most obvious one. Every regression a test suite catches before merge is a deployment that doesn't become a failed one. Every regression it misses (or catches too late, after the build already shipped) becomes a data point in that metric. A test suite with real coverage of the paths users actually take pulls this number down. A test suite that's thin, out of date, or too flaky to trust does the opposite, and not because the flaky tests themselves fail in production. It's because teams stop trusting the suite, start skipping steps, and ship changes the suite was never actually given the chance to check.

Lead time is the less obvious one, and it's where QA infrastructure shows up as a direct cost, not just a quality signal. If your test stage takes 40 minutes and half of those runs need a manual re-run because something flaked, that's 40 to 80 minutes sitting inside every single commit's path to production. Multiply that by however many changes ship in a week and the test suite isn't just a quality gate. It's a meaningful chunk of your lead time number, sitting there whether anyone's looking at it or not.

How a flaky or slow suite shows up in the numbers

Change failure rate and lead time feed each other in a way that's easy to miss when you look at them one at a time.

A flaky suite doesn't just waste time re-running. It trains people to stop reading test failures carefully. When a red build might mean a real bug or might mean the same three tests that fail every Tuesday, the rational response is to re-run first and investigate never. Real regressions start hiding inside that noise. They ship, and they show up later as a change failure rate that looks worse than the team expected given how much testing they're doing.

A slow suite pushes teams toward bigger batches. If running the tests and getting through review takes half a day, developers stack up more changes before shipping to make that overhead worth it. Bigger batches mean more surface area per deployment, which means a higher chance any given deployment includes something that breaks. That's the same mechanism growth-stage teams run into when manual QA can't keep pace with how much is shipping: the response isn't less risk, it's bigger, riskier releases spaced further apart.

Here's the part that matters most for the pitch to leadership: DORA's own research consistently finds that speed and stability aren't a tradeoff. Teams that deploy more often and faster also tend to have lower change failure rates and faster recovery, not higher failure rates in exchange for the speed. The dora.dev guide puts it plainly: the metrics are correlated for most teams, and top performers do well across all of them at once. A slow, flaky test suite doesn't buy you extra safety in exchange for slower lead time. It tends to cost you both.

Making the case without vanity metrics

"We wrote 400 new tests this quarter" doesn't tell an engineering leader anything about whether shipping got safer or faster. Neither does coverage percentage, which can climb while the tests that matter most stay untested and the ones that don't matter get covered three times over. These numbers describe QA activity. They don't describe delivery outcomes, and delivery outcomes are what leadership is actually accountable for.

The pitch that works is specific and time-bound. Pull your lead time numbers and isolate how much of that time sits in the test stage: queue time, run time, and the time lost to re-running flaky failures. Pull your change failure rate and check how many of those failures trace back to something a better test would have caught versus something no test could reasonably have found. That second split matters. Not every production incident is a QA gap, and claiming otherwise undercuts the case rather than building it.

Once you have those two numbers, the investment ask writes itself: fix the test infrastructure and lead time drops by X, or fix test coverage on the paths that keep causing incidents and change failure rate drops by Y. That's a case built on delivery outcomes leadership already tracks (or should be), not on how much QA work happened. It also extends the same logic the cost side of test automation already makes: the money isn't really in the build, it's in what a broken or slow suite costs every single week it stays broken.

If you want a starting point for where your team actually stands before you build that case, DORA publishes a short self-assessment at dora.dev/quickcheck. It won't give you your actual numbers, you still need to pull those from your own pipeline, but it's a fast way to see where the obvious gaps are before you go looking for them manually.

None of this replaces a broader testing strategy. DORA metrics tell you whether delivery is getting faster and safer. They don't tell you what to test, how to prioritize coverage, or how manual and automated testing should split the work, which is the ground covered in how modern teams structure a testing strategy. Treat DORA as the scoreboard, not the playbook.

qtrl runs tests in real browsers with structured coverage you can map directly back to the paths that actually cause production incidents, so a change failure rate conversation has something concrete behind it instead of a coverage percentage. Every agent run is logged with what it checked and why, which is what makes it possible to trace a failure back to a coverage gap instead of guessing.

Because runs are governed rather than left to wander, teams can push test execution into the pipeline without adding the flaky, hours-long stage that quietly inflates lead time. That's the part of the DORA story QA teams can actually own: a faster, more trustworthy test stage that helps deployment frequency and change failure rate move together instead of trading off against each other. See how it works.