Smoke vs sanity vs regression testing: what each one is for

Ask three QA engineers to define sanity testing and you'll get at least two answers. Ask them what runs after a deploy and you'll get one. The labels are fuzzy, but the jobs behind them are pretty clear.

Smoke, sanity, and regression testing all answer the same question ("is this build OK?") at different points in a release, with different budgets for time and depth. Here's what each one is for, where the definitions overlap, and how to decide what runs when.

The short version

• Smoke testing is broad and shallow. Does the build start, can people log in, do the main pages load? Minutes, not hours
• Sanity testing is narrow and a bit deeper. Did the thing we just changed actually get fixed, and did it break the things right next to it?
• Regression testing is broad and deep. Does everything that used to work still work?

Smoke testing: is it worth testing at all?

The name comes from hardware: power the board on and see if smoke comes out. In software, a smoke suite checks that a build is stable enough for anything else to be worth running. If login is broken, running 2,000 regression tests just produces 2,000 failures that all say the same thing.

A good smoke suite is small, fast, and boring. It touches each critical path once and doesn't try to be clever. Typical contents: the app boots, a user can sign in, the main navigation works, the one or two journeys that make money complete end to end, and core API health checks return what they should.

Smoke tests usually run right after a deploy to any environment, and they're the natural candidate for running against production on a schedule, as long as they're read-only. That's the overlap with shift-right testing: the same checks that gate a staging deploy can keep watching production afterward.

Sanity testing: where the definitions fight

Sanity is the messy one. The ISTQB glossary lists "sanity test" as a synonym for smoke test. Plenty of teams use it that way, and they aren't wrong.

But in day-to-day use, most teams mean something narrower: a quick, focused check on a specific change. A hotfix goes out for a broken discount code. Sanity testing is confirming that discount codes work now, that the cart total is right with and without one, and that checkout still completes. You're not re-testing the whole app, just the fix and its immediate neighborhood.

Our take: pick one definition, write it down, and move on. The argument over vocabulary isn't worth a meeting. What matters is that "we ran sanity" means the same thing to everyone who hears it in a release channel.

Regression testing: the expensive one

Regression testing checks that existing behavior still works after a change. It's the broadest and deepest of the three, and the one that grows without limit if nobody tends it. Every bug fix adds a test. Every feature adds a few. Very little ever gets removed.

Full regression on every commit stops being realistic long before most teams admit it. The practical answer is tiers: a fast subset on every pull request, the full set nightly or before a release, and risk-based selection deciding which tests land in which tier. When the suite gets slow enough that people start skipping it, that's the signal to prune it, not to add more runners.

Retesting sometimes gets lumped in here too. It's narrower: re-running the exact test that failed, to confirm the bug is fixed. Regression asks whether the fix broke anything else.

How they fit together in a pipeline

In a typical setup, the three run at different points and gate different things:

1. On every pull request: unit tests plus a regression subset chosen by risk and by what the change touches
2. On every deploy to staging: smoke first. If it fails, stop there
3. Nightly, or before a release: the full regression suite
4. After a hotfix: sanity on the changed area, then smoke once it's deployed
5. In production, on a schedule: the read-only part of smoke

Notice that one test can live in more than one suite. The login check is part of smoke, part of regression, and probably part of every sanity pass that touches auth. The suites are views over a single library of test cases, not separate piles of tests. Teams that keep them as separate copies end up maintaining the same check three times, and the copies drift apart.

Common mistakes

A smoke suite that keeps growing. Somebody adds "just one more important test" every sprint, and a year later smoke takes 40 minutes and nobody waits for it. Give smoke a time budget and defend it.

Treating sanity as optional. Hotfixes are rushed by definition, which is exactly why they cause second incidents. Ten minutes of focused checking around the fix is cheap next to the alternative.

Running regression only before big releases. If the full suite runs once a month, every failure comes with a month of commits to dig through. Nightly runs keep the list of suspects short.

In qtrl, smoke, sanity, and regression are test plans built from the same library of test cases, so the login check lives in one place and shows up wherever it's needed. You can run any plan against any environment on demand or on a schedule, and every result is recorded per run and per environment.

That makes it easy to see which suite caught what, and to keep smoke small while regression grows. Try it free.