How to review a spec before the agent builds it

In a spec-driven workflow, the spec gets read by an agent within minutes of being written. Whatever it leaves open, the agent fills in. It doesn't ask what "quickly" means or what should happen when the cart is empty. It picks something reasonable, writes the code, and moves on to the next task.

That makes the spec review the cheapest bug-finding session in the whole cycle. A question that's quick to answer in a requirements doc turns into a code change, a test change and a re-review once it's been built. And QA is usually the person in the room who's best at asking it.

Here's how we'd run that review before anyone, human or agent, builds from the spec: the kinds of problems to look for, a checklist for each criterion, ways to find the cases nobody wrote down, and where an AI reviewer earns its place.

Why the spec review matters more now

In what is spec-driven testing, we described code and tests as two independent readings of the same spec. When they disagree, you've found a bug somewhere. That independence has one weak spot. If the spec itself is wrong or vague, both readings can inherit the same mistake, agree with each other, and go green together.

The spec review is the only point where you can protect both readings at once. Fix a vague criterion there and the code and the tests both get a better input. Miss it, and you're counting on the code and the tests to guess differently, so the disagreement shows up as a failure. Sometimes they do. Often they make the same reasonable guess.

There's some evidence on how much vagueness costs once a model is doing the building. Kiro's write-up of its requirements analysis feature cites a 2025 study (Larbi et al.) where turning clear prompts into ambiguous, incomplete or contradictory ones cut first-try pass rates by 20–40%. The same study found 60–90% of the syntactically valid code produced from those prompts was semantically wrong. It's code-generation research, not a study of your team's specs, but the direction is hard to argue with.

The four kinds of requirement bugs

Kiro's team sorts requirement bugs into four classes, and it's a useful frame for a human review too. Two of them you can spot one criterion at a time. The other two only show up when you read the whole spec together.

Wrong level of detail. The criterion is either a mission statement ("checkout should be fast and easy") or an implementation recipe ("call the pricing service and cache the result in Redis"). Neither is something you can test from the outside. The fix is to rewrite it as an observable outcome: "when a signed-in user with a saved card clicks Pay, the confirmation page appears without asking for card details again."

Ambiguity. One sentence supports two reasonable builds. "Users can cancel an order before it ships" reads fine until you ask whether "ships" means label printed, handed to the carrier, or marked shipped in your system. An agent will choose one. Your test generator might choose another.

Inconsistency. Two criteria can't both be true. One says guests can check out, another says every order needs an account ID. Each looks fine in isolation, which is why these survive reviews that go line by line.

Incompleteness. The spec is silent about some inputs or states. It covers a successful payment and a declined card, but not a payment that times out. Agents are good at producing something for that gap. Whether it's what you wanted is a coin toss.

A checklist to run on every criterion

Go through each acceptance criterion and ask these. If you can't answer one from the text, that's a question for the author, not something to assume.

• Can a user observe it? If the outcome only exists in a database or a log, an end-to-end test can't check it. Note which level should (unit, API, contract) or reword it to the visible effect
• What triggers it, and who? "The user" is often three different roles with different permissions
• What must not change? A rejected coupon should leave the total alone. Criteria rarely say so, and tests rarely check it unless they do
• Are numbers concrete, with units and boundaries? "Up to 10 items" needs to say whether 10 is allowed
• What does the error path look like? Not just that it fails, but what the user sees and whether they can recover
• Is there a hidden "and"? Two outcomes that can fail separately belong in two criteria
• Does a word mean something specific in your domain? "Active", "shipped", "verified" and "deleted" come up a lot. Agree on the meaning and write it into the spec, because the agent doesn't know your conventions

EARS-style wording (WHEN this happens, THE system SHALL do that) makes most of these questions easier to answer, because it forces a trigger and an outcome into every line. Our spec-driven development post has templates.

Find the cases nobody wrote down

The checklist catches problems in what's written. Incompleteness is about what isn't, and it needs a different technique. Three that work well in a review:

Build a small state table. List the states that matter for the feature down one side (signed out, signed in, trial, expired, admin) and the main actions across the top. Every empty cell is a behavior the spec doesn't define. You won't need all of them, but you should decide which ones you're leaving out on purpose.

Ask "and then what?" After each happy path, ask what the user does next. They refresh. They hit back. They open the same page in a second tab and do it again. Double submits and stale tabs are a reliable source of bugs, and specs almost never mention them.

Push the inputs to their edges. Empty, zero, one, the maximum, one over the maximum, very long text, non-ASCII names, different time zones. Our guide to writing test cases from requirements goes deeper on boundary analysis. The difference here is timing: you do it before the build, so the answers go into the spec instead of into bug reports.

Use an AI reviewer for the tedious part

Models are good at spotting vague words, missing error paths and criteria that contradict each other, especially across a long spec where a person's attention runs out. The spec-driven tools have started building this in.

GitHub's Spec Kit has three optional quality gates for this. Its clarify command asks up to five targeted questions about underspecified areas and writes the answers back into the spec. Its checklist command generates what the docs call "unit tests for your requirements", checking the spec for completeness, clarity and consistency rather than checking code. Its analyze command does a read-only consistency pass across the spec, plan and task list.

Kiro goes further with requirements analysis. An LLM first rewrites the criteria into testable statements, then translates them into formal logic so an SMT solver can check whether they can all hold at once and which situations none of them cover. You never see the logic. Each finding comes back as a two-option question: keep the requirement as it is, or apply a specific proposed change. It's a neat split of work, with the model handling language and the solver handling proof.

If you're not using either tool, a plain prompt does a decent version. Give a model the spec and ask it to list every criterion with two plausible readings, every input or state the spec doesn't cover, and every pair of criteria that conflict. Ask for questions, not rewrites. A list of questions keeps the decisions with your team.

Know where AI review stops. It checks the spec against itself. It can't know that your refund rules depend on a regulation, or that support always restores deleted projects on request. Kiro's own team calls this kind of unstated domain knowledge the main gap in requirements analysis. That's the part a person who knows the product has to bring.

How to run the review

Keep it short and keep it before the build. For a single story, a short session with the author, a developer and someone from QA is usually enough. Run the AI pass first so the meeting spends its time on the questions that need judgment, not on spotting the word "quickly".

The output should be two things: an edited spec, and a short list of decisions you made deliberately ("we're not handling payment timeouts in this story, see PROJ-430"). Give each criterion a stable ID while you're there. You'll need it to trace tests back later, and renumbering after the fact breaks every link.

One warning from the BDD years. If QA ends up writing the spec alone, it becomes test documentation that product and engineering don't read, and the shared understanding that made it valuable disappears. QA should ask the questions. The author should own the answers.

What a reviewed spec buys you downstream

A spec that's been through this review produces better code, and it also produces better tests. When the criteria name a trigger, a visible outcome and the error path, a generated test has real expected results to check instead of guesses. When something fails later, the criterion is precise enough to tell you whether the code, the test or the spec moved. That's the subject of our guide to turning acceptance criteria into E2E tests.

qtrl generates browser tests from your requirements and Jira acceptance criteria, then runs them against your real app with the expected results locked to what the spec says. The clearer the criteria going in, the less you'll touch the drafts coming out. Try it on a story you've just reviewed.