Spec Kit vs Kiro vs OpenSpec: how each one checks the work

Spec Kit, Kiro and OpenSpec share a basic shape. You write down what you want, the agent turns it into a plan and a task list, and then it writes the code. Most comparisons of the three spend their time on the first half of that loop: how you write specs, which agents are supported, how much ceremony each one adds.

This one looks at the other end. Once the agent says it's done, how does each tool check that the result matches the spec? And what do you still need to bring yourself? If you work in QA, that's the part of the workflow that ends up on your desk.

Everything below is based on each project's own docs as of mid-October 2026. All three ship changes often, so check the current docs before you pick.

Side by side

That last row isn't a criticism. None of the three set out to be an end-to-end testing tool. It's just the gap you need to plan for, and we'll come back to it.

Spec Kit: gates on the spec, a convergence pass on the code

GitHub's Spec Kit is the most widely adopted of the three, with more than 140,000 GitHub stars, and it works with a long list of coding agents. Its core flow runs from constitution and specify through plan, tasks and implement, and it now ends with a converge step.

Most of Spec Kit's quality tooling points at the spec. The reference docs describe clarify, which asks up to five targeted questions about underspecified areas, and checklist, which generates "unit tests for your requirements" to check that the spec is complete, clear and consistent. Analyze does a read-only consistency check across the spec, plan and tasks. The docs suggest adding these gates for anything with real ambiguity, and implement checks the checklist status before it starts.

On the code side, converge assesses the codebase against the feature's spec, plan and tasks to confirm nothing was missed. It can only add tasks, never remove them, so you loop implement and converge until it reports that the work has converged. That's a useful completeness check. It's also an agent reading code, so it answers "did we build every item?" better than "does each item behave the way the spec says?"

Spec Kit's separate bug-fixing flow is worth a look if you care about verification. It splits work into assess, fix and test, and the test step ends in a verdict of verified, partial or failed. The README puts it plainly: missing verification is not a successful fix. We like that rule a lot.

Kiro: the strongest checks on both sides of the code

Kiro is AWS's agentic IDE, and it puts more formal machinery into verification than the other two.

Before any design or code, its optional requirements analysis (announced in May 2026) checks the requirements file for four classes of bugs: wrong level of detail, ambiguity, inconsistency and incompleteness. An LLM rewrites the criteria into testable EARS statements and translates them into formal logic. An SMT solver then checks whether the rules can all hold together and which situations none of them cover. Findings come back as two-option questions, so you never have to read the logic yourself.

After the code is written, Kiro's correctness feature derives properties from the EARS requirements ("for any authenticated user and any active listing, the user can view that listing") and runs property-based tests against the implementation when you execute tasks. Each property links back to the requirement and task it came from. When one fails, Kiro shrinks the input to a minimal case and asks whether to fix the code, the test or the requirement. The docs are honest about the limit: it provides evidence of correctness, not proof.

Two scope notes. Property tests are optional by default and only available in the IDE. And they test logic at the code level, which is where property-based testing shines, rather than flows a user clicks through.

OpenSpec: light on ceremony, verification as advice

OpenSpec (about 72,000 stars) is built around changes. You propose a change, which creates a folder with a proposal, spec deltas, a design and tasks. The agent applies it, and archiving the change merges the deltas into your main specs. Specs are plain Markdown: a requirement written with SHALL, followed by scenarios with WHEN and THEN steps. It's about as easy to read as a spec format gets.

That archive step is a quiet strength. Because every change has to update the specs to finish, OpenSpec makes it harder for the spec to fall behind the code, which is the problem most teams hit first.

Verification is deliberately soft. The CLI's validate command checks specs and changes for structural problems, like missing sections or a change with no spec deltas. The verify command, in the expanded profile, reviews the implementation for completeness (tasks done, requirements implemented, scenarios covered), correctness (matches spec intent, edge cases handled) and coherence (design decisions reflected in code). The docs say it doesn't block archive. It reports critical issues, warnings and suggestions, and you decide.

What all three leave to you

Each tool has a way to ask "does the code match the spec?" In Spec Kit and OpenSpec, an agent answers that by reading the code. In Kiro, generated properties exercise the code directly. All three are real improvements on asking a coding agent if it's finished.

What none of them do is open the deployed app and walk through the spec's scenarios the way a user would. These are tools for producing software, and checking a running system from the outside is a different job, so it sits outside what they're built for. The gap matters for two reasons:

• Integration bugs live between the pieces. A converge pass or a property test can pass on every module while the checkout page still shows the wrong total
• Independence. When the same family of agents writes the code and then judges it, both can share the same misreading of the spec. A check that only sees the running app and the spec can't inherit what the code got wrong. We made the longer case for this in what is spec-driven testing

There's also a people question behind this. SmartBear's 2026 survey of 1,436 technology professionals found that 44% say a human reviews 60% or less of what their agents produce. Whatever verification the tool does, a lot of output isn't getting a second look from a person.

Where Playwright's planner fits in

Playwright's test agents are the closest thing to a browser-level answer in the same ecosystem. The planner explores your app and writes a Markdown test plan under specs/. The generator turns that plan into Playwright tests, checking selectors and assertions live, and the healer repairs tests that fail.

One thing to watch. A plan written by exploring the app describes what the app does today. If the app is wrong, the plan can be wrong in the same way, and the tests will confirm the bug. The planner accepts a product requirements document as an optional input. Give it your spec every time, and review the plan's expected results against the criteria, not against what you see on screen. The healer is good at fixing a renamed button, and the docs say it skips a test when it decides the functionality itself is broken. That second call is a judgment about intent, so it's worth a person checking it against the spec. Our post on how self-healing tests work covers where that line sits.

So which one should you pick?

If verification is your main concern and your team is happy in its IDE, Kiro does the most. Requirements analysis catches spec bugs before they become code, and property tests are the only built-in check here that actually executes against the implementation.

If your team uses several coding agents, or you want gates you can turn on per feature, Spec Kit is the safe default. Turn on clarify and checklist for anything non-trivial. Those are the cheapest wins in the whole flow.

If you mostly change an existing codebase and want the spec to stay current with little overhead, OpenSpec's change-and-archive model is the easiest to live with. Treat verify's warnings as a review list, since nothing will stop you from archiving past them.

Whichever you choose, add a check that runs the spec's scenarios against the deployed app on every change. The scenarios you already wrote in WHEN and THEN form are most of a browser test. Our guide to turning acceptance criteria into E2E tests walks through it step by step.

qtrl takes the acceptance criteria from your spec or Jira and runs them as tests in a real browser, against the environment you choose. The expected results stay locked to the spec, and every run records screenshots and video. Your SDD tool builds from the spec. qtrl checks the result from the outside. Try it with your next spec.