Multi-agent QA pipelines: specialized agents per test layer

Give one AI agent the whole testing job, explore the app, write the tests, run them, work out why something failed, and for the first couple dozen tests it looks great. Then the app grows past a handful of flows, a run fails at 2am, and you're staring at one long transcript trying to work out whether the agent picked a bad selector, wrote a bad assertion, or found a real bug. All three live in the same trace, produced by the same model call, with no seam between them.

That seam is the thing worth building on purpose. Instead of one agent doing exploration, authoring, execution, and triage in a single loop, split the work across agents that each do one of those jobs and hand off to the next. It's a natural next step from the basic agent loop and from proportional governance, and it solves a problem neither of those posts fully covers: what happens once you need more than one agent working on the same test suite at the same time.

What one agent doing everything actually costs you

A single agent that explores, authors, executes, and triages is running four different jobs through one context window and one confidence threshold. That's the real cost, and it shows up in three places.

First, the context gets crowded. Knowing which button opens the settings panel is a different kind of knowledge than knowing whether a failed assertion means the app broke or the test was wrong. Cram both into one running context and the agent starts making authoring decisions with exploration-quality reasoning, or the reverse.

Second, everything gets the same confidence bar. Exploring a new page and guessing wrong costs you a wasted screenshot. Auto-approving a test that clicks the wrong "Continue" button costs you a false pass in your regression suite. Those two mistakes shouldn't clear the same threshold, but a monolithic agent making both kinds of call in the same loop tends to apply one bar to everything, because it doesn't have a clean boundary telling it when to switch.

Third, and this is the one teams feel first: when something goes wrong, you can't tell where. Anthropic's own writeup on building effective agents makes a related point from the other direction: add complexity only when it demonstrably improves outcomes, and favor patterns that keep each step's reasoning visible. A single loop that explores, writes, runs, and judges its own work doesn't give you that visibility. You get a transcript, not an audit trail.

Four agents, four narrow jobs

The split that maps cleanly onto a QA pipeline has four stages: exploration, authoring, execution, and triage. Each one takes a narrower slice of the problem, which is exactly what makes it auditable.

The exploration agent behaves a lot like the "observe" level from proportional governance: read access, no state changes, a full log of where it went. The authoring agent sits closer to "advise" and "act with approval," since a generated test shouldn't count as real coverage until something, human or agent, has looked at what it actually asserts. Execution is where an agent can run fairly autonomously, because the blast radius of running a test is small and well understood. Triage is arguably the highest-stakes stage of the four, because its output decides whether a human's attention gets spent on something real.

That last point deserves its own mention. A lot of what looks like agent unreliability is actually the agent choosing a different valid path through the app, not a bug in your product. A dedicated triage agent that has seen the difference between path-variance and an actual regression is worth more than an execution agent trying to make that same call in the middle of a run it's still in the process of finishing.

Failure isolation is the underrated benefit

Splitting the pipeline buys you something beyond auditability: when one stage breaks, the others keep their work. If the execution agent hits a flaky environment and a run times out, the tests the authoring agent wrote are still valid. Nobody has to re-explore the app or rewrite the test. Compare that to a monolithic agent that hit the same timeout mid-loop: depending on how its context got corrupted, you might not trust anything downstream of that point, including work that had nothing to do with the failure.

The same logic runs the other way. If the triage agent starts misclassifying failures, say it's been too aggressive about calling things flake, that's a contained, fixable problem. You adjust triage. You don't have to distrust the tests themselves, because authoring is a separate agent with its own track record.

Where the coordination problems actually live

None of this is free. The moment you have four agents instead of one, you've traded a context problem for a coordination problem, and it shows up in three specific places.

Shared state. Every agent needs the same picture of the app: what flows exist, which tests already cover them, what the last ten runs looked like, which heals or fixes are pending review. If exploration keeps that picture in its own head and authoring keeps a different one, you get drift, an authoring agent writing a test for a flow exploration already flagged as removed, or a triage agent that doesn't know a selector was already updated last week. The fix is a shared record all four agents read from and write to, not four private memories that occasionally get synced.

Redundant work. Anthropic's writeup on the multi-agent research system they built is candid about this: early versions had separate subagents independently investigating the same topic because task descriptions were vague enough to overlap. The same trap is easy to fall into here. An exploration agent and a triage agent can both end up re-mapping the same checkout flow if neither one checks whether the other already did. Clear task boundaries and a shared state store are the fix, not adding a fifth agent to referee the first four.

Who has final say. This is the one teams usually skip until it bites them. Authoring writes a test asserting one thing. Execution runs it and gets a different result than authoring expected. Triage has to decide who was right, without a built-in way to break the tie. LangChain's documentation on multi-agent architectures frames this as a handoff problem: agents transferring control need an explicit protocol for what gets passed along and who owns the decision at each point, not an implicit assumption that whoever spoke last is right. In a QA pipeline, the honest answer is usually that none of the agents gets final say on anything above a confidence threshold. That decision routes to a human, at the review gate between authoring and execution or the one after triage, exactly the "act with approval" rung from proportional governance.

What a clean handoff actually looks like

The handoff between agents is where most of the above either gets solved or doesn't. A handoff that's just a sentence in a chat log ("found a checkout flow, might be worth testing") gives the next agent nothing to check its work against. A handoff with structure does:

{
  "handoff": "exploration_to_authoring",
  "flow_id": "checkout_with_discount_code",
  "discovered_by": "exploration-agent-3",
  "confidence": 0.88,
  "steps_observed": [
    "add item to cart",
    "apply discount code",
    "proceed to payment",
    "confirm order"
  ],
  "coverage_status": "untested",
  "related_tests": [],
  "notes": "discount field validates client-side before the submit call fires"
}

Now the authoring agent knows exactly what it's working from, the triage agent can trace a later failure back to this exact discovery, and a human reviewing the pipeline can see why a test exists without reconstructing the reasoning from scratch. It's the same principle as a good heal log: record the intent, not just the outcome, and every downstream agent (and every human) can verify the chain instead of trusting it blind.

Start with two agents, not four

You don't need to stand up all four roles on day one. The highest-value split for most teams is separating execution from triage first, since that's where agentic testing tools already draw the line: an agent that runs tests and reports raw results, and a second pass that decides what those results mean. Add authoring as its own agent once you trust triage's judgment. Add exploration last, once you have enough tested flows that finding new ones is the bottleneck rather than fixing the ones you have.

qtrl's pipeline is built around this separation rather than one agent trying to do it all. Exploration and authoring stay observable and reviewable, execution runs against real browsers, and triage results route through the same review-and-approve workflow that governs every other autonomy level on the platform, so a low-confidence handoff lands in front of a person instead of quietly becoming truth.

Every stage writes to the same shared record of what's been tested and why, so an authoring decision from three weeks ago is still traceable when triage needs it today. See how it works.