All posts
EngineeringRAGrequirementstest-generationgroundingtraceabilityAI

Stop Testing Our Guesses. Test Your Requirements.

Most AI test tools infer what your software should do, then test their own assumptions. QuantumVerifi now grounds every test in your actual requirements and docs — cited to the source, with contradictions flagged. Bring a PRD, a spec, a Confluence export; get tests that verify what you actually asked for.

QuantumVerifi Team3 June 20264 min read

Here's the quiet problem with every "AI writes your tests" tool: it's testing its own guesses.

It looks at your code or your site, infers what the software is supposed to do, and generates tests against that inference. When a reviewer asks "does this test the thing we actually specced?", the honest answer is "no — it tests what the model assumed." For a toy project, fine. For a regulated product with a real PRD and acceptance criteria, that gap is the whole ballgame.

So we closed it. QuantumVerifi now grounds test generation in your own documentation — on demand, per analysis. You bring the requirements; we generate tests that verify them, and cite the source for every one.

How it works#

When you start an analysis — URL, API, or repo — you can attach your supporting docs right there: a PRD, an acceptance-criteria sheet, an API guide, a Confluence export. PDF, Markdown, plain text, HTML. Then a retrieval-augmented pipeline kicks in:

  1. Extract & chunk — we pull the text (yes, real PDFs) and split it section-aware, preserving the heading path so a chunk knows it came from Checkout › Payment.
  2. Embed & index — each chunk is embedded and stored in a per-tenant vector store. Your docs never mingle with anyone else's.
  3. Retrieve & ground — as tests are generated, the most relevant excerpts are pulled in and injected into the generation prompt. The model writes tests against your documented behavior, not its assumptions.
  4. Cite — every grounded test carries a traceability comment back to its source:
TypeScript
1// ─── Traceability (grounded in customer docs) ───────────── 2// Requirement: REQ-012 3// Source: PRD.md § Checkout > Payment 4// The checkout must accept credit cards. 5// Payment failures must show a clear error. 6// ──────────────────────────────────────── 7test.describe('REQ-012', () => { ... });

That's test → requirement → document section, baked into the test itself. For anyone who's ever had to prove coverage to an auditor, that line is gold.

It reads your docs better than your team does#

Grounding is table stakes. The interesting part is what the system notices while it's in there.

It flags contradictions. Point it at a doc corpus and it audits for statements that can't both be true. In a real run, we fed it a spec where one section said "the cart allows up to 10 items per order" and another said "users can add a maximum of 20 items." It caught it — quoted both passages, named both sources, and explained the conflict. Your docs disagree with themselves more often than you'd like to admit; now you find out before the tests are written, not after.

It finds the gaps. It cross-references the surfaces of your app against your requirements and surfaces the undocumented ones — the pages and routes nobody wrote a requirement for. "Here be dragons," made explicit.

Every mode, on demand#

This isn't a URL-mode bolt-on. Attach docs and grounding applies across:

  • URL → E2E tests grounded in your acceptance criteria, as validated journeys
  • API → contract tests that assert your documented rate limits, auth rules, validation, and error contracts
  • Repo → unit tests that assert the intended behavior your docs describe, not just the code's current shape

And it's fail-open by design: no docs, no problem — you get the same great inferred tests you always did. The docs only ever make it sharper.

Why this is the right architecture#

Two things make this more than a feature.

Your data is the asset. The validated, structured corpus you build up — requirements, specs, the tests that actually passed — is portable and permanent. Models get deprecated every few months; your grounded knowledge doesn't. (It's the same principle behind our self-improving flywheel — you own the thing that compounds.)

It's sovereign-ready. The whole pipeline — embeddings, vector store, retrieval — runs on infrastructure you can control. For regulated buyers, that means your requirements and source never leave your network. (More on that in our sovereign-install piece.)

The shift#

The old pitch was "AI generates your tests." Ours is sharper:

Bring your requirements. We'll verify them — grounded in your own docs, cited to the source, with the contradictions flagged before a single test runs.

Not our guesses. Yours, proven.

Attach a doc on your next analysis — /analyze, any mode, expand the requirements section. Tests come back grounded and cited.