The Training Flywheel: How QuantumVerifi Learns from Every Analysis

QuantumVerifi uses large language models (GPT-4.1, Claude) for everything: generating tests, understanding code, writing documentation, analysing security. These are general-purpose models — they're good at many things but they don't know your specific codebase, your test patterns, or your team's conventions.

The Training Engine changes that. It turns every analysis into a learning opportunity, fine-tuning a custom model that gets better at your specific workload over time.

How the Flywheel Works#

The flywheel is a four-step cycle:

1. Analyse

You run a normal analysis — URL, repo, API, or performance mode. QuantumVerifi generates tests, evaluates security, checks accessibility, and produces a grade.

2. Harvest

After the analysis completes, the system automatically extracts training episodes — examples of successful test generation, accurate security findings, and correct code understanding. Each episode is a structured input/output pair:

• Input: The context the model received (code files, crawl data, instructions)
• Output: The model's response that produced a good result (passing tests, valid security findings)

Only successful outcomes become episodes. Failed tests, hallucinated findings, and low-confidence results are filtered out. Quality over quantity.

3. Train

When enough episodes accumulate, you can train a QLoRA adapter — a lightweight fine-tuning layer that sits on top of a smaller base model (typically 7B–14B parameters). Training runs on GPU infrastructure (A100) and takes 15–45 minutes depending on dataset size.

The adapter is small (typically 50–200MB) but encodes your specific patterns. It doesn't replace the base model — it adjusts the model's behaviour to match what works for your codebase.

4. Route

Once trained, you activate the adapter. Future analyses route through your fine-tuned model instead of the general-purpose model. The adapter handles operations where it has training data (test generation, code understanding), while the general-purpose model handles everything else.

Then the cycle repeats — the fine-tuned model produces better results, which become better training episodes, which produce a better adapter.

What Gets Fine-Tuned#

Not every LLM call goes through the adapter. Different operations have different fine-tuning value:

Why Not Just Use GPT-4.1?#

General-purpose models are powerful but they have three limitations for repeated analysis:

Cost: GPT-4.1 costs ~$0.08 per analysis. A fine-tuned 14B model running on your infrastructure costs ~$0.01. At scale, this is an 8x reduction.

Speed: Smaller fine-tuned models generate tokens faster. A 14B model produces 80-120 tokens/second vs ~30-50 for GPT-4.1 via API.

Relevance: After 50+ analyses on your codebase, the adapter knows your test framework, your assertion style, your common imports, and your error patterns. It stops generating require('supertest') when your project uses axios.

Adapter Architecture#

The Training Engine uses QLoRA (Quantized Low-Rank Adaptation) — a parameter-efficient fine-tuning method that:

1. Freezes the base model weights (no modification to the original model)
2. Adds small trainable matrices (rank 16–64) to attention layers
3. Quantizes the base model to 4-bit precision during training (reduces GPU memory from 28GB to ~8GB)

The result is a small adapter file that can be loaded on top of any compatible base model. You can train multiple adapters for different operations and swap them at runtime.

Base Models

Training Dashboard#

The Training Engine dashboard (Settings > Training) shows:

• Active adapters — Which operations are routed through fine-tuned models
• Training jobs — History of training runs with status, duration, and metrics
• Episodes harvested — How many training examples have been collected
• Model routing — Which model handles each operation

From the dashboard, you can:

• Start a new training job
• Activate or deactivate adapters
• Test inference with a trained adapter before activating it
• Monitor the flywheel status

Episode Quality#

Not all episodes are created equal. The harvesting pipeline applies quality filters:

• Passing tests only — Only test generation episodes where the tests actually pass validation
• Confidence threshold — Security findings must have high confidence to become episodes
• Deduplication — Similar episodes are deduplicated to prevent overfitting
• Diversity sampling — Ensures the training set covers multiple languages, frameworks, and patterns

A typical dataset after 100 analyses contains 500–2,000 high-quality episodes across test generation, code understanding, and self-healing operations.

Privacy and Isolation#

Training data is fully isolated per tenant:

• Episodes are stored in your tenant's namespace — no cross-tenant data access
• Adapters are scoped to your account — only your analyses use your adapter
• Training runs on dedicated GPU instances — no shared compute
• You can delete all training data at any time

The base model weights are shared (they're public models), but your adapter — the part that contains your learned patterns — is private to your account.

When to Train#

Training makes sense when:

1. You have 50+ analyses — Enough episodes to learn meaningful patterns
2. You analyse similar codebases — A team working on React + TypeScript benefits more than someone analysing random repos
3. You care about cost — At 100+ analyses/month, the cost savings justify the GPU training cost
4. You want faster generation — Smaller models produce results faster

Training doesn't help much when:

• You're running occasional one-off analyses
• Every analysis is on a completely different tech stack
• You're on the free plan (training is available on Scale and Enterprise plans)

Availability#

The Training Engine is available on Scale and Enterprise plans. It requires GPU infrastructure for training and inference, which runs on dedicated AWS ECS instances.

• Training: A100 GPU (QLoRA fine-tuning, 15–45 minutes per run)
• Inference: A100 GPU (vLLM serving with LoRA adapter hot-loading)
• Episode harvesting: Automatic, no GPU required

Contact us at [email protected] to enable the Training Engine for your account.