Implementation
3 of 10 planned modules are real and wired today.
Here's exactly which, and exactly what's next. Nothing below is rounded up.
Requirement Risk Assessor
Scores how risky a requirement is before a single test is written.
Real LLM call, real Pydantic-validated I/O, real knowledge-graph persistence. Before building its prompt it pulls the top-3 similar past requirements for the tenant (TF-IDF cosine similarity ≥ 0.2, stdlib-only — no embedding model) and injects their real stored risk level and real human review decision as few-shot context. This is retrieval-augmented prompting, not fine-tuning — no model weights change anywhere in this system.
Test Suite Generator
Turns a requirement and its risk assessment into candidate test cases.
Reads the risk assessment back from the knowledge graph — not resent by the client — then makes a real LLM call to generate test cases (title, steps, expected result, priority), persisted with status “draft.” Does not yet use retrieval memory itself; only Module 1's prompt does.
Review Gate
The human checkpoint before a generated test counts as real.
A real human approve/reject decision, persisted to the same TestCase node, feeding straight back into Module 1's memory loop for the next similar requirement. No LLM call happens on this route at all — it was exercised completely unmocked, over real HTTP.
Execution Orchestrator
Runs generated tests via Playwright, orchestrated through Temporal.
Still in the repository, and still broken — on purpose kept, not deleted, so the gap is visible instead of hidden. It hardcodes a passing result for every test regardless of outcome, and has an unresolved bug (a NameError from a missing import). Temporal is fully provisioned in the infrastructure config; the workflow logic itself still needs to be written for real. Nothing downstream of execution should be trusted until this module is real — it's the top item on the roadmap for exactly that reason.
Defect Triage
Classifies defects using knowledge-graph context.
Real Pydantic schemas and real async logic, structurally sound — not wired to any API route yet. The fastest legitimate next win: connect it the same way modules 1–3 were connected.
Release Readiness Advisor
Produces a go/no-go recommendation, never an autonomous release trigger.
Same tier as Module 5 — real schemas and logic sitting behind no route. Its report generator file is empty.
Knowledge Graph Query Surface
Natural-language queries over the knowledge graph.
Explicitly commented as a stub in its own source. It calls a hardcoded placeholder root id that nothing in the graph ever actually creates, so even the real traversal logic underneath it has nothing to find.
Production Feedback Loop
Feeds production telemetry back into risk scoring.
Has some real shape, but no real telemetry ingestion exists anywhere yet to feed it. Structurally premature before Module 4 is real — see the roadmap.
QA Command Center
Serves the dashboard's backend surface.
Zero routes — a bare router with nothing registered. The command-center backend file is empty. (The real Next.js frontend you're using right now talks directly to services/api, not to this module.)
Integration & Extensibility Layer
Wires external tools — Jira, GitHub, Slack, Playwright — into the pipeline.
The Jira adapter is genuinely production-quality: real Basic Auth, real ticket translation, ~370 lines of real logic — the only connector that is. The GitHub, Slack, and Playwright adapters are still in the repository too, exactly as they are: one-line stub classes with no real logic behind them. None of the four are called from any live route yet. The Connector protocol itself, and its genuinely thorough test suite, is the real foundation here — the stubs exist to show the shape the next three adapters need to fill, not to pretend they're already filled.
Engineering decisions
Why it's built this way
Error-honesty over silent fallbacks
Every LLM call that fails raises a real error, all the way up the stack. No route silently returns an empty success or a fabricated default when the model call breaks — which is a specific, common failure mode in vibe-coded AI scaffolds that this codebase's own engineering history found and fixed.
Retrieval before fine-tuning
Module 1's memory loop is retrieval-augmented prompting — real stored requirements and real human decisions injected as context — not model fine-tuning. No model weights change anywhere in this system. That's a deliberate sequencing choice, not a limitation; see /roadmap for why it's the correct first step.
Interface-first persistence
The knowledge-graph client is defined as an interface with three implementations — an in-memory stub, a SQLite-backed store, and a Neo4j production client — selected entirely through environment configuration. No call site anywhere in the codebase knows or cares which one is active.
The human checkpoint is structural, not incidental
Module 3's review gate isn't a UI nicety bolted on afterward — it's the point in the pipeline where a generated test becomes real, and the decision it produces is what feeds Module 1's memory loop. The architecture makes the human sign-off load-bearing.
Current limitations
Stated plainly
- Module 4 (Execution Orchestrator) is in the repository and it's broken — left that way on purpose rather than deleted or hidden. It hardcodes a passing result for every test and has an unresolved bug (a missing import). Nothing downstream of execution should be trusted until it's rebuilt for real, which is exactly why it's first on the roadmap.
- There is no authentication. The auth check is a one-line placeholder; nothing is enforced.
- Retrieval memory covers Module 1's prompt only. Test generation and defect triage don't use retrieval yet.
- The Neo4j-backed KG client has real driver code behind the correct interface, but has never been exercised against a live Neo4j instance in this environment — built, not proven.
- There is currently no automated way to check whether a change to a reasoning prompt made output better or worse — only that it changed. Evaluation is unbuilt, not just unwired.
- The Jira connector is genuinely production-quality but isn't wired into the running flow yet. The GitHub, Slack, and Playwright connectors are also in the repository, and they're exactly what they look like: one-line stub classes with no real logic behind them yet.
This is exactly the order it should get fixed in.
See the roadmap