Lalitkumar Bhamare · Beyond Automation · Tokyo Test Fest 2026
Tokyo Test Fest 2026 · Conference talk
Beyond Automation
How Agents Amplify (Not Replace) Testers with Agentic QCSD
Lalitkumar BhamareQE Thought Leader, EMEA · Accenture QESCreator of QCSD and Agentic QCSD
13 November 2026 · 15:25 to 15:55 Osaki Bright Core Hall, Tokyo
Photo: Takashi Miyazaki / Unsplash
About me
Before we begin
Lalitkumar Bhamare
Sr. Manager, Accenture Song · Germany EMEA Lead, Accenture QES Thought Leadership
More of my worktalesoftesting.com
Most organisations chase quality through process. I have spent my career proving it comes from consciousness: a deliberate, human-centred awareness woven into every stage of software delivery.
CEO, Chief Editor and Co-founder: Tea-time with Testers
Former VP Education and Board of Directors: Association for Software Testing, USA
Co-author: Global State of Testing Report
Contributing author: Taking Testing Seriously (Bach and Bolton)
Creator: QCSD and Agentic QCSD
Creator: QX (Quality Experience) framework
International keynote speaker: testing and quality coach
Advisory board member: German tech start-ups
EuroSTAR 2022Best Paper Winnerfor Quality Conscious Software Delivery
PractiTest 2025Emerging QA Leader of the Year
01Section 01
The faster horse syndrome
Every generation of tools made testing faster. The cost of poor software quality kept rising.
Photo: Angelica Hasbon / Unsplash
The faster horse syndrome
Same rider, same road, a faster horse
Every generation of tools changed how fast we test. None of them changed what we test for, or the mindset behind it.
01 · What speed without quality costs
$2.41T
lost to poor software quality in the US in 2022, before counting $1.52T of technical debt
$5.4Bestimated Fortune 500 losses from a single faulty software update in July 2024
CISQ 2022 · Parametrix 2024
02 · What AI is accelerating
−7.2%
delivery stability where teams adopted AI
8×more duplicated code blocks in 2024, and 40% less refactoring
DORA 2024 · GitClear 2025
03 · Where testing spends its AI
70%
of AI use in testing goes to writing test cases
4.5%of teams are measured on business value. 56% are measured on coverage.
State of Testing 2026
We keep buying speed, and we keep paying for it in quality.
Photo: Catrin Ellis / Unsplash
02Section 02
Volume, Value, and Testing
What volume-driven testing costs, and what testing that creates value looks like.
Photo: Sasun Bughdaryan / Unsplash
How a volume-driven testing culture feeds the loop
Section 02 · Volume, Value, and Testing
Volume-driven testing is not neutral. It feeds itself, and every turn leaves quality a little worse.
Progress is measured by volumeTest cases, coverage, scripts
Testing starts after the buildNo time for questions before code
Defects are found lateWhen they cost the most to fix
Rework and firefightingThe sprint goes to fixing, not thinking
Upfront thinking is cut firstRisk analysis waits for a free afternoon
More pressure to show progressSo we count more tests
Degraded qualityEach turn adds more tests and less information
A reinforcing loop
Nothing in this loop asks whether the testing produced information anyone could act on. Every turn adds volume and removes time for thinking.
AI on the volume side
Point AI at writing more test cases, and the same loop simply turns faster.
What breaks the loop
Information that arrives while a decision is still cheap to change.
Upfront rigour before the build.
Not more tests after it.
What testing that creates value looks like
Section 02 · Volume, Value, and Testing
Most testing is measured on the tip. The value sits below the waterline.
What most teams countTest casesAutomated scriptsAcceptance criteria passed
What actually creates value
Shape what gets built
Risk assessment with the business
Quality criteria agreed for each release
Questioning requirements for gaps and contradictions
Testability asks before the design is fixed
Experience and accessibility reviews of designs
Widen what we look at
Product factors assessment for broad coverage
Deep, charter-based exploration
Performance, security and resilience risks
Test data that reflects real users
Experiments and what-if questions
Make the system easier to trust
Smart automation where it pays back
Logs, hooks and monitors asked for early
Contract and integration risks across teams
Risk-based quality gates with evidence
Learn and feed back
Learning from telemetry and error logs
Root cause analysis that changes how we work
Quality measures the business already tracks
Coaching developers and product owners on testing
Reporting risk in business language
Testing creates value when its information changes a decision while that decision is still cheap to change.
How the 70/30 problem makes value creation harder
Section 02 · Volume, Value, and Testing
Clerical and process burden takes the time that cognitive work needs. Testers feel it first, and the whole team pays for it.
70% · clerical and process work
30% · cognitive work
What fills the 70%
What gets squeezed into the 30%
Testers
Writing and maintaining test cases and scripts, status reports, triaging failing runs
Risk judgment, exploration, testability, test strategy
Developers
Rework on late defects, ticket and pipeline admin, handoffs
Designing for testability, thinking through edge cases, pairing on risk
Product owners
Backlog administration, status meetings, re-explaining requirements
Clarifying intent, agreeing quality criteria, deciding what not to build
Delivery leads
Reporting, coordination, chasing dependencies
Removing systemic risks, improving how the team works
The 70/30 problem
Clerical and process work crowds out the thinking, for every role. Everything below the waterline has to fit into the 30%.
03Section 03
Quality Conscious Software Delivery
What QCSD is, how it runs and what it changed.
Photo: Jaipreet Singh / Unsplash
Introducing QCSD
Section 03 · Quality Conscious Software Delivery
Quality Conscious Software Delivery
Delivery of quality products by quality-conscious people using quality-empowering processes
using value-driven testing to facilitate it all.
Three notions of quality · QualiTri
Quality is never the property of the product alone
Products are built by people
People work in projects and follow processes
Project, people and product quality affect each other
How it runs · the 4E cycle
Enable, Engage, Execute, Evaluate
The five phases
Section 03 · Quality Conscious Software Delivery
Every task, by every role, at every phase of the SDLC, done with more consciousness for quality.
Enable and engageExecuteEvaluate
Supporting activities
Testability improvement sessionsBi-weekly tester exchangePeriodic assessment of automation suitesLearning from production error logs
QCSD made upfront rigour work
Section 03 · Quality Conscious Software Delivery
Upfront rigour stopped being an ideal and became how the team worked. Lead time per release shows it.
In developmentBlockedIn QAMedian, before
18–49
Days lead time per release, before
11–13
Days lead time per release, after
Recognition and adoption
EuroSTAR 2022 Best Paper Award
QCSD was awarded the EuroSTAR Software Testing Best Paper award in 2022. Read the paper
Published industry case study
iFM engineering published its own case study of adopting QCSD.
Multiple Accenture clients
QCSD has been applied across several Accenture client deliveries.
The approach worked.Keeping it going, release after release, was the hard part.
04Section 04
The hidden challenges around upfront rigour
The underlying factors that kept steady upfront rigour out of reach, even when it clearly worked.
Photo: Luis Villasmil / Unsplash
Why upfront rigour stayed inconsistent
Section 04 · The hidden challenges around upfront rigour
Upfront rigour worked when it happened. These are the reasons it did not happen every time.
Clerical load
About 70% of a team’s capacity goes to clerical work. Upfront rigour gets what is left.
70%
30%
Cognitive overload
Seven product factors, every story, every sprint, held in people’s heads.
Fatigue
Depth depends on who is in the room and how tired they are. Sessions get shallower over time.
Fading motivation
Preparation is invisible effort. The clerical work is what gets counted and rewarded.
Delivery pressure
When a deadline moves, the thinking ahead is cut first.
The real question for AI in testing
Not a lack of ability or effort. A lack of capacity. Can AI give testers that capacity back, and make them more effective with it?
05Section 05
Agentic QCSD: Amplify, not replace
Upfront rigour carried by AI agents, the judgment designed into them, and the people who still make the call.
Photo: Mick Haupt / Unsplash
Agentic QCSD: QCSD power-charged with AI agents
Section 05 · Agentic QCSD: Amplify, not replace
Same goal, same people, same processes. Upfront rigour made possible, affordable and more intelligent.
The sessions stay. A swarm of AI agents does the preparation for every story, and people keep the decision.
QCSD · since 2019The session, run by people
Agentic QCSD · todayCarried by an Agentic QE swarm or AI agent
Ideation
Requirements before ideation
testability scoringSkill
QX session
qe-qx-partnerAI agent
Impact mapping
human session · qe-qx-partner for evidenceHuman session
Refinement
Story analysis with SFDIPOT
qcsd-refinement-swarmSwarm
Product coverage session
qe-product-factors-assessorAI agent
Development
Developer testing for acceptance criteria
qe-test-architectAI agent
Code quality checks
qcsd-development-swarmSwarm
CI/CD
Automated quality gates
qcsd-cicd-swarm · qe-queen-coordinatorSwarm
Production
Production telemetry and error logs
qcsd-production-swarmSwarm
The 70/30 ratio, for the whole team
Cost, speed and quality together. The rigour that used to cost people-days now runs at the speed of the sprint, with every human gate still in place.
People direct. Swarms of AI agents do the work. Agentic QE, the intelligence engine, learns from every sprint.
Human direction · validation is the control
5 human gates · GO, CONDITIONAL or NO-GO verdicts for every artefact · QualiTri dashboards for people, project and product
AI agent swarm layer · 8+2 swarms · 74+ AI agents · 75+ skills
Swarm orchestration · Queen coordinator
DesignDesign review swarm · QX
DiscoveryIdeation swarm
Test designRefinement swarm
Refine and buildDevelopment swarm
CI/CDCI/CD swarm
ProductionTelemetry · L3 loop
Cross-phase exchangeevery phase feeds every other · 13 DDD bounded contexts · PACTS principles
Write learnings
Read context · 2 ms
Intelligence engine · Agentic QE · every sprint smarter than the last
Knowledge graph memorySquad context and cross-phase memory
Fast indexAbout 2 ms reads; unchanged work is skipped
Learning pipelinePatterns reused across sprints and projects
Offline consolidationRun history distilled into reusable patterns
Self-tuning retrievalRetrieval and routing tune themselves
Self-review loopAI agents critique AI agents
Integration layer · 11+ platforms and systems · LLM-agnostic
GitLab · Jira · Confluence · SAP RFC · Kafka · Salesforce APIs · payment gateways · CI/CD · any approved model channel, with the organisation’s own keys
Cross-cutting rails
Cost-aware model routing5 routing tiers and a local booster · 70–85% lower token cost
Self-healing34+ known failure signatures · autonomous recovery
Quality gatesAnti-sycophancy scoring · hollow tests rejected · edge cases from learned history
Standards and ADRsCentral ADRs and readiness rules, a paved road for every squad
Applied at every layer, on every run, not bolted on afterwards.
Production learning re-primes planning, and the delivery lifecycle is only the first application: the same swarms, skills and governance assemble into many other solutions.
Amplify, not replace
Section 05 · Agentic QCSD: Amplify, not replace
The answer to that question: testers get capacity back, and AI agents make their judgment sharper.
The testerbrings the judgment
Critical thinking
Risk judgment
Testability assessment
Deep exploration
Business and domain context
Ethical judgment
Creative what-if questions
The final call
Amplifyreplace nothing
AI agentsbring the preparation
Preparation for every story
Breadth across all seven product factors
Evidence behind every verdict
Memory that carries across sprints
The same rigour at sprint six as at sprint one
Speed that frees people to think
A{ı}bility
Designed to Reason. Built to Resonate.
Demos run on A{i}bility · Accenture’s software delivery intelligence platform, built on Agentic QCSD
QX AI agent review of a live customer journey
Demo 1 · QX AI agent · Design review · for the designer
A QX AI agent reviews a live customer journey. The designer decides. R replay
Discovery document review before Technical Discovery
Demo 2 · Business discovery · for the TPO
Eight AI agents review a discovery document. The TPO decides. R replay
Test strategy from requirements before development
Demo 3 · Test strategy · for testers and developers
One AI agent drafts the strategy and states its own limits. Testers decide. R replay
Many more ways to amplify testers, beyond automation
Section 05 · Agentic QCSD: Amplify, not replace
Agentic QCSD with the Agentic QE framework: AI agents for every kind of testing work, not just writing tests.
Test Generation7
Test ArchitectTDD SpecialistTDD RedTDD GreenTDD RefactorProperty TesterTest Idea Rewriter
Test Execution4
Parallel ExecutorFlaky Test HunterRetry HandlerIntegration Tester
Queen-led hierarchical coordination · work stealing for 3 to 5 times throughput · PACTS principles throughout · including the 14-AI-agent n8n workflow-testing swarm
The takeaway
Testing has always been an information business
AI agents can bring more of that information to the surface, and help testers communicate it so it changes decisions. If we use them right.
01
FromAI that writes more tests
ToAI agents that surface what no one had time to find
Risks, gaps and contradictions across all seven product factors, for every story, before the build.
02
FromReports nobody acts on
ToEvidence that changes decisions while they are cheap
Every verdict carries its reasons, told in the language of the business.
03
FromTrusting what the AI produced
ToTesters who question it and make the call
Judgment, context and ethics stay with people. AI agents make them sharper.
Used right, AI makes testers more significant than ever.
Amplify the thinking.
Not the volume.
Dedication
For Keizo Tatsumi
A lifelong tester and my friend of ten years. He is one of the reasons I am in Tokyo today.
1976Joined Fujitsu’s software inspection department, later led its Software QA Division
1987Co-authored “Conceptual support for test case design” at COMPSAC, early work on combinatorial testing
TodayJapan’s software testing historian, NPO ASTER member, and lead of the Japanese translation of the State of Testing report
Record AI adoption, the same old metrics, and where a tester’s time goes.
State of Testing 2026 · The 70% problem
Photo: Homa Appliances / Unsplash
Record AI adoption, same old metrics
Chapter 00 · The diagnosis
State of Testing Report 2026, by PractiTest and Tea-time with Testers.
76.8%
of teams use AI in testing
70%
of that AI usage goes to test case creation
4.5%
are measured on NPS or business value
The industry upgraded its tools but not its thinking. We are still in the test factory, measured on pass and fail, rewarded for volume rather than value.
Source: State of Testing 2026. 56% of teams are measured on test coverage.
The 70% problem · where a tester’s time goes
70% clerical
30% expert
Clerical burden
Documentation and artefact management
Repetitive analysis and reporting
Automation maintenance
Status updates and test reporting
Gathering insights from failing tests
Expert work
Critical thinking and risk judgment
Testability assessment
Deep exploration
Context analysis
Strategic testing
The 70/30 split is Lalit’s own observation across teams, not a published figure.
03Section 03
Quality Conscious Software Delivery
Where QCSD came from, what it is, what it changed, and what it costs to run by hand.
Origin · What QCSD is · Five phases · Outcome · Cost of rigour
Photo: Jaipreet Singh / Unsplash
Where QCSD came from
Chapter 01 · Born out of necessity
A real delivery team, a real product, and a backlog that kept growing faster than the team could ship.
A passionate team that was determined to make things better
What the team was living with
Why projects stay stuck · Jerry Weinberg's feedback effect
Value of quality unclearNobody can say what quality is worth
Engineering actions not aligned with business goalsWork is busy, not valuable
No motivation to achieve qualityQuality loses every trade-off
No know-how to achieve itNobody learns how to build it in
No motivation to measure itSo its value stays unclear
System collapseEach turn of the loop makes the next one worse
The cost of upfront rigour
Chapter 05 · Why not every team can do it
QCSD works because it is always cheaper to do things right the first time. Doing them right the first time means putting skilled people in a room before a line of code exists. Add the sessions one feature needs.
People in the room for one feature
0sessions
0people seats
Everyepic and every sprint
Senior people's time, repeated for every feature
The first thing cut when a deadline moves
Depends on a coach in the room to run it well
The cost, speed and quality trade-off
Chapter 05 · Why not every team can do it
The trade-off delivery teams have lived with for 30 years
Cost, speed, quality: pick two
Upfront rigour has always pulled quality up at the price of cost or speed.
Select two corners.
04Section 04
Agentic QCSD and Designed Agency
Upfront rigour carried by AI agents, and the judgment designed into them.
Agentic QCSD · From session to swarm · The 70/30 split · Symptoms answered · Designed Agency
Photo: Mick Haupt / Unsplash
How Agentic QCSD changes the 70/30 split
Chapter 06 · Agentic QCSD
The clerical 70% moves to AI agents. The expert 30% becomes the tester’s work at every gate.
70% clerical · now prepared by AI agents
30% expert · the tester’s work
Prepared by AI agents for every story
Documentation and artefacts→Phase reports drafted for every story
Repetitive analysis→SFDIPOT and risk scoring on every story
Automation maintenance→Tests generated from the test ideas
Status updates and reporting→GO, CONDITIONAL or NO-GO verdicts with evidence
Insights from failing tests→Root cause analysis and defect prediction
Where testers now spend their time
Risk judgmentDecide which risks block the story
Critical thinkingChallenge the verdict and its evidence
TestabilityAsk for the change that makes it testable
Deep explorationExplore where the AI agents did not look
ContextBring the business and domain knowledge
Nothing is decided for you. AI agents prepare the evidence; every gate still has a person who makes the call.
Symptoms and the methods that answer them
Chapter 07 · Built by design
Built by design to break the loop. Every symptom the team lived with has a named QCSD method, tried and tested in delivery since 2019, and each method is now carried by an Agentic QE swarm or AI agent.
Select a symptom.
Agentic QCSD is built on Designed Agency
Chapter 08 · Designed Agency
The industry adopted the word agentic faster than it examined what the word means. Designed Agency compels us to think about and define what it means in our context.
Why most AI agents are called agentic
An LLM decides the next step
It calls tools
It runs in a loop
What that leaves out
Encoding judgment without architecture is just a prompt
Architecture without encoded judgment is just plumbing
Without honest approximation there is no way of knowing whether what was built is any good
Designed Agency
Judgment deliberately designed in by practitioners who know the work deeply enough to encode it, architect it and measure it honestly.
Encoded the threads, the material the cloth is made of
Structured the loom, the instrument that acts on the material
Approximated the contact between threads that gives the cloth its texture
The cloth needs all three together. Agentic QCSD and the Agentic QE framework are built on all three.
The three dimensions of Designed Agency
Chapter 08 · Designed Agency
The three dimensions and how they show up in the AI agents. Select a dimension.
Static analysis and local embeddings inside your workspace. No provider key, no tokens. In the container, prepare-workshop.sh has already run these three commands.
05Section 0513:35–13:55
The engine
Swarm architecture, feedback loops, and the Agentic QE framework from its creator.
Architecture · The engine · AQE intro by Dragan
Photo: James Wainscoat / Unsplash
Swarm architecture and feedback loops
Architecture · 13:35–13:40
Five swarms, one per phase, each handing its findings forward. Four feedback loops carry what each phase learned back to the phase that can act on it.
Ideation9 AI agents · 4 skills
›
Refinement10 AI agents · 3 skills
›
Development10 AI agents · 4 skills
›
CI/CD10 AI agents · 4 skills
›
Production12 AI agents · 5 skills
The engine that powers it all
Architecture
The Agentic QE framework, from its creator
AQE intro · Dragan Spiridonov · video · 14 min
Dragan Spiridonov · Head of Agentic Quality Engineering, Cognitum One · creator of the Agentic QE framework · github.com/proffesor-for-testing/agentic-qe
06Section 0613:55–15:35
Hands-on lab
Five exercises on one checkout app, from the epic to the release call.
Ideation · Refinement · Development · CI/CD · Self-learning
Photo: Rahul Mishra / Unsplash
Ideation: surface risk before any code
Exercise 1 · Ideation · 13:55–14:20
About 20 minutes · qcsd-ideation-swarm · installed as a skill, not a command file
Prompt · paste into your AI agent
/qcsd-ideation-swarm
Analyze the guest-checkout epic in requirements/epic-checkout.md, using requirements/user-stories.md and requirements/acceptance-criteria.md for context. Work from these files only, with no web research. Skip the security audit: Exercise 4 covers security.
Save all reports under reports/01-ideation-swarm/.
Save learnings and persist patterns.
Save toreports/01-ideation-swarm/Two versions in LAB.md
While it runs
Read the epic yourself. Which verdict would you give: GO, CONDITIONAL or NO-GO?
Before you start
Your AI agent is signed in inside this workspace and the AQE MCP server is approved. Not on Claude Code? LAB.md has a step list.
Exercise 1 debrief
Last 5 minutes of the slot · in pairs, then the room
Ideation: surface risk before any code
1In the report
Which verdict did you get?
NO-GO, CONDITIONAL or GO. Find it and its top blockers in the executive summary.
2In pairs
Which contradiction between the epic, the stories and the criteria did it find?
Check one against the files in requirements/. Is it real?
3In the room
What would you refuse to start building without?
Show of hands on the verdict, then one blocker per table.
Open reports/01-ideation-swarm/
Our measured run (open after the debrief)
Measured on 6 November 2026, agentic-qe 3.15.18, Claude Code, unattended: 6.7 minutes, $2.25. Interactive runs take longer because you approve tool calls. Verdict in this run: CONDITIONAL, 39 risks (4 critical), testability 59%, requirements completeness 46%. The skill’s stricter threshold table would read NO-GO, and the report says so.
14:20–14:35
Coffee break
Back at 14:35. Keep the report from Exercise 1 open.
Next · Exercise 2 · Refinement: product factors before coverage
Photo: Karl Fredrickson / Unsplash
Refinement: product factors before coverage
Exercise 2 · Refinement · 14:35–14:50
About 10 minutes · qe-product-factors-assessor
Prompt · paste into your AI agent
Use qe-product-factors-assessor to analyse the guest-checkout product from requirements/epic-checkout.md and requirements/user-stories.md. Produce a product-factors (SFDIPOT) assessment and save it to reports/02-refinement-product-factors.md.
Save learnings and persist patterns.
Save toreports/02-refinement-product-factors.mdTwo versions in LAB.md
While it runs
Which of the seven factors do you expect to carry the most risk here?
Which test idea would you not have thought of yourself?
Then find one that is generic enough to fit any checkout.
3In the room
Which factor carries the real risk for this business?
One factor per table, with the reason.
Open reports/02-refinement-product-factors.md
Our measured run (open after the debrief)
Measured on 6 November 2026, agentic-qe 3.15.18, Claude Code, unattended: 3.3 minutes, $0.52. The AI agent produced 74 test ideas across the seven SFDIPOT factors, 23 requirement gaps and 11 ranked risks. It did not read the acceptance-criteria file, which is worth asking the room about.
Development: tests for the highest-risk module
Exercise 3 · Development · 14:50–15:10
About 15 minutes · qe-test-architect
Prompt · paste into your AI agent
Use qe-test-architect to generate a comprehensive test file for the payment-retry logic in src/lib/payment-retry.ts. Use the test ideas in reports/02-refinement-product-factors.md as input, and include property-based tests for the module's invariants.
Save the test file as tests/lib/payment-retry.architect.test.ts and a short rationale to reports/03-development-tests.md.
Save learnings and persist patterns.
Save totests/lib/payment-retry.architect.test.tsTwo versions in LAB.md
While it runs
Open src/lib/payment-retry.ts. Name one invariant its tests must hold.
Order matters
This step reads what Exercise 2 wrote. Run them in order.
Exercise 3 debrief
Last 5 minutes of the slot · in pairs, then the room
Development: tests for the highest-risk module
1In the report
Do the generated tests run, and do they pass?
For a failing test, decide whether the test or the code is wrong.
2In pairs
Which invariants did the property-based tests choose?
Look for backoff bounds, retry limits and idempotency.
3In the room
Are those the invariants that matter?
One missing invariant per table.
Run npm test -- --run tests/lib/payment-retry.architect.test.ts
Our measured run (open after the debrief)
Measured on 6 November 2026, agentic-qe 3.15.18, Claude Code, unattended: 3.0 minutes, $0.55. The AI agent wrote 74 tests, all passing, with fast-check property tests. One it.fails test documents that jitter is added after the delay cap. It also recorded that every thrown error is retried, even a card_declined code.
CI/CD: verify, gate, decide
Exercise 4 · CI/CD · 15:10–15:25
About 10 minutes · qe-queen-coordinator
Prompt · paste into your AI agent
Use qe-queen-coordinator to run a verification-only quality assessment of src/lib/payment-retry.ts (do NOT generate tests). Analyse coverage gaps with risk scoring, security-scan the module, apply a 90% quality gate, and give a GO / CONDITIONAL / NO-GO deployment recommendation with the top blockers.
Save the consolidated report to reports/04-cicd-quality-assessment.md.
Save learnings and persist patterns.
Save toreports/04-cicd-quality-assessment.mdTwo versions in LAB.md
While it runs
Read src/lib/payment-retry.ts. Would you ship it? Write down your answer.
Coverage clears 90%
payment-retry.ts is at 100% of statements and 97.29% of branches before Exercise 3 adds tests. After it, 109 of 109 tests pass at 100% coverage. Coverage alone does not make the release call.
Exercise 4 debrief
Last 5 minutes of the slot · in pairs, then the room
CI/CD: verify, gate, decide
1In the report
Which recommendation did you get?
NO-GO, CONDITIONAL or GO. Compare the gate score with the coverage figure.
2In pairs
Did it report the missing idempotency key and the backoff cap applied before jitter?
Both are in the module’s source.
3In the room
Would you ship it?
Show of hands, then what you would be willing to ship without.
Open reports/04-cicd-quality-assessment.md
Our measured run (open after the debrief)
Measured on 6 November 2026, agentic-qe 3.15.18, Claude Code, unattended: 1.4 minutes, $0.34. Verdict in this run: NO-GO with 109 of 109 tests passing at 100% coverage. Blockers: no idempotency key, no gateway timeout, every thrown error retried. The quality_assess gate call returned insufficient evidence, so the AI agent applied the 90% threshold itself.
Self-learning: put the fleet’s memory to work
Exercise 5 · Self-learning · 15:25–15:35
About 10 minutes · same prompt for every tool
Prompt · paste into your AI agent
Recall from AQE memory what the fleet learned about this checkout app across Exercises 1–4: payment retry, cart state, checkout form accessibility, and request validation.
First list what you recalled: each pattern's name and one line on what it says. Then consolidate that into a one-page brief (top risks, testability gaps, contradictions, and the release verdict), framed as either:
• an onboarding brief for someone joining the project today, or
• a handoff document for the next person enhancing the checkout app.
Where memory returns nothing for a topic, take it from reports/01 to 04 and label that part REPORTS-BASED.
Save the brief to reports/05-handoff-brief.md.
Save toreports/05-handoff-brief.mdSame prompt for every tool
While it runs
Write down three risks you remember from Exercises 1 to 4, then check the recall.
What the fleet recalls
Top risks, ranked across the exercises
Testability gaps that block quality work
Contradictions between epic, stories and criteria
The release verdict and its blockers
Exercise 5 debrief
Last 5 minutes of the slot · in pairs, then the room
Self-learning: put the fleet’s memory to work
1In the report
Does the recalled list name this codebase?
payment-retry.ts, CartContext and CheckoutForm.tsx should all appear.
2In pairs
Which parts of the brief are REPORTS-BASED?
Those topics were not in memory. Compare them with your own reports.
3In the room
Would you hand this brief to the next person on this code?
One thing per table that you would add first.
Open reports/05-handoff-brief.md
Our measured run (open after the debrief)
Measured on 6 November 2026, agentic-qe 3.15.18, Claude Code, unattended: 1.0 minutes, $0.29. Memory returned four patterns from the learning namespace, the same four for every topic. Cart state, form accessibility and request validation came from the reports. A brief built partly from reports is a valid result; it must say so.
Stretch 2b · Exploratory charters from the product factors
Stretch · after the session
About 20 minutes · exploratory-testing-advanced
Prompt · paste into your AI agent
/exploratory-testing-advanced
Using the test ideas in reports/02-refinement-product-factors.md, write three time-boxed exploratory charters for the guest checkout and run a first pass of each against src/app/checkout, src/components/CheckoutForm.tsx and src/lib/. Label every finding EXECUTED, STATIC or INFERRED; list the questions a human must answer.
Save to reports/02b-exploratory-charters.md.
Save learnings and persist patterns.
Save toreports/02b-exploratory-charters.md
Your call
Which charter you would run first, and which findings are evidence rather than guesses.
Stretch 2c · Defect risk profile: where will the bugs be?
Stretch · after the session
About 15 minutes · qe-defect-predictor
Prompt · paste into your AI agent
Use qe-defect-predictor to build a defect risk profile for every module in src/lib/ and src/components/CheckoutForm.tsx from complexity, coupling and coverage (npm run test:coverage). Rank the modules, give the top three risk factors for each with its evidence class, and say whether src/lib/payment-retry.ts belongs at the top of the list, and why.
Save to reports/02c-defect-risk-profile.md.
Save learnings and persist patterns.
Save toreports/02c-defect-risk-profile.md
Your call
Coverage already reads 100% on payment-retry.ts. If it still ranks high, decide whether you agree with the reason.
07Section 0715:35–16:00
Apply
What the swarms prepare, how to interrogate them with PACTS, and a roadmap for your context.
Summary · PACTS · Roadmap · Take-aways
Photo: Mateusz Sobczak / Unsplash
What the swarms prepare, and what you decide
Summary
PhaseOn the board before you sit downStill your call
IdeationQuality criteria, ranked risks, a testability read on every acceptance criterionWhether the epic is ready, and what you refuse to start without
RefinementThe product broken into its factors, with test ideas against each oneWhich factors carry the real risk for this business
DevelopmentRunnable tests for the riskiest module, including its invariantsWhether the invariants it chose are the ones that matter
CI/CDCoverage gaps, a security scan, a gate result, a recommendationThe release call, and what you are willing to ship without
Self-learningA one-page brief consolidated from what the fleet rememberedWhether that is the story you would hand the next person on this code
Nothing in the right-hand column is delegated.
PACTS: five interrogation tools
Apply PACTS · 15:35
Each property is a question, not a definition. Walk it against your reports, score each report 0 to 3 per property in pairs, and use the probe when the answer is thin.
P
Proactive
Did it flag risk before being asked?
“Cite the exact src/ lines that drove your conclusion.”
A
Autonomous
Did it decide what to inspect for itself?
“What is the most important thing you did not look at?”
C
Collaborative
Did the AI agents reference each other?
“Do AI agents A and B agree on the top three risks?”
T
Targeted
Was it fitted to this checkout code?
“Rewrite this for our context, not a generic checklist.”
S
Structured
Can you trace each conclusion to its evidence?
“Show the trail: which AI agent decided this, from which input, and why?”
Compare engines. If your pair has a Claude Code user and someone on another tool, compare the two reports for the same exercise. That gap is the value of the orchestration layer.