/ Quality engineering
Turning business risk into a release decision you can defend.
Most engineering teams have automation and still ship defects. They cannot tell a product bug from a test bug, and when someone asks whether the release is ready, the answer comes down to whoever sounds most confident. DawerOne builds the system that fixes that, and runs it inside your team.
Six stages between a business risk and a signed release.
If requirements are never traced to tests, the coverage number is fiction, and so is every decision built on top of it.
- 01
Risk
We rank the business risks first. Everything after this is measured against that ranking.
- 02
Requirements
Each requirement is a versioned file with an owner and a risk score. CI checks that tests exist for it.
- 03
Layers
Unit, contract, API, integration, UI and accessibility. A lint rule enforces which layer a test belongs in.
- 04
Pipelines
Gates run at pull request, UAT, staging and production. Each stage has its own time budget and severity policy.
- 05
Evidence
Failures arrive already triaged and assigned. Executives and engineers get separate reporting off the same run.
- 06
Decision
The run produces a release recommendation, and stores the reasoning behind it.
A requirement is a file, with an owner and a score.
Eleven of these define what the system has to do. Tests declare which one they cover, and CI fails if a requirement has none.
id: REQ-ORD-003
title: Order submission is safe to retry
businessOwner: VP Commercial Operations
risk: high
criticality: revenue
riskScore: 15
impactedSystems: [order-api, postgres]
acceptanceCriteria:
- Given an Idempotency-Key, when the same request is
sent twice, then exactly one order exists and the
second response is 200 with Idempotent-Replay trueRisk and criticality drive coverage targets. Acceptance criteria drive the tests.
Most suites drift towards the browser, and get slower every sprint.
A test at the wrong layer costs more to run and tells you less when it fails. Below is the shape of the reference suite, by test count and by execution time.
Accessibility
Keyboard and screen reader paths
2%6 tests
UI
Journeys that need a real browser
6%17 tests
Integration
Services wired together
7%20 tests
API and contract
Behaviour at the boundary
40%119 tests
Unit
Logic, in memory
45%134 tests
Each layer as a share of the suite. Most teams draw this shape on a whiteboard and then build the inverse of it.
Test breakdown
296 tests
Time to run
78s to run
Unit
<1ms to run · low to write
Triage points at a function
API
103ms to run · low to write
Triage points at an endpoint and its contract
Contract
113ms to run · low to write
Triage points at which side drifted
Integration
655ms to run · moderate to write
Triage points at a boundary between two services
Accessibility
2.8s to run · moderate to write
Triage points at a specific rendered element
UI
2.1s to run · high to write
Triage points at anything from a selector to a real bug
Unit, contract and API tests are 86% of the suite and 16% of the execution time. UI and accessibility are 8% of the suite and 67% of the time. That is the pyramid doing its job: fast feedback at the base, browser tests kept to the journeys that need a browser. A lint rule enforces the shape as the suite grows.
Execution time is measured on this run. Authoring effort is our estimate.
Latency has a budget, and the budget blocks releases.
Order search
REQ-ORD-008The screen every sales representative opens dozens of times a day
Budget
p95 under 400ms
Load
20 concurrent users, 30s
Requests
1,838
Order workflow
REQ-ORD-005The full commercial transaction: create, submit, read back
Budget
p95 under 1200ms
Load
8 concurrent users, 30s
Requests
2,577
Rate limit
REQ-ORD-008The limiter should degrade gracefully under a burst
Budget
p95 under 500ms
Load
15 concurrent users, 15s
Requests
2,738
One thresholds file is the source of truth. Both the k6 scripts and the fallback harness read it, so the two can never disagree about what fast enough means. On the reference run the worst p95 was 89ms against a 400ms budget, with no breaches.
"order-search": {
"description": "The screen every sales representative opens
dozens of times a day",
"requirement": "REQ-ORD-008",
"vus": 20,
"thresholds": {
"p95Ms": 400, "p99Ms": 800,
"errorRatePct": 1, "minThroughputRps": 20
}
}Every scenario names the requirement it protects.
Passes locally, fails in CI, and everyone stops believing the suite.
Once a team learns a red build might mean nothing, the gate has stopped working.
No fixed sleeps
Waits are explicit, with a timeout and a stated condition. A suite full of hard-coded sleeps is slow when it passes and useless when it fails.
Per-worker isolation
Parallel workers never share state, so a test cannot pass or fail because of what another worker did first.
Deterministic data
Run the API suite twice with no database reset and it still passes. Most suites cannot, which is why they only work against a clean environment.
Traceability checked in CI
A traceability check runs on every pull request. A requirement with no test, or a test citing a requirement that does not exist, fails the build.
A change passes three gates, and each one can stop it.
The gate policy is a YAML file, so a delivery lead can read it and change a threshold through a pull request without touching the framework. Blockers fail the stage. Warnings do not stop the build; they downgrade the release recommendation to CONDITIONAL GO.
Pull request
Every push. Under 12 minutes.
- Any failed test
- Any unit failure
- Contract verification
- Flake budget exceeded
Test environment
Deployed environment. On merge to main.
- Any failed test
- High risk coverage below target
- Automation reliability
- Duplicated coverage across layers
Pre-release
Full evidence set. Before shipping.
- Any failed test
- High risk coverage below target
- Latency over threshold
- Contract verification
- Performance headroom
- Automation reliability
- Flake budget
A check whose evidence is missing counts as inconclusive, and an inconclusive blocker fails the stage. That is the rule that stops a broken reporting step from being read as a clean run.
- id: high-risk-coverage
description: Every critical and high-risk workflow must be protected
metric: coverage.highRiskProtectedPct
operator: gte
threshold: 100
severity: blocker
remediation: Restore the missing coverage, or record an
explicit, owned risk acceptance.One of nine checks. Data, not code, so a threshold can be argued with in a pull request.
Agents do the reading. People stay accountable for the call.
This is how DawerOne practises quality engineering. Seven agents work across the cycle, from challenging a requirement before it is built to drafting the release narrative, and they run against the repository itself rather than a summary of it. Each is tied to a schema, and a person signs every output.
The difference is not that this happens faster. It is that it happens at all. Re-checking coverage against risk on every change, reviewing every pull request against the house standards, classifying every failure with the evidence behind it: teams skip that work by hand because it does not pay for itself. Here it is continuous.
Week 1
Requirement analysis
Reads each business requirement and returns the risk, the missing acceptance criteria and the ambiguities, before anyone writes a test.
Two weeks of requirements workshops
Week 1
Test layer recommendation
Says which layer proves each scenario, and where the layers above it only need a smoke test.
The argument that ends with everything becoming a UI test
Week 2
Test plan
Turns a requirement into the task list and the test data it needs, including negative and boundary cases.
Blank page time at the start of every ticket
Ongoing
Coverage gap
Compares the risk model against what is actually protected and names what is exposed.
Finding out during an incident
Every PR
PR review
Checks new automation against the house standards and runs inside CI.
Standards drifting the moment the team grows
On red
Failure analysis
Proposes product, test or environment for each failure, with the evidence it used.
The triage meeting
Release
Release summary
Drafts the narrative, the talking points and the questions to expect.
Writing the release note at 8pm
The reviewer that runs on every pull request
It reports only what it can point at: a file and a line. Any high-severity finding returns changes-requested, and the build reflects it. This is how standards survive a team doubling in size.
It also states the limit of its own review: it checks what it has rules for, so a clean agent pass is not the same as a clean review.
agents/prompts/pr-review.md
- 01Fixed sleeps: waitForTimeout or setTimeout inside a spec
- 02Raw HTTP in specs, bypassing the client layer
- 03Missing traceability: no requirement annotation
- 04Happy path only, with nothing asserting a failure response
- 05Focused tests that silently disable the rest of the file
- 06Fragile locators: CSS or class selectors instead of test ids or roles
- 07Weak assertions such as toBeTruthy on a response
- 08Duplicate coverage already owned by a lower layer
Four things the agents are not allowed to do
An agent that fabricates a metric is worse than no agent, because it is faster at producing something that looks like evidence. Every prompt carries the same constraints and the schemas enforce them.
It cannot invent
Paths, test names, metrics and requirement ids must come from the supplied context. Saying no performance evidence was produced is a valid finding. A plausible number is not.
It cannot ramble
Output is JSON validated against a published schema. No prose, no markdown, nothing that needs interpreting before it can be used.
It cannot assert without evidence
Every finding points at a file, a line, a metric or an acceptance criterion.
It cannot approve itself
Every schema requires a sign-off block naming an accountable role, and the agent must leave the status pending.
/ A look inside
You are seeing one agent's checklist and the rules all seven work under.
The prompts, the schemas, the layer decision table and the playbooks that turn them into a working pipeline come with the engagement. We walk through the whole method on a call, against your codebase rather than ours.
Teams re-run failing suites because nobody can say what the failure means.
Product bug
The system behaved wrongly. It becomes a defect with a named owner and a severity.
Test bug
The test was wrong. It gets fixed. Re-running it would only hide the problem.
Environment
The run never produced a valid result, so it does not count as evidence either way.
Three possible answers, each with its reasoning stored.
The recommendation comes from the quality gates, workflow coverage, automation reliability and latency. It is recorded at the time, so six weeks later you can still see what was known and who signed off on what.
Gates pass, high risk workflows are covered, reliability and latency sit inside their thresholds.
Nothing is blocking, but a residual risk is open. It ships once a named person accepts that risk on the record.
A blocking gate failed, or the evidence for one is missing. Both stop the release.
# Release readiness - R-2026.07.31
**Recommendation: CONDITIONAL GO**
| Quality gate | pre-release -> pass |
| Critical/high workflows protected | 11 of 11 |
| Automation reliability | 100.0% |
## Why
- No blockers, but 1 condition(s) require a named owner
and an agreed follow-up.
- Worst observed p95 latency 88.98 ms, 0 breaches.Generated by the run. Nobody types this.
Defects you already know about are on the ledger
Every release carries known issues. Recording them is what separates an accepted risk from a surprise, and it lets the release report be built from evidence the pipeline can reach. Severity decides what each one does.
Critical
Blocks the release automatically.
Major
Becomes a release condition with a named owner who accepts it.
Minor
Recorded against the requirement it affects, with no effect on the recommendation.
Four rules the system enforces on us.
Each of these is written into the codebase somewhere, in a gate policy, a lint rule, or the logic that produces the release recommendation.
“Absence of evidence is not evidence of quality.”
“A condition is not a defect. It is a residual risk the release can proceed with only if a named person accepts it.”
“A gate slower than a coffee break gets routed around.”
“Test count is the one number that rewards exactly the wrong behaviour.”
We are trying to make ourselves unnecessary.
DawerOne runs the quality system inside your team and coaches your people while doing it. Everything we build stays in your repository, in formats your engineers already read, so the capability is yours whether or not we are still here.
We hold the gate
The rituals, the quality gate and the release decision run on a set cadence with us accountable for them. When a KPI slips we act on it, rather than reporting that it slipped.
We coach while we do it
Your engineers work the system with us rather than around us. Enablement is measured like anything else here, against a target above seventy percent.
It lives in your repository
Requirements, gate policy, pipelines and reporting are files in your repo, in formats your team already reads. There is nothing to extract if we leave.
“If the client could not run this model for a quarter without us, we have built the wrong thing.”
Fifteen minutes, on a call.
The same coffee break we budget for a pull request gate. We start the system, run the tests, break a control on purpose, watch the release get blocked, then fix it and watch it clear. It runs on one machine with an embedded database and no network access, so nothing touches your systems or ours.
If it is worth a longer conversation after that, we book one. We are not asking for your morning.
Want this running on your delivery?
We build and operate this inside engineering teams, and coach yours while we do it.