From discovery to acceptance: requirements traceability for large public‑sector system replacements

The executive decision at stake

Senior leaders typically face two binary-seeming choices: (a) approve supplier build and integration evidence to proceed to staged transition and cutover; or (b) delay and require further work. In truth the decision is conditional: leaders should accept an evidence pack that demonstrates traceability from discovery outcomes through to test results, migration reconciliation and operational runbooks. The real decision is whether the programme has reduced uncertainty to an acceptable level for the approved level of operational risk.

Why requirements traceability fails in large public programmes

Failure to make requirements testable is rarely a technical error: it’s a delivery design error. Common root causes are (1) weak discovery outputs — process maps, user journeys and policy constraints are incomplete or not versioned; (2) fragmented ownership — policy sponsors, product teams, technical leads and suppliers treat requirements differently; (3) poor data discovery — entity definitions, master data and interface contracts are undefined; (4) supplier‑centric carving — suppliers deliver components to contractual acceptance criteria that do not map to end‑to‑end service tests; and (5) missing traceability artefacts — there is no Requirements Traceability Matrix (RTM) linking each user need to acceptance tests, data fields, migration scripts and operational controls.

These failures combine in delivery. A supplier may pass an API contract test while the system still fails an end‑to‑end clinical or financial reconciliation. Boards see green test reports but the live service cannot reconcile numbers or meet statutory reporting — a classic evidence‑to‑service gap.

How process, data, systems, people, suppliers and governance interact

Traceability is the plumbing that ties these elements together. Discovery produces user journeys and process maps (business context). Business analysts convert those into capability and functional requirements (what the system must do). Architects define data models and interface contracts (how systems will exchange information). Suppliers realise functionality against those contracts. Test engineers convert requirements into acceptance tests across unit, component, SIT (systems integration testing), UAT (user acceptance testing) and PAT (post‑acceptance testing). Operations own runbooks, monitoring and rollback. Governance defines decision gates, risk tolerances and who signs each artefact.

If any link is weak, the assurance chain breaks. For example: an ambiguous requirement for 'patient summary completeness' maps to multiple data items across source systems. Without a canonical data definition and migration reconciliation reports, UAT cannot demonstrate the requirement is met and operations cannot be confident the live service will be safe.

The delivery lifecycle actions that matter (from discovery to stabilisation)

Not every phase of a programme is listed here — only the stages that matter for traceability and approval: 1) Discovery: validated user journeys, process maps, policy constraints and a prioritized list of measurable user outcomes; 2) Requirements conversion: capability, functional and non‑functional requirements with a unique identifier; 3) Architecture and data modelling: canonical data model, field‑level mapping and interface contracts; 4) Build and supplier acceptance: component acceptance criteria aligned to the requirement identifiers; 5) Integration testing (SIT): test harnesses that execute end‑to‑end scenarios and produce reproducible test logs; 6) Migration and reconciliation: test migrations, reconciliation scripts, reconciliation reports and migration rollback test evidence; 7) UAT and operational validation: observed end‑user scenario tests, operational runbooks and training evidence; 8) Cutover rehearsal and PAT: full dress rehearsals with failover and rollback validated on live‑like data; 9) Early live measurement and stabilisation: defined metrics, dashboards and time‑boxed stabilisation gates.

Concrete artefacts, decision gates and the evidence pack boards should expect

A credible evidence pack for a board or SRO must include the following, each explicitly linked to a requirement identifier in the RTM: (a) Discovery dossier: approved user journeys, process maps, a stakeholder‑mapped dependency register and the decisions that closed scope questions; (b) Requirements Traceability Matrix (RTM): each requirement ID linked to design, data fields, test cases, migration scripts, acceptance status and owner; (c) Data model and field mapping: canonical definitions, source‑to‑target mapping, synthetic and anonymised sample records used in test migrations; (d) Interface contracts and API conformance test logs: signed contract documents and automated test evidence with timestamps; (e) Test strategy and execution reports: SIT, UAT and non‑functional test results with pass/fail metrics, defect lists prioritised by business impact and corrective action plans; (f) Migration runbooks and reconciliation reports: dry‑run results, record‑level reconciliation summaries and a defined rollback point; (g) Cutover and rollback playbooks: step‑by‑step tasks, owners, critical path timing, communications and explicit rollback triggers; (h) Operational acceptance evidence: runbooks, monitoring dashboards, on‑call rosters, training completion certificates and fallbacks for the first 14 days; (i) Supplier evidence: SLAs, acceptance certificates, evidence of escrow or reversibility tests where relevant; (j) Decision rubric: explicit acceptance thresholds for required KPIs, outstanding defects tolerated (if any) and conditional controls for phased transition.

Each artefact must be signed off by the accountable owner named in the governance register and its status reflected in the DCA/DAR-level assurance used by the project (the programme should map to the government assurance framework in force).

Typical failure modes and how to spot them in the evidence pack

Watch for these practical red flags: • RTM gaps: requirement IDs without linked test cases or data field mappings; • Synthetic test data only: full migration stress tests have not run on representative data; • Interface stability issues: frequent API contract changes between SIT and UAT with no formal change ring‑fence; • Defect distribution skewed to critical business flows: many trivial defects hide two or three showstopper defects unresolved; • Unverifiable reconciliation: migration reconciliation reports that aggregate counts but do not provide record‑level traceability; • Operational unreadiness: runbooks exist but staff training completion is <80% or training evidence is anecdotal. Any of these should trigger conditional hold points or targeted remedial work before leaders approve transition.

How boards should frame conditional acceptance: a practical rubric

Boards should avoid binary sign‑offs. Instead use conditional acceptance with explicit actions and measurement. Examples: • Conditional go‑to‑cutover: accept if the RTM shows 100% of critical requirements mapped to passing SIT tests, migration dry‑run reconciles within an agreed tolerance, and runbook owners are trained and observed performing critical tasks. • Conditional staged live: accept first tranche to a low‑risk cohort if PAT shows zero high‑impact defects for X cycles and monitoring thresholds are green; escalate otherwise. • Hold with a bounded remediation plan: require an agreed list of defects, owners, dates and acceptance re‑test criteria before any move to cutover.

Each conditional acceptance must include an executive‑owned rollback trigger and the costed impact of delay versus mitigations — the board needs to know the likely service impact and the delivery path to resolution.

What leadership should delegate and what they must own

Delegate: the technical design, the construction of the RTM, defect triage, test execution and supplier corrective actions. Own: the definition of acceptable operational risk, the decision rubric for conditional acceptance, the prioritisation of critical requirements and the approval of any scope trade-offs that affect statutory duties or safety. Senior leaders must receive a single consolidated evidence pack, not a stack of compartmentalised reports.

Antares recommended actions

These are Antares's recommended first actions for organisations turning the issues in this article into practical governance and delivery.

1. Require a single consolidated evidence pack for approval that includes a signed RTM linking every critical requirement to design artefacts, test cases, migration scripts and operational controls.
2. Set acceptance thresholds before entering SIT: 100% mapping of critical requirements in the RTM; automated SIT harness executing end‑to‑end scenarios; migration dry‑run with record‑level reconciliation within agreed tolerances.
3. Use conditional acceptance: define exactly which defects may be tolerated for phased live (with owners, fix dates and re‑test evidence) and mandate a rollback trigger for any unrecoverable operational failure.
4. Mandate supplier obligations for traceability: contractually require suppliers to deliver testable acceptance criteria, automated test logs, migration scripts and participation in cutover rehearsals; include reversibility/escape package tests where the data footprint or continuity risk is high.
5. Make operational readiness measurable: require runbooks, monitoring dashboards, on‑call rosters, observed process walkthroughs and >80% validated training completion before allowing live transition for any patient‑facing or safety‑critical service.
6. Require an independent assurance snapshot (DCA/DAR or NISTA/IPA‑style review) immediately before any final cutover approval, focused on the RTM and migration reconciliation evidence rather than generic progress reporting.