Learning Objectives
By the end of this lesson you will be able to:
- Describe how the DHGS migration was built and iterated through the trial run programme
- Explain the testing approach and what the team was testing for at each stage
- Describe the execution and cutover process
- Understand how the legacy systems were decommissioned following go-live
The Build Phase
The build phase for DHGS followed directly from the technical design. The ETL Content Matrix and the System Retirement Plan were the primary inputs. The outputs were working ETL routines, a tested load process, and a validated data set ready for go-live.
The build was iterative. The team did not build everything and then test it - they built a first version of each ETL routine, ran it against a subset of the data, reviewed the results, fixed the issues, and iterated. This approach made it possible to surface problems early, when they were cheaper to fix.
Version control was enforced throughout. Each iteration of the ETL code was tagged. Any change to a transformation rule triggered a corresponding update to the ETL Content Matrix and a new version tag. This discipline was not bureaucratic overhead - it was the mechanism that kept the team aligned and made defects traceable.
Trial Runs
Before the final go-live, the DHGS team ran a series of trial migrations. Each trial run was a full execution of the migration process - extract, transform, load, validate - against a copy of the target system in a test environment.
The trial run programme served several purposes:
Technical validation. Do the ETL routines run without errors? Do they complete within acceptable time limits? Does the data land in the target in the expected format?
Business validation. Do the loaded records look right to the people who know the data? Are the transformation rules producing the expected outcomes? Are there surprises - records that should have loaded but did not, or records that loaded with unexpected values?
Volume and timing. How long does the full migration take to run? Does it fit within the cutover window? If not, what can be optimised?
Reconciliation. Does the record count in the target match the expected count from the source? Are the totals consistent? The reconciliation checks were defined during the design phase and ran automatically as part of the load process, but the team also performed manual spot-checks on records of particular interest.
Each trial run produced a set of findings that fed back into the ETL design and the DQR List. Issues found in a trial run were categorised: blocking (must be fixed before go-live), significant (should be fixed if possible), or minor (can be addressed post-go-live). Only blocking issues stopped the next trial run from proceeding.
Remediation During Build
The build phase was also when the bulk of the data quality remediation happened. Items on the DQR List that required business decisions - the pricing errors in DQR0004, the equipment type mapping in DQR0012 - were being worked through by the business data owners in parallel with the technical build.
The team tracked DQR status closely. At the start of the build phase, the list had a significant number of open items. Each week, the team reviewed progress against the DQR List and escalated items that were at risk of not being resolved before the final trial run.
The DQR process requires a working relationship between the technical team and the business. The technical team needs decisions. The business needs to understand what it is being asked to decide, without being overwhelmed by technical detail it cannot interpret. The DQR item format - with a clear description of the issue, the volume affected, the proposed resolution, and the decision required - was designed to make that interface work.
For DHGS, most DQR items were resolved before the final trial run. A small number were deferred - agreed as post-migration activities with defined owners and timescales. These were documented in the project’s risk and issue log and tracked through to closure.
The Final Trial Run
The final trial run was treated as a dress rehearsal for go-live. The team ran the full migration process - with the current version of the ETL rules, against a recent extract of the source data - under conditions as close to go-live conditions as possible.
The final trial run had a defined set of acceptance criteria. If those criteria were not met, go-live would not proceed. For DHGS, the acceptance criteria included:
- The ETL process completing within the cutover window
- Record counts reconciling within an agreed tolerance
- All blocking DQR items resolved
- Agreed business sign-off from each data owner
All criteria were met. The project was cleared for go-live.
Cutover Execution
The cutover for DHGS was a hard cutover - the legacy system was frozen at a defined point, the final extract was taken, the migration was run, and the target went live. There was no period of parallel running in which both systems were simultaneously active as systems of record.
The cutover sequence:
- Source freeze. At the agreed time, the legacy KP Equipment Table was set to read-only. No further changes would be made to it.
- Final extract. The final data extract was taken from the frozen source and loaded to the staging area.
- Final transformation and load. The ETL process was run. The team monitored progress in real time.
- Validation. Automated reconciliation checks ran as part of the load. The team reviewed results.
- Business sign-off. Each business data owner confirmed that their area of data looked correct in the target. Issues were raised and classified.
- Go-live declaration. The project sponsor declared go-live. The target system became the system of record.
- Legacy transition. The legacy system was made available in read-only mode for the transition period specified in the System Retirement Plan.
The cutover ran to plan. There were minor discrepancies in a small number of records - fewer than had been present in the final trial run. These were documented and addressed through the post-go-live support process.
Go-Live Support
In the days immediately following go-live, the team provided close support to the business. Issues reported by users were triaged: some were data issues requiring correction in the target, some were user confusion about the new system, and some were training gaps that had not been fully addressed before go-live.
The data issues were handled through a post-migration data correction process - a controlled mechanism for making changes to the target data without compromising the integrity of the migration record. Every correction was logged, with a reference back to the original source record and the reason for the correction.
This process is important for a reason that goes beyond the immediate fix. If a migration produces a significant volume of post-go-live corrections, that is a signal that something in the process - in the analysis, the remediation, or the testing - did not work as well as it should have. Tracking corrections makes that signal visible.
Legacy Decommissioning
At the end of the transition period, the legacy systems were decommissioned according to the System Retirement Plan.
For the KP Equipment Table, decommissioning involved:
- Confirming that all users had transitioned to the target system
- Confirming that no active processes were still dependent on the legacy system
- Taking a final archive backup of the legacy data
- Switching off access to the system
Data that had been explicitly excluded from migration - historical records beyond the age threshold, equipment records without site assignments that had not been resolved - was retained in archive in accordance with the data retention policy specified in the SRP. It was not destroyed; it was simply no longer operationally accessible.
The decommissioning of the legacy system was the formal endpoint of the migration. It confirmed that there was now a single system of record for DHGS’s equipment data, and that the legacy system was no longer a shadow source of alternative truth.
Key Takeaways
- The DHGS build phase was iterative - ETL routines were built, tested against subsets, and refined before being run at full scale
- The trial run programme validated the ETL technically, confirmed business expectations, established timing, and drove DQR closure
- The final trial run used defined acceptance criteria; go-live proceeded only when all criteria were met
- The cutover was a hard cutover - source freeze, extract, transform, load, validate, business sign-off, go-live declaration - executed in a defined sequence
- Post-go-live corrections were tracked as a signal of process quality, not just as operational fixes
- Legacy decommissioning was the formal endpoint: a single system of record, legacy systems retired, excluded data archived under retention policy