Learning Objectives
By the end of this lesson you will be able to:
- Describe the main categories of data migration software
- Explain how the software landscape relates to the PDM methodology
- Identify the factors that determine which category of tool is appropriate for a given migration
📦 The DHGS Story - Episode 6: The tools behind the project. This module looks at the software DHGS used to run all of this - extract, transform, load, and the DQR tracking. The honest note: the right tools are changing fast (AI especially), so what matters is that the tooling serves the PDM process, not the other way round. DHGS could have run the same methodology with very different software.
Module 08 then brings the whole DHGS migration together, end to end, for you to work through.
Software and Methodology
PDM is tool-agnostic. The methodology can be applied using any combination of software tools, or with no specialist tools at all on smaller projects. What PDM specifies is the process - the software is chosen to support that process.
That said, tool selection has significant consequences for project cost, timeline, and capability. A tool that is poorly matched to the migration’s scale or complexity will slow the project down and create technical risk. Understanding the software landscape is part of making a good strategy.
Categories of Migration Software
General-Purpose ETL Tools
Examples: Microsoft SSIS, Informatica PowerCenter, Talend, IBM DataStage
These tools are designed for both data integration (ongoing) and data migration (one-time). They offer:
- Graphical design interfaces that reduce coding
- Rich transformation libraries
- Scheduling and monitoring capabilities
- Logging and auditing
Best suited to: large-scale migrations, long-running programmes, organisations with existing ETL tool investments, and scenarios requiring complex transformations.
Limitations: licensing cost; require specialist skills; can be overweight for small migrations.
Purpose-Built Migration Tools
Examples: Tools built specifically for ERP-to-ERP migrations (e.g. SAP migration tools, Oracle data migration utilities)
These tools are optimised for specific platform pairs. They understand the data structures of both source and target systems and reduce the mapping effort for standard data objects.
Best suited to: standard platform migrations (e.g. legacy SAP to S/4HANA, legacy Oracle to Oracle Cloud) where the data structures are well understood.
Limitations: inflexible; poor performance when the migration involves non-standard data, complex transformations, or multiple source systems.
Scripting-Based Approaches
Examples: Python (pandas, SQLAlchemy), SQL stored procedures, PowerShell
Custom-written scripts that extract, transform, and load data. Low licensing cost. Flexible. Increasingly capable as data manipulation libraries mature.
Best suited to: smaller migrations; technically sophisticated teams; scenarios where the transformation requirements are unusual enough that standard tools are a poor fit.
Limitations: higher build cost; more documentation required; harder to hand over to non-technical teams.
Data Quality Tools
Examples: Trillium, Experian Aperture, Informatica Data Quality, Melissa Data
Used for data profiling, deduplication, address standardisation, and data quality remediation. Often used in combination with an ETL tool rather than as the primary migration platform.
In PDM terms: data quality tools support the DQR process by automating the quantification and (in some cases) the automated fix of identified issues.
The Wire Card
The Wire Card is a PDM tool for rapid software selection assessment. It provides a structured way to evaluate candidate tools against the key selection criteria before committing to a detailed evaluation.
The Wire Card covers:
- Scale: can the tool handle the volume?
- Complexity: can the tool handle the transformation requirements?
- Budget: is the licensing cost within the project budget?
- Skills: does the team have the skills, or can they be acquired?
- Integration: does the tool work with the source and target platforms?
- PDM compatibility: does the tool support PDM’s release management and auditability requirements?
The Wire Card is not a formal vendor assessment - it is a rapid filter to narrow the field before more detailed evaluation begins.
Software and the DMZ
In a supplier-led engagement, the choice of migration software often sits on the supplier side of the DMZ. The client should ensure that:
- The chosen tool supports the auditability and data lineage requirements defined in the SRPs
- The client has access to the tool’s outputs (migration logs, validation reports) as specified in the DMZ
- The tool’s licensing does not create commercial lock-in that complicates future migrations
Key Takeaways
- PDM is tool-agnostic; software is chosen to support the process, not the other way around
- The main categories are: general-purpose ETL tools, purpose-built migration tools, scripting approaches, and data quality tools
- The Wire Card provides a rapid assessment framework for software selection
- In supplier-led engagements, the tool choice must align with the DMZ and auditability requirements