Data Migration Plan — The Migration Scope

In the last blog of the legacy data migration series, I covered the rationale or ‘The Why?’ for crafting a data migration plan when decommissioning legacy business systems. I listed ten components one should cover when assembling a data migration plan. This blog covers the first component in-depth — The Data Migration Scope.

Scope it out.

Defining the scope of data to be migrated into the new system is the most significant exercise when decommissioning legacy business systems with decades (s) of transactional data. It is a cross-functional business decision where stakeholders from multiple business units should be involved, especially the legal or compliance team. The amount of historical data needed in the new system depends on two major factors :

• Uninterrupted continuity of business operations in the new system post data migration.
• Adherence to law and regulations (generic and industry specific) about legacy data retention

With decisions about the scope of data for migration on one side, the other side of the coin would demand decisions about legacy data archival or purging methods. From a business analyst’s perspective, you can live happily with only seven years of historical data in the new system. However, your legal team may suggest migrating the last ten years of data to comply with a particular data retention law (national or state-level).

On the technical front, clarify the number of instances of the legacy system early on, i.e., only one centralised instance is running, or multiple business locations have their dedicated instance. For the latter case, with numerous instances of the legacy system, you will face reconciliation challenges of duplicate records and the nature of decommissioning, i.e phased or big-bang. For example — when a customer or patient record exists in multiple instances of the legacy system, business rules about which record is considered the latest and the order or nature of decommissioning (phased or big-bang) must be documented in the first component of a data migration plan.

Organisations with higher data governance maturity and data lifecycle management would have an enterprise data architect or a similar role. Such stakeholders could rightly recommend leveraging existing data archival policies while making decisions about the archival or purging out-of-scope data [1]. In such ideal case, such stakeholder with enterprise-wide data responsibilities role would liaise with decision-support/business-intelligence/reporting teams on re-factoring existing reports and dashboards. In diverse cases, the stakeholders may expect data archival decisions and reporting re-factoring as a part of the deliverables from the data migration project. I recommend verifying these stakeholder expectations early on and documenting risk assessment of the scope selected, articulating each risk with its impact and likelihood on the business.

You could also have multiple options for scope selection, each with a distinct set of risks and benefits associated with it. In such cases, the data migration plan should propose all options and capture the stakeholder’s decision on the option selected. Another critical thing to mind here is that Scope is calculated from the day of cutover — i.e. Scope of the last ten years means data accumulated from 1st July 2012 for the planned cutover date of 1st July 2022. Factor this in when planning an incremental decommissioning with multiple cutover dates for the legacy system — i.e. gradual decommissioning by business unit/department/site over a period of time.

In Summary

The first component of the plan, namely the data migration scope, should answer the following questions of business and technical stakeholders:

• Can we quantify the volume of data to be migrated to the new system, and what is the reasoning behind this volume?
• Is there a single legacy system or multiple instances of it at different locations?
• Should we migrate all data across the enterprise in one go or take a cautious phased approach with multiple small migrations?
• What are the pros and cons of the big-bang and phased approach?
• If multiple instances of the legacy system are running, what would be the most logical order for decommissioning them?
• What would happen to the data that is outside the scope, and what if we need the out-of-scope historical data in the future while it is not available in the new business system?
• Are target data structures available as Excel files or as SQL scripts to create target tables in a staging environment , or final data upload is via REST APIs?
• Are target tables or APIs stable with documentation available, or would they change with the project progression?
• What information is available about the legacy system’s data structure, and what is the status of in-house or vendor-based technical support for it?

Next

In the next blog, I will cover the second component — Data Segments and Compliances. Please reach out in comments if you have any question, feedback or suggestion. I am also preparing Excel and Word templates for my readers who want to kick-start drafting a data migration plan based on my questionnaire across ten key components.

I hope decision makers (tech or business) who are directly or indirectly associated with legacy system migrations will find my legacy data migration plan series helpful. I highly appreciate constructive comments/feedback and questions. Feel free to share your thoughts and reach out for early access to data migration plan templates.

Reference:

1. DMBoK — Data Management Body of Knowledge (2nd Edition) Chapter 7— Data Storage and Operations