What does a successful cloud data migration mean?

Last month, we hosted an insightful webinar titled “4 Steps for a Successful Data Migration”. The aim for the session was to help any professional responsible for the process of moving data to the cloud and leveraging the capabilities of the cloud computer environment to store and use data in an optimal manner.

While the webinar explored in detail all steps to completion including data stack selection and organizational roll out, often the first and most important step is overlooked. Defining success and setting the right goals can make or break the success of your entire project.

Here are some considerations that anyone planning to migrate their data to the cloud should take into consideration.

Where do I start?

1. Know what you have: having a deep understanding of your current state. Assess your current sources in terms of data size, data format, data location — and even the data relationship as there might be multiple layers of data architecture that need to be taken into account.
2. Set a time frame: establish a time base goal. How quickly can you get data to the cloud and show value? Make sure that you can prove quick wins along the way and share the benefits of the migration sooner rather than later.
3. Play to the strength of the cloud: migrating to the cloud is much more than simply replicating what you have on premise to the cloud. Even if that would be your first step, then you will incrementally convert and redesign your data model to the strengths and capabilities of your cloud data warehouse.

Break down your migration process into two clear phases.

Phase One: AS IS MIGRATION

This is the “quick win”. In a nutshell, it means replicating what you had on premise to the cloud. This can be done very quickly. It involves replicating the data and metadata, automating this process for future data refreshes, and swapping the connections of downstream dashboards and applications to pull from the new cloud data warehouse.

Phase Two: INCREMENTAL RE-DESIGN

To truly migrate to the cloud, you need to play to the strengths of the cloud. This involves gradually redesigning and building the new platform to feed directly from sources, priorities, etc. In this incremental conversion phase you want to incorporate a ‘bottom up’ iterative approach to your conversion process — so subject areas of your data warehouse can be weighted by their conversion benefits.

What does it mean to “redesign your data model” for the cloud?

It’s easier said than done. Data modelling for a cloud-native data warehouse can be challenging. In essence, the goal is to balance your storage optimization with simplicity of consumption. For example, on the technical side you can optimize capabilities of the cloud by storing data in a less structured format, but this might not necessarily optimize ease of use for end users. Many end users will only have basic SQL skills and therefore could lead to a decrease in adoption by BI teams that aren’t able to get what they need. So the design approach should aim to optimize the ease of use for as many business use cases as possible while recognizing the scheme optimization to optimize cost and performance.

In addition, it’s important to consider best practices when it comes to designing your cloud data warehouse. It’s important to separate physical areas of your data warehouse. Firstly, one to store the data in its raw format from the source systems. Secondly, have an area that acts as your central location for transformations such as flattening or performing joins. Lastly, a semantic or reporting layer that is linkable to business applications or data governance tools — which will feed the basis of your data consumption.

Hope these expert tips help you kickstart a successful data migration process. Of course, it is only the beginning. If you want to learn more about this, plus the 3 other steps to achieve a successful data migration to the cloud you can watch the recording of the full webinar here.