Speed Equals Success in a Data Warehouse Migration
A quickly executed cloud migration based on a solid implementation plan is extraordinarily valuable
You wouldn’t know it by looking at me, but I used to be a lean, mean Marine. Infantry, to be precise. And we had a saying when moving through unfavorable territory: “Speed is security.” No matter how much firepower you packed and how much time you invested in preparation, there were always points in a mission or an exercise where a team was highly exposed and extra vulnerable.
While years of Oreos, missed workouts, and soda has eaten away at the “lean and mean,” the phrase “speed is my security” is a maxim that I endorse today when it comes to major IT projects like cloud migrations.
Don’t Be Lazy — Planning is Always Critical
As a young Marine, my peers and I would often use the ‘speed is my security’ phrase to avoid doing the up-front work of analyzing, researching, and planning. This holds for a cloud data warehouse migration as well. Failing to take the time to plan is lazy and can end careers. I’ve seen it happen.
Over-planning is also a common pitfall. I’ve been part of a cloud migration that involved over 10,000 Business Intelligence reports. The initial consultants we brought in to assess and build a plan got lost in the 10,000 reports. Three months and seven figures later, we politely showed them the door and started putting a real implementation plan together with consultants experienced in cloud migration.
Take the time to put an excellent plan together. In the long run, this will ensure that your cloud migration goes smoothly. I strongly encourage anyone interested in a cloud migration journey to consider bringing in qualified consultants to help you start. Consultants can help you configure your cloud environment, optimize performance and cost, help with tool selection, and help build a plan covering all the bases from engaging the business to retiring technical debt. The adage that an ounce of planning is worth a pound of pain is true in cloud data warehouse migrations. I was part of a team that had to do it twice. The first time we did it on our own, and the second time we did it successfully with Hashmap.
Define Success
We can dive into what an implementation plan of a data warehouse migration should look like in future posts. For this post, I think it is critical to highlight that the project team engages the business and determines its success. For those 10,000+ reports I referenced above, we had to draw the proverbial line in the sand as a project team (composed of members of the business and members from IT) on what our project could and could not accomplish. We bucketed data products into four categories:
- Mission-critical to the enterprise — We would include data products that were part of the SOW and that we intended to refactor and/or rebuild off of the data in the new cloud data warehouse
- Mission-critical to departments — We would build out the infrastructure and empower the owner of the data product to refactor using the new cloud data warehouse
- Nice to have — We would provide training and guidance, but it was up to the business to refactor the data product
- Should retire — We would outline why these data products should not persist. Either they hadn’t been used in over 12 months and/or the data sources and logic were wrong
We also felt that success meant enabling future programs and functions (data science, ML, AI, etc.), that we should retire legacy systems, and that the broader organization should be set up to succeed in the new environment.
Move That Data Fast
I’m writing this in February of 2021. In the past twelve months, the stock market has hit all-time highs, the economy plummeted into the steepest recession in recent history, we’re in a worldwide pandemic, and the list of drama from 2020 can go on. While 2020 is (hopefully) an extreme case where the world can change dramatically in a 12-month period, most years bring significant change. New leadership can orient on new initiatives, acquisitions, and divestitures can tie up IT staff for months, the adoption of new tools or the upgrade and migration of old tools drain resources, staff attrition is always an issue, and budgets can easily hijack a cloud migration.
At best, the cloud data warehouse's buildout is delayed by a few months should something like a software upgrade or an acquisition occur. At worst, teams are caught with two active warehouses that require constant maintenance and updating. No team wants to be caught in the middle of a cloud migration and realize that adopting a new operational SaaS solution means updating the on-prem data warehouse tables and logic AND the cloud’s data warehouse tables and logic. An event like this can add months to your migration timeline.
This is where I see contractors adding immense value to a cloud migration project. Note that contractors fill a very different role than a consultant. While a consultant is there to guide strategy and advise on best practices based on your company’s specific culture, tech stack, and deliverables, a contractor is there to get stuff done. That’s not to say that the two roles are mutually exclusive (often, the same consultants transition over to serve in the role of contractors). Still, I think there are two distinct outside resources that a company can benefit from a cloud migration.
The benefit of moving quickly and bringing in additional team members goes beyond accelerating the project. It also opens up time for the company’s data team to spend more time engaging the business, coordinating training and working sessions, and taking time to refine the original plan as needed.
In the end, a quickly executed cloud migration based on a solid implementation plan is extraordinarily valuable because it:
- Minimizes the impact on the business. A cloud migration is a disruptive event as teams have to learn new technologies, new skills, and refactor reporting. Moving quickly allows the organization to adjust quickly. This might seem counter-intuitive, but it isn’t. By making the needed data available in the new data warehouse quickly, the business can focus on the refactoring process. This is much better than the business slowly having to refactor multiple reports over a series of months, waiting for the right data to get migrated, getting midway through a refactor only to find that a critical table hasn’t been migrated over, etc.
- Decreases the risk of competing priorities. The less time spent on a migration means the less risk that can occur from the events mentioned above — acquisitions, layoffs, divestitures, reorgs, etc.
- Decreases the challenges of having two sources of the truth. The less time IT spends maintaining multiple data warehouses, the better. The less time the finance team generates KPIs from one warehouse while operations generate KPIs from another data warehouse, the better!
- Most cloud data migrations are sold on cost optimization. Dragging out the project only erodes the economic benefits of the project.
Final Thoughts
If you have any questions or are thinking about kicking off a cloud data warehouse, please reach out to our sales team. We have a lot of resources that we make available to anyone visiting our website. Our sales team can set up a free consultation that will help you determine your business's right course of action.
Hashmap, an NTT DATA Company, offers a range of enablement workshops and assessment services, cloud modernization and migration services, and consulting service packages as part of our data and cloud service offerings. We would be glad to work through your specific requirements.
Hashmap’s Data & Cloud Migration and Modernization Workshop is an interactive, two-hour experience for you and your team to help understand how to accelerate desired outcomes, reduce risk, and enable modern data readiness. We’ll talk through options and make sure that everyone has a good understanding of what should be prioritized, typical project phases, and how to mitigate risk. Sign up today for our complimentary workshop.
Feel free to share on other channels and be sure and keep up with all new content from Hashmap, an NTT Data Company, here. To listen in on a casual conversation about all things data engineering and the cloud, check out Hashmap’s podcast Hashmap on Tap as well on Spotify, Apple, Google, and other popular streaming apps.
5 Cloud Data Platforms, 33 Dimensions, 400+ hours of detailed analysis — 1 incredible Cloud Data Platform Benchmark Analysis:
Other Tools and Content You Might Like
Scottie Bryan is a Delivery Manager with Hashmap, an NTT Data Company, providing Data, Cloud, IoT, and AI/ML solutions and consulting expertise across industries with a group of innovative technologists and domain experts accelerating high-value business outcomes for our customers. Connect with Scottie on LinkedIn.
