Couchdrop
Published in

Couchdrop

What is a delta migration?

Cloud and data migrations can be time consuming and while you are migrating your initial wave of content, data may change or become out of date by the time you and your users go to access it. This is where delta or partial migrations are useful.

A delta migration only moves modified files since the last migration that was executed. This is done by comparing the modified timestamps between the source and destination platforms. There are a couple benefits from using delta migrations, the remainder of this article will touch on a few of these.

Updated Files

The primary reason why service providers and companies use delta migrations with Movebot is to ensure that the data has made it across and is up to date prior to teams picking up the new platform. When moving 20 terabytes of data, this isn’t going to appear overnight. We recommend, depending on how much data you have, that you do the mass migration a week or two prior to your go-live date. Once the first wave has completed, we would recommend enabling regular delta migrations (or a sync) to ensure that any modified files have been migrated across. It would look something like;

Wave 1. Initial mass migration, this would move all of your data, content and users across to the new platform.

Wave 2- n. At 3–4 day intervals run a delta migration (sync) as required until the weekend before the go live.

Wave (final). Over the weekend prior to the go live of the new platform run a final delta migration so that all of the data is up to date (this is if you have not got cloud sync enabled — otherwise this would be automatic)

Speed

If you wanted to transfer all of your modified files without a delta migration, you would have to consider running the mass migration again, depending on how much data you have, this could be time consuming and may not have all of the updated data migrated across prior to the go live date. As the delta migration only transfers modified files, the amount of files being transferred is tiny in comparison and can often be transferred in less than an hour. This decreases time waiting for large datasets to be migrated across and overall increases speed of adoption.

Cost

Data migrations are not cheap, as much as Movebot prides itself on being at least 1/5th the price of its’ competitors there is still a cost to migrate the data. Often this cost is one way or another per a gigabyte (as this is the biggest cost to data migration tools). Having to continuously do mass migrations in place of delta migrations would be costly and not a productive project. With delta migrations, as you are only moving files that have been modified, often this is only kilobytes in place of gigabytes or even terabytes of data. Because you are only moving small amounts of data and only data that has been modified, your overhead and overall cost has been decreased.

In summary, the delta migration is a neat feature that saves you time, financially and ensures that productivity is not impacted as the new platform can be adopted quickly and seamless by teams as if they were using the same platform.

Related Posts

Originally published at https://www.movebot.io.

--

--

--

Couchdrop is the secure file gateway and cloud SFTP / MFT platform. Couchdrop acts as your secure access method between systems or for clients to send you files to your cloud storage backend, that can be completely automated.

Recommended from Medium

Installing Apache Hadoop on Ubuntu

Get Photo Data from Google Drive!

Multithreading in Java

Ready. Set. Go Hyperspeed!

3 Reasons why Ansible is Replacing Homegrown Automation

How to deploy React app to Github Pages

Managing the project with subproject — Git submodule

How to acquire drivers for rescue media & ReDeploy

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Jayden Bartram

Jayden Bartram

COO of Couchdrop and Movebot

More from Medium

Migrating between Dropbox accounts

transfer between dropbox accounts

Beyond CI/CD: Top 10 challenges in operating modern cloud applications

Show a helpful pytest summary on GitHub Actions

What is a service mesh?