Address Data Residency in Software Architecture
Data Residency is a new challenge which many “multi-national” companies are facing. There are other generic terms to describe this challenge “Local Vs Global Markets”, “Data Localization”, “Data Privacy Laws” etc. All are used interchangeably. But what is Data Residency and why its important to today’s and tomorrow’s enterprises? Why it was not a challenge in legacy enterprises? and what is the penalty for not abiding by these laws?
Data localization or data residency law requires data about a nation’s citizens or residents to be collected, processed, and/or stored inside the country, often before being transferred internationally.
As per the definition, the Data Residency laws of a given country defines how the data about its citizens are collected, processed and/or stored inside the country. There are few countries which are strict about not transferring the data beyond its own borders, and there are few other countries which allows to transfer the data across the borders but the “Data Collector” should always maintain a local copy within the countries borders. With growing concerns around the security of data especially, PHI, PII and PCI categorized data, many more countries/governments are getting started to join the bandwagon. At a high level, the requirements can be classified into 3 major categories:
There are no international laws which restrict the data movement yet. Each country started defining its own laws and scope of data which falls under the law. For example, below are few countries and their respective scoped data:
In legacy applications, the data was stored physically in one location and most of the times there was no need to move the data around. With Globalization, thanks to cloud, the companies can store and access data anywhere in the world. The advantage with this approach is improved user experience, and faster to market products. However, this also raises a very big red flag around the data security. The governments of countries have become more vigilant about their citizens data and its respective movement. And the governments are ready to impose very heavy fines on the Companies which are not abiding by the laws.
So its important that the next time we are designing a multi-cloud/multi-national requirement we understand the “Data Privacy Laws” and “Data Residency” laws in particular before designing the solution.
If the Local Vs. Global data is not classified and thought through before the implementation, it can be a big break to the companies dream of rapidly scaling across different markets. The design of such application can be very challenging depending on how many laws/countries you are trying to abide. There are some companies such as InCountry, Salesforce-Hyperforce, Azure etc. which support in the journey of achieving such requirements. It has local data centers which collects the data from the respective markets. Then this data is processed according to the Privacy Laws. It can be as simple as anonymizing the data and as tricky as maintaining a copy in the local data center and ripping out all the compliance related information such as PHI, PII etc. After the data is processed according to the local laws we can load them into a MDM on which the global analytics can be run.
