Shifting Towards A Logical Data Warehouse in 4 Steps
This blog was penned by Jonathan Wisgerhof, Senior Architect, Kadenza
The logical data warehouse is the data architecture of the future — fast, flexible and ideal to support self-service BI. In the past year, I have published quite a lot regarding the architectural advantages of the logical data warehouse (read my previous blog posts here and here).
In an age where the role of information continues to become more business-critical and where data analysis plays a differentiating role in primary business processes, the demand for an architecture that can supply information faster with greater flexibility has drastically increased. Not really a surprise when you consider the fact that a lack of correct information can bring business processes to a complete halt. Information has thus become the fuel of the 21st century and demands a new engine!
Hybrid Architecture
But what about the investments that we have already made in traditional data warehouses, data marts, analytics, reports, and dashboards? Has all that money been wasted? Must we build everything from scratch? No, definitely not! I previously recommended opting for the hybrid approach with data virtualization. I would like to explain how one can migrate to such a hybrid architecture in a controllable fashion without having to close the information system. A well-organized, step-by-step, ‘agile‘ migration retains as much of the existing investments as possible and minimizes risks. The architecture of the logical data warehouse is perfect for this!
Data Virtualization
One of the ways in which the logical data warehouse can be implemented, is by means of the data virtualization concept. Data virtualization platforms typically offer functionalities ranging from the ability to decouple the source system to the publication of information for BI tools and all possible information services. The basis for our migration approach is ‘decoupling’ by means of virtual data marts, but before we go into detail, I will provide you with an overview of the logical data warehouse architecture based on data virtualization:
The connect layer ensures decoupling of the data sources. This layer takes care of the access to the required data in the connected source systems. The combine layer generates reusable, integrated elements of the data and finally, the publish layer serves as the uniform information model (containing all business logic) used by applications and services. This uniform information in the publish layer can be made available for use in various ways: virtual database, views, web services, QVDs, etc.
Migration Strategy
During the migration to a logical data warehouse, the information system must naturally remain open. The supply of information should not be jeopardized under any circumstance, however, we want to create the possibility of adding new data and information to flow outside of the traditional, customary architecture of the classic data warehouse. An additional reason to implement the migration in small, controlled steps, is so that the existing information flow is not compromised. Migrating to a hybrid architecture thus allows the system to remain open and keeps clients satisfied.
