Use Microsoft Fabric Database Mirroring to Centralize Your Data
Discover how Microsoft Fabric’s new Database Mirroring feature, is set to transform data management and analytics and learn how integration and real-time data replication can unlock unprecedented operational efficiency and insights, all with just a few clicks.
Businesses are continuously searching for more effective ways to handle their data in the field of data management and analytics to improve operational efficiency. In this sense, Mirroring functionality in Microsoft Fabric was recently launched. This feature, which is currently accessible in public preview as of March 26, 2024, promises to completely transform the way that companies handle and use their data.
The Need for Data-Driven Insights
Data-driven insights are more important than ever in today’s world to innovate, make well-informed decisions, and improve goods and services. The problem, however, comes when data is spread among multiple sources, each of which stores data in an incompatible format, such as databases, data warehouses, and applications. This discrepancy makes efficient data analysis and cross-referencing challenging, requiring costly and time-consuming data migration projects.
A Solution: Mirroring in Microsoft Fabric
Microsoft Fabric’s mirroring feature streamlines the data integration process, reducing it from procedures taking hours, days, or weeks to a few clicks and seconds. With the help of this quick technique, data can be continuously and seamlessly accessed from databases or data warehouses into OneLake, doing away with the need for pipelines and enabling data insights in almost real-time.
You will have access to the Mirroring options in Synapse Data Warehouse Experience in Fabric.
Currently, you have the option to mirror an Azure SQL Database, Cosmos DB, and Snowflake database.
How It Works
Database Mirroring
Mirroring eliminates the need to install extra software or switch database clients to provide safe, centralized database access and management within Fabric. Just need to enter your connection information and log in safely to make your database instantly accessible in Fabric as a Mirrored database.
Let´s pick an Azure SQL database to mirror:
1- Give a name to the mirrored database in Fabric
2- A new connection pop-up will show up.
3- Then you get the option to mirror all the data on the database you’ve chosen.
Or select specific tables that you want to mirror.
⚠️There are a few limitations when Mirroring:
- Not supported on Free, Basic, or Standard tier Single Database (S0, S1, S2) and Database in Elastic pool with max eDTUs < 100 or max vCore < 1.
- Can mirror a table because it is a system-versioned temporal history or ledger history table.
- Can’t mirror tables that don’t have a primary key.
- Be careful with some specific data types that can´t be mirrored.
Real-time, Frictionless Data Replication
The method does not require an elaborate setup or ETL to replicate data. Data is updated consistently and reliably with the same connection details, negating the need for coding or intervention. Data is then synchronized in near real-time with each transaction, including the creation of new tables and changes to already-existing data, after an initial snapshot is taken.
Once you select your tables in Fabric, the mirroring starts, after that, you can stop the replication process, monitor replication, and configure your replication settings.
You can Monitor your replication status and check the running status, total rows replicated, and last refresh date.
Using Change Data Capture (CDC)
Using the CDC technology found in the source database, replication transforms the data into the appropriate Delta tables before uploading it to OneLake and to ensure efficient use of computational resources, the logic runs when the source has changed before replicating the data, with that, OneLake guarantees that the data is always available for use in any Fabric task thanks to this control and monitoring, which also provides insights into mirroring operations and data refresh status.
SQL Endpoint
In parallel, you have an SQL Endpoint where you can leverage all the features of the Data Warehouse experience.
Here you can run queries on your mirrored tables and cross data with other data points that you may have, create semantic models, and build reports over your data. You can leverage all the Fabric features on Mirrored database.
Benefits
Elimination of Data Silos
With database mirroring you can unify and govern your data within Fabric Onelake, guaranteeing that the data from various sources can be accessed in one place.
Cost and Time Efficiency
Data migration and integration have historically required sophisticated infrastructures, specialized knowledge, and time-consuming work. Database Mirroring provides a simplified, intuitive solution that completely transforms this process using replication and synchronization of data to automatize the process and minimize the need for human intervention.
Enhanced Decision-Making
Near real-time data insights enable businesses to make informed decisions.
This fast mirroring ensures that decision-makers have access to the most current data.
In Summary
Microsoft Fabric’s database mirroring is a major advancement in analytics and data management. Businesses can now concentrate more on using their data for strategic decisions instead of worrying about the challenges of data migration, the data integration process is now simplified thanks to this easy method to make the data available. It will be interesting to watch how this technology develops and how it continues to influence data-driven strategies in the future for businesses using Microsoft Fabric.
What’s more? For just $5 a month, become a Medium Member and enjoy the liberty of limitless access to every masterpiece on Medium. By subscribing via my page, you not only contribute to my work but also play a crucial role in enhancing the quality of my work. Your support means the world! 😊
