Microsoft Fabric: The Game-Changer?
Introduction
When you look at reports like Matt Turck’s MAD landscape, it’s mind-boggling how many different tools there are in our data world. That number has only skyrocketed in the last few years and quite frankly, probably isn’t slowing down any time soon. Microsoft’s announcement of Microsoft Fabric, an end-to-end data analytics platform, is taking that MAD landscape head-on and presenting itself as an attractive alternative. Will it actually fulfill everything, though?
Microsoft Fabric: A Background
Perhaps the easiest way to understand what exactly Fabric can do comes from looking at the figure below:
Essentially, you have your tool for connecting to data and orchestrating pipelines (Data Factory), various Synapse tools that strike a strong resemblance to platforms like Databricks, a BI platform, and data observability, all unified under a single data lake. Yes, that’s a very basic summary, but that’s essentially the problem that Fabric is trying to solve: unifying the mess that is the data landscape in 2023.
As a DE who has worked with a number of tools over the years, the premise behind Fabric definitely makes sense. With an increased focus on enabling professionals of all backgrounds to get whatever they need out of data, offering integration with Microsoft 365 is a very logical choice. Self-service analytics is the future (if not already the present), so platforms like these make that all the more a reality.
Advantages
What exactly are the advantages of Fabric? For one, it’s driven by the data lake (OneLake in this case), using open data formats like Parquet and Delta for all the necessary workloads. This lake model will also eventually be supporting data management and security-related concerns, which is certainly a relief to data governance practitioners everywhere.
Fabric will also be powered by AI. I’m certainly not an expert in the generative AI phase that’s starting to take over the way we currently do things, but I do know that it can bring a lot of benefits with it when used properly. Having that integrated into every step of the analytics process will reduce the amount of time and effort it takes for new pipelines and ML models to get built, which is always a plus.
Another benefit worth mentioning is how a unified platform will help reduce costs. Whether it be data duplication or multiple forms of compute, the current data landscape can make cost optimization quite the challenge. A single pool of computing that can be used for all Fabric workloads will certainly reduce those costs while also narrowing the scope of focus for further optimizations to said costs.
Thoughts
So, am I ready to hop on the Fabric bandwagon? Right now, my take is to wait and see how this can really be leveraged to mitigate some of the challenges us data teams face on a regular basis. For one, the claim to “bring together all the data and analytics tools that organizations need” is not a claim you want to make very often in the data world. It’s continuously evolving and what we consider to be exhaustive today will fall quite short of that claim in the matter of a few months. Will Microsoft be able to keep pace with the demands of the data world?
One of the bigger elephants in the room with this announcement is the reliance on Azure and all its services. Data engineers use open-source tools to avoid vendor lock-in, so why should we be forfeiting that right by going through a single Cloud? The migration effort from another Cloud to Azure is certainly a process that not everyone wants to take. I do think Fabric will put pressure on the other big guns (AWS and Google Cloud) to release similar offerings, so in due time, that migration may not be necessary after all.
Conclusion
All in all, the Microsoft Fabric certainly was a cool one to see, but we always have to take these things with a grain of salt. Perhaps AWS re:Invent will be bringing a similar announcement towards the end of the year. By then, we’d definitely have more time to properly evaluate whether Fabric lives up to the hype or not.
