The Top Azure Data Platform Announcements from MS Ignite
Microsoft Ignite took place this year with a range of online webinars & ask the experts sessions taking place between Tuesday 2nd and Thursday 5th of March.
While Microsoft kicked of the session with an impressive new Mixed Reality platform called Microsoft Mesh, Day 1 also included announcements relating to new features and functions in a number of Azure Data and Analytics Services.
Azure Synapse Pathway
Building on the theme of Data Modernisation, Microsoft announced a new service called “Azure Synapse Pathway” aimed at accelerating the process of migrating existing data warehouse solutions to Azure Synapse.
In the past, converting thousands of lines of SQL code, written in whatever SQL dialect your data warehouse platform supported was the biggest body of work involved in a data warehouse migration. Now, Azure Synapse Pathway can connect to and scan your database, producing an assessment report outlining which objects can be migrated to Azure Synapse Analytics. SQL objects in the database are automatically converted to T-SQL and optimsed to run on Azure Synapse. Today, Azure Synapse Pathway covers database schema and table objects from IBM Netezza, SQL Server, and Snowflake, and support for Teradata, AWS Redshift, BigQuery & Hive is on the way. The SQL translation capabilities are also in development while the preview is ongoing and will expand with time. (No mention of Oracle yet, but I have to assume it’s on the way too.)
Seeing more with Purview
Speaking of Oracle, Azure’s new Data Governance platform, Purview, now supports the cataloguing of data from Oracle databases. This extends the reach that already covers on-prem Hive, SAP, SQL Server, and Teradata, multi-cloud data sources, and Power BI data sets. Purview is Microsoft’s new data catalogue & data governance solution, enabling organisations to create a holistic view of all its data, whether it’s on-prem or in the cloud. The tool also provides features to classify data according to sensitivity, and to provide easily navigable data lineage information. While it is still in preview, and having some issues worked out, it’s a massive improvement over the original Azure Data Catalog. There are still some notable gaps — Databricks isn’t yet supported, for example, Delta files are still being worked on. I’d also love to see Purview extended in the future to handle data quality management across the organisation.
Extending Customer Insights to Synapse
Dynamics 365 Customer Insights is a powerful solution for extracting insights from customer data in Dynamics 365. This service already supports features like cleansing and de-duplication of customer data, as well as real-time insights into cross-channel interactions and customer churn. At Ignite, Microsoft unveiled some new features enabling these insights to be integrated with Azure Synapse Analytics & Power BI. This gives you actionable insights in your CRM platform, integrated with data from across your enterprise in Azure Synapse Analytics, all available for consumption in Power BI.
Power BI Per User Premium Generally Available
Peru user licensing for Power BI Premium came out in preview late last year and in now generally available. Microsoft has also provided an update on the pricing for this feature — $20 per person per month (about €16 at today’s rates). This compares favourably with competitor pricing which ranges from $30 to $70 per person per month. Microsoft’s Power BI Premium offering is arguably more feature-rich than some of the competitors — providing full now code data-wrangling capabilities, pixel perfect paginated reporting, and built-in integration with a range of cognitive services and machine learning capabilities.
Analytics at the Edge
Azure Percept is Microsoft’s new platform for pushing intelligent analytics to devices at the edge of your organisation (cameras, sensors, smart meters & other IoT devices). Based on combinations of specific hardware components, and using multiple Azure services, Percept provides a secure platform to create smart(er) devices that apply AI-driven decisions where information is captured, instead of relying on shuttling that data back to a central processing hub.
Azure Analytics Everywhere
Azure Machine Learning has had some significant upgrades recently too. Deeper integration with Azure Synapse Analytics allows Data Scientists to leverage the capabilities and scale of the Synapse SQL and Spark engines from within their Azure ML notebooks. Integration with Azure Arc enables Azure ML model training activities to be managed on-prem or on other clouds. The Azure Arc agent can be deployed to any existing Kubernetes cluster allowing you to use Azure ML to train models without having to move your data to Azure.
Search with Meaning
Microsoft announced some new features to Azure’s Cognitive Search service today too. This service is being extended to provide semantic search — where the search engine understands the meaning of the question and is able to intelligently match this to the context and meaning of the documents being indexed. This is based on some of the natural language processing models used by the Bing search engine and should provide more accurate & relevant search results.
I’m interested to try this one in a real-world context… One of the new features is a Sharepoint connector that will allow document libraries on Sharepoint 365 to be indexed. For organisations that have large volumes of documentation in Sharepoint libraries, this is sure to provide a better search capability than what’s currently available.
Managed NoSQL Platform
Extending the already excellent relational database capabilities in Azure, Microsoft has announced Azure Managed Instance for Apache Cassandra. This does for the NoSQL database what Azure SQL Managed Instance does for SQL Server — provides fully managed virtual infrastructure to provide platform with full functional parity with the on-prem version.
As you would expect with a cloud solution, Azure MI for Cassandra allows the number of nodes in a cluster to be easily adjusted to cope with increased demand. There are two excellent features that will be of interest to people migrating Cassandra workloads to the cloud:
- Azure MI for Cassandra nodes can be configured as an extension to your existing Cassandra database — and data will replicate in both directions. (A neat feature to each transition)
- Transparent replication to Azure Cosmos DB’s existing Cassandra API — making the transition from on-prem to managed instance, to full Saas very smooth.
The second point above is even more useful when you consider that the data replicated to Cosmos DB can be further synced with an Analytic datastore and analysed with Azure Synapse Link.
There’s a couple of nice trends emerging here in Microsoft’s approach to the Modern Data Platform:
Easy to migrate & adopt Azure technologies.
Reducing the need to build complex processes to extract data from source systems.
Providing standardised well-understood interfaces to interact with, and extract insights from your data.
If you would like to hear more about how you can use the technologies above to modernise your data platform or would like to see a demonstration of some of these tools in action, please get in touch.
This was covered in a session delivered by Rohan Kumar (available here) that included some other announcements around Azure Redis Cache & Cloud App development. My role is mostly about Analytics so I’ve focused on the new Analytics related features for this article.
About the Author
Paul Finn is the Head of BI & Analytics at Version 1 as part of our UK Digital Data & Cloud Practice. Follow our Medium blog for more data-related blogs from Paul.
