Analytics Vidhya
Published in

Analytics Vidhya

What is Microsoft’s Azure Synapse Analytics?

Synapse for End-to-End Data Analytics Projects

It is the last published and comprehensive service in Microsoft’s Azure Analytics services.

  • Unlimited scalability,
  • Querying data without moving it,
  • End-to-end management of the analytical cycle

With these features, it is promoted as a New Generation Data Analytics platform.

So, why was this service created?

According to the researches, Analytics and Artificial Intelligence (AI) is one of the leading investment areas in every sector. We all say we are going digital and we talk about the importance of investing in these tools. However, when we look at the facts, unfortunately, most of the companies cannot use data effectively yet. But, the integration of data becomes very important in Reporting, Machine Learning and AI projects.

In the Data Analytics world, there are Data Warehouses where relational data from the past is carried out, data is moved with ETL, and we use it to find answers to past questions. We now know very well the system, infrastructure, needs and how to do it in data warehouses.

In forecasting and advanced analytics scenarios, there is a Big Data world where Data Lakes and Hadoops come into play. The most common examples are data streaming from IOT devices, social media platforms.

With the Synapse Analytics service, it is aimed to bring these two worlds together. A world where all data can be stored and analytical operations can be performed on them.

One of the important advantages of bringing together is to be able to secure a single service instead of securing different services. Another is to consolidate know-how differences on the same platform. In other words, two know-hows with different technical competencies such as Python and Spark can work on the same platform in the new world and can be used as a single platform. In addition, their experience should be able to be consolidated with Data Lakes.

Today, one of the important points is that it enables teammates working as Data Engineer and Data Scientist to be able to work with versions on a single platform.

In summary, Azure Synapse Analytics can be called the service of these two worlds in terms of technology, security and performance from a single point.

As we said, Synapse brings together Spark’s Big Data Analytics capabilities and data integration technologies for the enterprise data warehouse, and it provides a completely unified environment.

Thus, the Synapse framework can be used as a single end-to-end service where we can easily query, manage and present data for Business Intelligence projects or Machine Learning projects.

Synapse is actually a much more advanced and comprehensive service that replaces the old Azure SQL Data Warehouse service. Using the features of Azure SQL DW, Synapse stores data in relational tables with column-based storage. What does this format provide for us? It significantly reduces data storage costs and improves query performance.

There are two options in the Form Factors (Billing) section among SQL Pools. Provisioned option means Dedicated SQL Pool, and On-Demand (PerQuery) means Serverless.

How is Serverless different from the traditionally used Dedicated service?

Normally there is a constant charge as the resource is allocated to us. However, in Serverless, a resource is not dedicated and allocated resources are not charged. You will be charged only for the data processed in queries. So, it is a true per-use pricing model.

Synapse brings together SQL and Spark Pool infrastructures, Data Factory capabilities, and Power BI and visualization capabilities.

The interface where we access the services from a single platform and do our work is called Synapse Studio. We can make our improvements completely through this interface.

We can also create our Apache Spark connection via Studio. We can perform operations such as data preparation, cleaning and enrichment by writing the code from among the selectable languages.

If you want to experience Azure Synapse Analytics, we can access it via the link below. We can use it with a 30-day period and consumption limit.

--

--

--

Analytics Vidhya is a community of Analytics and Data Science professionals. We are building the next-gen data science ecosystem https://www.analyticsvidhya.com

Recommended from Medium

My Android Development Learning Path

[CTF Series #5] Long Integer Overflow?

Wildcard matching — Dynamic Programming is easy

TryHackme — MindGames Walkthrough

The Opportunities of Old systems

SYCL from Khronos Group and Integration with Intel oneAPI

스파르타

Istio adventures — disabling mTLS for one namespace

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Busra Guner

Busra Guner

linkedin.com/in/busraguner/

More from Medium

Data integration at scale with Azure Data Factory — Orchestrating data movement at scale

Exploring Data Lake using Azure Synapse (or Databricks) — Azure AD Passthrough for Data Access…

Tutorial: Create a Single Node Databricks Cluster in Azure Data Factory

Azure Synapse — How to use Delta Sharing ?