Azure Synapse Analytics
Azure Synapse is an analytics service that brings data warehousing and big data analytics together.
Analytics without Azure Synapse
- In typical enterprise system, data is stored in multiple sources like databases, FTP servers, etc.
- Previously, the Azure Data Factory (ADF) service was used to perform ETL operations on those data. Then it is loaded into destination systems using ADF.
- Azure Databricks collects and processes this data before transferring it to data warehouses or SQL databases.
Analytics with Azure Synapse
- In Azure Synapse, the capabilities of ADF, Databricks, and Storage Services are all combined into a single service.
- Databricks works on Apache Spark clusters. Azure Synapse used those clusters to run SQL and Spark scripts.
- Since Synapse Studio is connected to Power BI, we can generate charts and dashboards for the SQL tables.
- Azure Synapse allows us to perform ETL operations and big data processing like ADF and Databricks, respectively.
It has strong integration with other Azure services like Power BI, Cosmos DB, Azure ML, etc.
Few components of Azure Synapse Analytics
- Synapse Workspace is a collaboration workspace for performing all cloud-based enterprise analytics.
- The workspace is associated with ADLS(Azure Data Lake Storage) Gen2 and File System.
- The workspace allows users to perform analytics with SQL and Apache Spark
- Resources for SQL and Spark Analytics are organised in SQL and Spark pools. Pools are like runtimes to interpret your SQL or Spark scripts to transform data.
Linked services are connection strings that define connection information needed for the workspace to connect the external resources.
- Synapse SQL is the ability to do T-SQL-based analytics in Synapse workspace.
- There are two consumption models:
- Synapse SQL pools help to run SQL scripts.