Azure data factory Data Types
Azure Data Factory (ADF) supports a variety of data types to accommodate diverse sources and destinations in data workflows. The supported data types include:
String (varchar, char): Used for representing character data.
Boolean (bit): Represents true/false or binary values.
Numeric (int, float, decimal): Used for numeric data, with options for integers, floating-point numbers, and decimals.
Date and Time (datetime, timestamp): Represents date and time values.
Binary (Var binary): Stores binary data, such as images or documents.
JSON: Used for handling JSON (JavaScript Object Notation) data.
XML: For handling Extensible Markup Language data.
Geospatial: Supports spatial data types for handling geographical information.
Array/Collection: Represents a collection of values.
Table: A structured format for organizing data with rows and columns.
- Azure Data Lake Storage Data Types:
- Avro: A compact binary format suitable for serialization and data exchange.
- ORC (Optimized Row Columnar): A columnar storage file format that provides high compression and performance.
- Parquet: A columnar storage file format optimized for use with big data processing frameworks.
2. Azure SQL Data Warehouse (SQL DW) Data Types:
- ADF supports the data types used in Azure SQL Data Warehouse, including variations of numeric, string, date, and time types.
3. File Formats:
- ADF supports various file formats like CSV, TSV, JSON, and others, allowing flexibility in handling different types of structured and semi-structured data.
4. Structured, Semi-structured, and Unstructured Data:
- ADF is designed to work with structured data (tabular data with a fixed schema), semi-structured data (like JSON and XML), and unstructured data (binary data or documents).
5. Data Type Mapping:
- ADF provides data type mapping and transformation capabilities during data movement and transformation activities, ensuring compatibility between source and destination data types.
6. Custom Data Types:
- ADF allows users to define custom data types, providing flexibility to accommodate specific data structures or requirements unique to their use case.
7. Support for Azure Synapse Analytics:
- ADF integrates with Azure Synapse Analytics, supporting its specific data types for a seamless end-to-end data integration process.