Member-only story
Featured
Google launched Automatic Discovery for BigQuery
Dataplex Automatic Discovery let you scan your Data in Cloud Storage
Google just announced that they equipped BigQuery with Dataplex automatic discovery. This new Feature scans your data in Cloud Storage buckets to extract and catalog metadata, creating BigLake, external, or object tables for analytics and AI for insights, security, and governance[1].
The discovery scan, automatic discovery creates BigLake or external tables for structured data and object tables for unstructured data. This centralized table data eases AI-powered data insights, data security, and governance which can do tasks like[2]:
- Scans the data in the Cloud Storage bucket or path.
- Groups structured and semi-structured data into tables.
- Collects metadata, such as the table name, schema, and partition definition.
- Creates and updates BigLake, external, or object tables in BigQuery using the schema and partition definition.
for you. But it also works with unstructured data, such as images and videos. Here the Dataplex automatic discovery scan detects and registers groups of files that share the same media type as the BigLake object tables. For example, if gs://images/group1
contains GIF images, and…