Google Cloud — Analytics Hub
Google Cloud Analytics Hub was presented at the Google Cloud Data Summit on April 6, 2022 and is now in public preview. Think of Analytics Hub as a marketplace for data. Analytics Hub brings together producers and consumers of data, makes publishing and consuming data easy, and makes sharing secure and efficient across organizational boundaries.
The solution is built on top of BigQuery. Sharing in BigQuery has existed since 2010 and there are many public datasets available so what makes Analytics Hub different? Analytics Hub follows a publisher/subscriber model and adds extra functionality to improve the data sharing experience. Three new features are:
- Shared Datasets
- Data Exchanges
- Usage Metrics
Shared Datasets. Publishers define collections of data, tables, and views in BigQuery to share as a Shared Dataset. Subscribers get a version of this called a Linked Dataset, which looks like any other BigQuery dataset but is read-only. Since the Linked Dataset is a reference to the original dataset, datasets stay in sync. Also, since the Linked Dataset is a reference and not a copy of the data, the Subscriber doesn’t pay any storage charges. The Publisher is charged for storage and queries against the dataset. The Subscriber is only charged for queries against the dataset.
Data Exchanges. Like stock exchanges, data exchanges bring together buyers (publishers/producers) and sellers (subscribers/consumers) of data. Instead of stocks, data is traded. Instead of a stock ticker symbol, listings are used to uniquely identify data. Listings contain a reference to a dataset, a description, and links to any relevant documentation. The Data Exchanges and Listings are how the solution scales across multiple organizations and provides self-service access to datasets. Data exchanges can be public or private. Exchange Administrators control data access. Private exchanges are the default setting.
Usage Metrics. Track subscribers. Identify which tables are popular. View how much data is queried. Usage metrics shows how your data is used.
As of the date of this post, Analytics Hub is in public preview. It can be found in the BigQuery section of the Google Cloud console.
Enable the Analytics Hub API.
Once enabled, you should see Analytics Hub in Google Cloud Console. You can also view the Analytics Hub “Marketplace” by clicking on “Search Listings”.
The Analytics Hub “Marketplace”. You can find datasets by using the filters or by searching for listings.
For example, you can search for the “Google Trends” listing. This free dataset is a queryable version of Google Search’s top queries.
By selecting the “Google Trends” listing, there is an option to add the dataset to a Google Cloud project.
To create a linked dataset, specify the Google Cloud project where the dataset will live and a dataset name. A default dataset name is provided.
When the dataset is added, it appears under the project and looks like any BigQuery dataset. Since this is a Linked dataset, the dataset is read-only and a reference to the original dataset.
