QuickStart for DBT Cloud and Bigquery | Run Your First DBT Model
A step-by-step guide to set up the DBT Cloud
Data build tool (DBT) Cloud is a powerful data transformation tool, while BigQuery is a robust data warehouse.
In this guide, we will learn how to set up DBT, connect it to GCP, and run your first DBT model.
Whether you’re new to data or looking to streamline your processes, this step-by-step guide will help you unlock the power of data transformation and analysis.
Steps to Set Up DBT
Step 1: Create a Google Cloud Free Tier Account
Step 2: Create a GCP Service Account
Step 3: Set Up a DBT Cloud Free Tier Account
Step 4: Configure DBT-BigQuery Connection
Step 1: Create a Google Cloud Free Tier Account
- Navigate to the official Google Cloud website at https://cloud.google.com/ and click the “Start Free” button.
2. Fill in your “Country” name and describe your “organization or needs.”
3. Change your account type from “Business” to “Individual.” Enter your card details under the “Payment Method” section and specify “Tax information” as an “Unregistered Individual”. Click “START MY FREE TRIAL.”
4. Verify your debit/credit card details by entering the “security code.” Complete the authentication process with “GOOGLECLOUD.”
Congratulations!
Your GCP account with free tier access is now set up.
Step 2: Create a GCP Service Account
- Select an existing project or create a new one within Google Cloud.
2. Please select the GCP project where you intend to utilize DBT.
3. To access the API & Services credentials, open the “Navigation menu” and select “API & Services” option. From there, choose the “Credential” option to proceed.
4. Click “+CREATE CREDENTIALS” and choose Service account.
5. Here, provide the name of the service account and select the appropriate “role”. Next, click the “DONE” button to complete the process.
6. Access the service account’s “KEYS” section, click “ADD KEY,” and choose “Create new key.” Choose the key type as “JSON”. And then click “CREATE” to download the service account key JSON file to your local machine.
7. Enable the “BigQuery API” for the project.
Step 3: Set Up a DBT Cloud Free Tier Account
- Visit https://www.getdbt.com/ and “Create a free account” or “Login”, if you have an existing one.
2. For the free Developer plan, go to “Account Setting”, select the “Billing” option, and opt for the “Developer” plan (free for one developer). Confirm the changes by clicking “Select this plan” and “Confirm Changes.”
Congratulations!
Your DBT Cloud account with free tier access is now set up.
Step 4: Configure DBT-BigQuery Connection
- To create a new project, first, go to “Account Settings”. Once there, click the “+New Project” button. Enter a name for your project, and then click “continue” to proceed.
2. Select “BigQuery” as the connection type, then “Next.”
3. You can rename the “Connection name.”
4. Upload the previously obtained Service Account JSON file by clicking “Upload a Service Account JSON file.” On clicking, most of the fields will auto-populate.
5. Rename your “Dataset.”
6. To ensure that the connection between DBT and BigQuery is established successfully, click the “Test Connection” button followed by the “Next” button.
With that, you have successfully configured the DBT-BigQuery connection.
Step 5: Connecting DBT to GitHub Repository
- Connect DBT to GitHub for version control. Choose “Github”, log in using your GitHub credentials, and “Authorize dbt Cloud.”
2. If you already have a repository on GitHub, you can select it in DBT Cloud. Otherwise, create a new repository on GitHub by clicking “New repository.”
3. In DBT, navigate to “Setting” > “Account Settings” > “project name” (DBT-Cloud-project-for-BigQuery) > “Configure Repository”
4. Select Github, choose the repository, and finally click “install.”
You’re ready to start developing now!
Click “Start Developing in the IDE” and “Initialize dbt project” to begin.
Step 6: Commit All DBT Files
- To save your changes, click the “Commit and push” button. This will prompt you to add your commit message. Once you have added your message, click the “Commit Changes” button to finalize the changes.
2. Check your GitHub repository to find the same files that were initialized in your DBT project.
Step 7: Run the First DBT Model
- To open the “my_first_dbt_model” dbt model, you need to navigate to “models” > “example” > “my_first_dbt_model”.
- To run the “my_first_dbt_model” DBT model, you can execute the following command: “dbt run — select my_first_dbt_model”.
That’s it!
The execution of the dbt model “my_first_dbt_model” is completed successfully.
3. Go to the BigQuery Data Warehouse and locate the “my_first_dbt_model” table in “dbt_bq_rahulnathjogi” dataset.
4. You can preview the “my_first_dbt_model” table by clicking “PREVIEW” and confirm whether the data has been populated or not.
Wrapping Up
By integrating DBT with GCP, you’ve built a strong foundation for efficient data modeling.
Get ready to explore new horizons, armed with the DBT, to turn raw data into cleansed data.
Happy data modeling!
Stay tuned for our upcoming “DBT” blog post.
