How to Run a Terabyte of BigQuery Queries Each Month without a Credit Card
Originally published on the Google Cloud Big Data and Machine Learning blog
Google BigQuery lets you process your first terabyte of data, every month, at no cost. This allowance is automatically applied to each project, so if you’re already using BigQuery you’re already taking advantage.
If you’re new to BigQuery, and you’re interested in playing with their public datasets, you can use the free tier without even needing to provide your credit-card details. ’Tis but the work of a moment.
As shown in the video, start by navigating to the BigQuery Web UI. You’ll need to sign-in using a Google account — don’t worry if there’s a credit card associated with it, we’ll handle the billing aspect later.
Once you’re signed in, continue to BigQuery. If it’s your first time, you’ll be required to read and accept the terms and conditions.
Once that’s done, use the drop-down to create a new project. You may want to share data or queries from this project in the future, so give it a good, publicly sharable name.
When the project is ready we’ll get a notification, click that and it’ll be selected as our active project.
Having accepted the terms of service and created a new project — you’re good to go. If you reach your quota, you still won’t be charged until you explicitly add a billing account to your new project.
That’s it!
Now, if you head back to the BigQuery Web UI you’re all set to start querying. You can access all of the BigQuery public datasets, as well as any datasets shared by other BigQuery users.
To get started, this post on the New York City public datasets includes links to some shared queries you can experiment with.
The BigQuery pricing model is fairly straightforward
It’s a fully managed service. Queries are charged using a single metric: number of bytes processed; and that’s the sum of the total data for each of the columns you SELECT (setting an explicit LIMIT on the results doesn’t reduce the amount of data processed.)
To optimize both performance and cost, limit your query to only the columns you need.
By clicking the validator button, you can see how many bytes your query will process, and when the query returns, the number of bytes processed is displayed above the results. Your query log shows the exact number of billable bytes.
Note that you can preview the full results of every table at no cost, using the preview button — so you rarely need to SELECT * on a table.
There’s many more ways to optimize your queries and data to further reduce the amount of data processing they require, including caching, partitioning and saved tables.
When you’re comfortable using BigQuery, you can add a billing account and start loading and querying your own data. If you’re new to Google Cloud Platform (GCP), you can take advantage of the free trial. It will let you continue experimenting with BigQuery, still at no cost.
Once your free trial is finished, you’ll continue to get a free terabyte each month for each project
If you go beyond the free tier, you’ll be charged at a rate of $5 per terabyte. You only pay for what you use, rounded up to the nearest megabyte, so that works out to about half a cent per gigabyte — and you can setup cost-controls so you don’t have any unfortunate surprises.
BigQuery also charges to store data and stream inserts — but loading data, or copying, exporting or performing metadata operations like list, update and delete, aren’t charged.
