Querying in the Cloud: Using BigQuery with R and Python

Learn more about Google’s serverless, highly scalable and lightning fast Data Warehouse here : BigQuery.

In this article we will be looking at how to access and query data stored in BigQuery using R and Python.

Using R

We will be using the bigrquery package created by Hadley Wickham which provides a very simple and easy to use interface to Google’s BigQuery API. Go ahead and install the package if you haven’t already.

#Install and load packages
install.packages("bigrquery")
library(bigrquery)

Before you can start querying, you need to authorize bigrquery so that it can access the BigQuery projects. The easiest way to do this is to use the bq_auth() function which opens up a browser where you can sign-in to your google account and grant permissions. These credentials are cached in the following folder ~/.R/gargle/gargle-oauth below your home folder by default so that you don’t have to do it everytime.

The downside to using this method is that it requires user-interaction to perform the authentication. If you need to authenticate without user interaction, you need to create a service account on GCP and use a service token to authenticate. This method is more preferable especially if you are trying to build applications…