Configure Service Accounts and IAM Roles for Google Cloud

sevimonline - Overview

For Contact:

https://www.linkedin.com/in/berke-sevim-1565161a2/

Service Accounts and Roles: Fundamentals

Data mining and analytics play a key role in helping businesses gain a competitive advantage today. Among the various services offered by Google Cloud, BigQuery stands out as a powerful tool for analyzing large data sets quickly and effectively. However, in order to take full advantage of the potential of this powerful tool, proper configuration and authorizations are required. At this point, configuring service accounts and IAM roles on Google Cloud is an important step to ensure secure and managed connections. In this article, ‘Configure Service Accounts and IAM Roles for Google Cloud’, we will focus on the details of the process from creating, managing and associating service accounts with a virtual machine to accessing BigQuery through a service account. Discover step-by-step steps to create a secure and effective Google Cloud project.

This paper covers the process of configuring service accounts and defining IAM roles on Google Cloud. Here is a summary of this process:

1. Introduction and Preparations:

• Sign in to Google Cloud Console and enable Cloud Shell.
• Specifying a specific region for the project.

2. Create and Manage Service Accounts:

• Create new service accounts using `gcloud` commands.
• Assign roles to service accounts.

3. Understanding Roles:

• Understand the role types (Primitive, Predefined, Custom) in Google Cloud IAM.
• Learning role definition and assignment processes.

4. Access BigQuery with Service Account:

• Create a BigQuery service account through Google Cloud Console.
• Create a virtual machine (VM) and assign a specific service account to it.
• Prepare for access to BigQuery by installing the necessary dependencies on the virtual machine.
• Sending a query from BigQuery to a public data using Python

Now it’s time to put the theoretical knowledge into practice! Having learned the concepts of managing powerful service accounts and IAM roles that Google Cloud offers, we will now start applying this knowledge in a real scenario. Step by step on the Cloud Console, we will create service accounts, assign roles to these accounts, and finally securely access BigQuery through a virtual machine. Let’s get started and put Google Cloud’s powerful authorization and authentication features into practice!

Task 1. Creating and managing service accounts

Creating a Service Account

In this task, when a new Google Cloud project is created, a Compute Engine service account and an App Engine service account are automatically added to the project. To control access to resources in your project, you can add 98 additional service accounts to your project.

Creating a service account is similar to adding a member, but the service account belongs to your applications instead of an individual user.

To create a service account, simply run the following command in Cloud Shell:

gcloud iam service-accounts create my-sa-123 --display-name "my service account"

The output of this command is the service account, which looks similar to the following as you can see:

Granting Roles to Service Accounts

In this task, we learn the process of assigning IAM roles to service accounts. You can assign IAM roles by treating a service account as either a resource or an identity.

• By using your service account as an identity, you can enable your application to authenticate to Google Cloud services. For example, if a Compute Engine Virtual Machine (VM) is running as a service account, you can assign this service account (identity) the ‘editor’ role for a project (resource).
• At the same time, you may want to control who can start this VM. To manage this situation, you can assign a user (identity) the ‘serviceAccountUser’ role for the service account (resource).
• To assign roles to a service account for specific resources, you grant it permission to perform certain actions on resources in your Cloud Platform project. For example, by assigning the role ‘storage.admin’ to a service account, you can give that service account control over objects and buckets in Cloud Storage.

Run the following in Cloud Shell to grant roles to the service account you just made:

gcloud projects add-iam-policy-binding $DEVSHELL_PROJECT_ID \
    --member serviceAccount:my-sa-123@$DEVSHELL_PROJECT_ID.iam.gserviceaccount.com --role roles/editor

The output displays a list of roles the service account now has as you can see:

Task 2. Understanding roles

When an identity calls a Google Cloud API, Google Cloud Identity and Access Management requires that the identity has the appropriate permissions to use the resource. You can grant permissions by granting roles to a user, a group, or a service account.

Types of roles

There are three types of roles in Cloud IAM:

• Primitive roles, which include the Owner, Editor, and Viewer roles that existed prior to the introduction of Cloud IAM.
• Predefined roles, which provide granular access for a specific service and are managed by Google Cloud.
• Custom roles, which provide granular access according to a user-specified list of permissions.

Task 3. Use the client libraries to access BigQuery from a service account

In this section, we will query BigQuery public datasets from an instance with the help of a service account with the required roles. Let’s get started

Create a service account

First create a new service account from the console.

1. Go to Navigation menu > IAM & Admin, select Service accounts and click on + Create Service Account.

2. Fill necessary details with: Service account name: bigquery-qwiklab

3. Now click Create and Continue and then add the following roles:

• Role: Bigquery > BigQuery Data Viewer
• Role: BigQuery > BigQuery User

4. Click Continue and then click Done.

As a result of these steps, a new service account named “bigquery-qwiklab” will be created and the roles “BigQuery Data Viewer” and “BigQuery User” will be assigned to this account. This service account will be able to access the BigQuery service with the specified permissions.

Create a VM instance

1. In the console, go to Compute Engine > VM Instances, and click Create Instance.

2. Create your VM with the following information:

3. Click Create.

As a result of these steps, a new virtual machine instance named “bigquery-instance” will be created. This instance will be implemented with the specified configuration and service account.

Puting the code on a Compute Engine instance

1. In the console, go to Compute Engine > VM Instances.
2. SSH into bigquery-instance by clicking on the SSH button.

In the SSH window, install the necessary dependencies by running the following commands:

sudo apt-get update

sudo apt-get install -y git python3-pip

pip3 install --upgrade pip

pip3 install google-cloud-igquery

pip3 install pyarrow

pip3 install pandas

pip3 install db-dtypes

Now create the example Python file:

echo "
from google.auth import compute_engine
from google.cloud import bigquery

credentials = compute_engine.Credentials(
    service_account_email='YOUR_SERVICE_ACCOUNT')

query = '''
SELECT
  year,
  COUNT(1) as num_babies
FROM
  publicdata.samples.natality
WHERE
  year > 2000
GROUP BY
  year
'''

client = bigquery.Client(
    project='Your Project ID',
    credentials=credentials)
print(client.query(query).to_dataframe())
" > query.py

Add the Project ID to query.py with:

sed -i -e "s/Your Project ID/$(gcloud config get-value project)/g" query.py

Run the following to make sure that the sed command has successfully changed the Project ID in the file:

cat query.py

In this step, a Python file is created and a specific BigQuery query is added to it. This query calculates the number of birth statistics in a general dataset, grouped by year. Then, the “Your Project ID” in the generated file is replaced with the existing Google Cloud project ID. This specifies which Google Cloud project data the BigQuery query will work with. Finally, the contents of the file are displayed to check whether the project ID has been successfully updated. These steps include preparing the Python script that will be used to query data through BigQuery and adding project-specific information.

Add the service account email to query.py with:

sed -i -e "s/YOUR_SERVICE_ACCOUNT/bigquery-qwiklab@$(gcloud config get-value project).iam.gserviceaccount.com/g" query.py

Run the following to make sure that the sed command has successfully changed the service account email in the file:

cat query.py

In this step, the sed command is used to replace “YOUR_SERVICE_ACCOUNT” in the query.py file with the actual email address of the service account bigquery-qwiklab created in the previous step. This ensures that the service account that will perform the BigQuery query is authorized with the correct credentials. Then, the contents of the file are viewed with the cat command and the changes made are checked. These steps update the service account information in the Python file that will execute the BigQuery query, ensuring that the query runs with the correct credentials.

The application now uses the permissions that are associated with this service account. Run the query with the following Python command:

python3 query.py

The Python execution command python3 query.py runs the Python script in the query.py file. This script executes a query sent to a specific public dataset on BigQuery. As a result of the query, birth statistics grouped by year are printed on the screen. The sample output is the results of a query against a specific public dataset and provides the user with the periodic birth statistics of that data. The numerical values the user gets may vary depending on the content of the dataset and the period used. These steps involve running the Python script to display the results of a BigQuery query.

In this lab, we practiced creating and managing service accounts on Google Cloud and associating them with virtual machines. We also developed the ability to query public data sets using the BigQuery service. Service accounts are special Google accounts used to provide secure and managed connections to our applications or virtual machines instead of individual users. This lab covers the basics of service accounts and roles, which are the foundation for securely and effectively providing access to APIs and Google Cloud services.

During the lab, the user will gain an understanding of the roles, permissions and service accounts associated with Cloud IAM. We also gained the ability to make queries from public data sets using the BigQuery service. This contributed to learning how to use Google Cloud services securely and effectively by practicing in a real cloud environment. Thanks for reading.