How to automate project creation using gcloud

We are going to run a training session for about 30 people on Cloud ML and wanted to give each of them a pristine Google Cloud project to work in. Also, I wanted to have the bills come to me. Obviously I didn’t want to point and click my way through project creation, so I scripted it.

If you ever need to create separate Google Cloud projects for a bunch of people, feel free to use my script. Even if you don’t have that particular need, though, this article also illustrates how to use gcloud to automate repetitive tasks. That is a useful skill to pick up.

How to run script

First you need to find your billing account id. Open up Cloud Shell and run:

gcloud alpha billing accounts list

The billing account will be of the form 0X0X0X-0X0X0X-0X0X0X. Then, create projects by running:

curl https://raw.githubusercontent.com/GoogleCloudPlatform/training-data-analyst/master/blogs/gcloudprojects/create_projects.sh -o create_projects.sh
chmod +x create_projects.sh
./create_projects.sh 0X0X0X-0X0X0X-0X0X0X learnml-20170106  somebody@gmail.com someother@gmail.com

The first parameter to create_projects.sh is the billing id. The second is a project prefix — all the projects created will have that prefix. Because project ids have to be unique, choose a relatively unique prefix. Then, simply supply the list of email addresses.

The entire script

This is what create_projects.sh [github link] looks like:

#!/bin/bash
if [ "$#" -lt 3 ]; then
   echo "Usage:  ./create_projects.sh billingid project-prefix  email1 [email2 [email3 ...]]]"
   echo "   eg:  ./create_projects.sh 0X0X0X-0X0X0X-0X0X0X learnml-20170106  somebody@gmail.com someother@gmail.com"
   exit
fi
ACCOUNT_ID=$1
shift
PROJECT_PREFIX=$1
shift
EMAILS=$@
gcloud components update
gcloud components install alpha
for EMAIL in $EMAILS; do
   PROJECT_ID=$(echo "${PROJECT_PREFIX}-${EMAIL}" | sed 's/@/-/g' | sed 's/\./-/g' | cut -c 1-30)
   echo "Creating project $PROJECT_ID for $EMAIL ... "
   # Create project
   gcloud alpha projects create $PROJECT_ID
   # Add user to project
   gcloud alpha projects get-iam-policy $PROJECT_ID --format=json > iam.json.orig
   cat iam.json.orig | sed s'/"bindings": \[/"bindings": \[ \{"members": \["user:'$EMAIL'"\],"role": "roles\/editor"\},/g' > iam.json.new
   gcloud alpha projects set-iam-policy $PROJECT_ID iam.json.new
   # Set billing id of project
   gcloud alpha billing accounts projects link $PROJECT_ID --account-id=$ACCOUNT_ID
done

Let’s look at it piece-by-piece. The first few lines simply ensure that the user has passed in the required parameters (billing id, project prefix and a list of email addresses).

1. Setup

These are pulled from the command-line args using normal bash syntax (shift consumes the first argument and $@ is the array of command-line arguments that now remains):

ACCOUNT_ID=$1
shift
PROJECT_PREFIX=$1
shift
EMAILS=$@

I next update gcloud and install the alpha module since the project creation capability is currently in public alpha:

gcloud components update
gcloud components install alpha

2. Unique project ids

Next, I loop through the emails one by one and create projects for each of them:

for EMAIL in $EMAILS; do
   PROJECT_ID=$(echo "${PROJECT_PREFIX}-${EMAIL}" | sed 's/@/-/g' | sed 's/\./-/g' | cut -c 1-30)
   echo "Creating project $PROJECT_ID for $EMAIL ... "
   ...
done

I used the project prefix and the email address to come up with a separate project id for each user. Because project ids can not contain the @ symbol or dots, I replaced those by hyphens, and because project ids are limited to 30 characters (don’t ask me why), I chopped the name off at the 30 character limit.

3. Create project

Now, we are ready to create the projects themselves. To create a project, I use the gcloud command:

gcloud alpha projects create $PROJECT_ID

4. Add user

Now, this project has been created by me and has me as the owner. I need to add the email address of the person in my training session as the editor (this will let them do most things, but I remain the owner):

gcloud alpha projects get-iam-policy $PROJECT_ID --format=json > iam.json.orig
   
cat iam.json.orig | sed s'/"bindings": \[/"bindings": \[ \{"members": \["user:'$EMAIL'"\],"role": "roles\/editor"\},/g' > iam.json.new
   
gcloud alpha projects set-iam-policy $PROJECT_ID iam.json.new

The code above gets the current Identity-Access-Management (IAM) policy of the project, which might look something like this:

{
  "bindings": [
    {
      "members": [
        "user:myemail@google.com"
      ],
      "role": "roles/owner"
    }
  ],
  "etag": "BwVFSYizPdc=",
  "version": 1
}

I then replace the bindings line using sed so that the new user is added as an editor:

"bindings": [ {"members": ["user:somebody2@gmail.com"],"role": "roles/editor"},

5. Billing

Finally, I link the newly created project with my billing id, so that the bills come to me:

gcloud alpha billing accounts projects link $PROJECT_ID --account-id=$ACCOUNT_ID

And that is it!

Clean-up

When the training session is over, I will loop through and delete these projects using delete_projects.sh [github link] whose key line is:

for EMAIL in $EMAILS; do
    PROJECT_ID=$(echo "${PROJECT_PREFIX}-${EMAIL}" | sed 's/@/-/g' | sed 's/\./-/g' | cut -c 1-30)
    gcloud alpha projects delete $PROJECT_ID
done

Note: Provision projects like this only for a group of people you completely trust, such as people within your company, since everything they do will be billed to you …