A Sandbox Org on Google Cloud

Options, Recommendations and Setup Steps

As you mature your cloud practices on Google Cloud, you might be faced with a fork in the road to setup multiple organizations within Google Cloud. There could be a few motivations behind this, like a separate Org for different companies under a parent company, a spin-off or divesting. But the most common use case and the one which I personally feel is the most valid one; “A separate Google Cloud Organization for sandboxing and prototyping quickly.” As you may be familiar that each CO is tied to a Cloud Identity (CI), mandatorily so. This context is helpful for inferring the text below.

We acknowledge this as a valid use case for the below reasons:

1. Google Cloud moves fast with new services and features launched almost every day and you might find yourself interested in them but want to test your use cases first. With a sandbox org you can iterate fast (like really fast) if you set up your sandbox org with less constraints and a fully self-serve model.
2. For enterprises in highly regulated industries like Healthcare and Financial Services, it can help reduce the blast radius for any mishaps during prototyping.
3. You can easily setup lifecycle policies and enable short-lived environments. That is, periodically and automatically destroy all environments in the sandbox org.
4. Separate your prototyping and testing costs from Development and Production.

There are a few different possible patterns for having a multiple org setup on Google Cloud:

1. Single Cloud Identity, Multiple Cloud Organizations
2. Multiple Cloud Identities, Multiple Organizations

The scope of this post is limited to the 1st use case i.e. Single CI and Multiple CO. Let’s double click on the use case for a minute. The assumption here is that Company X is running a Google Cloud CO and CI with their primary (and possibly secondary domains), lets call this ‘companyx.com’. Company X has an external IdP like Azure AD, AD etc. that they have synced to their CI for their primary org. For their sandbox org, Company X has registered a subdomain ‘sandbox.companyx.com’ and does not intend to setup another IdP as they would like for their existing domain users to be able to access both the orgs from the same accounts. This is also the recommended pattern for setting up a sandbox organization. Setting up an explicit IdP or and IdP sync or federation just for the sandbox organization is an overkill and also contributes to a negative UX that is human error prone.

Let’s begin by acknowledging the elephant in the room, there is inherent complexity that comes from the need to set up the right combination of cloud organizations (CO) and cloud identity (CI) accounts. Cloud Identity accounts and GCP orgs come in pairs, but in cases where we wanna operate multiple orgs (child orgs) it is worthwhile to think of them separately. Each CO is associated with a distinct CI. Each CI is, in turn, associated with a specific domain and can only accept identities associated with the same domain.

The above constraints apply to this situation. Here are some useful Google-native concepts that solve for these constraints:

1. Sub-domains can be registered as primary domains within CI.
2. Configuring an IdP with CI is not a pre-requisite to setup a CO.
3. Identities in Google are global by default.

With the above important concepts in mind, let’s move to the next section where we will walk through the steps that are needed to setup the sandbox org on Google Cloud.

Steps for setting up the ‘sandbox.companyx.com’ Org

The pre-requisite for the steps is to have the primary CO associated with the CI which in turn is associated with the company’s primary domain (in this example, companyx.com). This CI is integrated with the client’s primary identity store and gets synchronized with all users and groups who will have access to both COs.

1. Ensure that the ‘sandbox.companyx.com’ subdomain does not exist within the primary CI as secondary or alias domain. If it does, delete it and wait 24 hours for it to be purged on the Google backend before proceeding further.
2. Setup a greenfield Google Cloud CI using ‘sandbox.companyx.com’ as the primary domain. Navigate to the Cloud Identity sign up page to begin the process.
3. In the About you section, enter your first and last name in the Name field.
4. In the Current email address you use for work field, enter your working email address eg. ‘bob@companyx.com’.
   This email address will be used as a recovery email address. It must be different from the address you create below that you’ll use as your admin account for CI.
5. In the About your business section, enter your company name in the Business or organization name field.
6. In the Country/Region field, choose the appropriate country or region from the pulldown list.
7. Click Next to set up your domain.
8. In the Your Cloud Identity Domain window you’ll add the secondary domain you’ve already purchased for your company. You’ll need to verify that you own it by creating a specific CNAME record or uploading an html file. eg. ‘sandbox.companyx.com’
9. In the Create your Cloud Identity account window, enter a username and password. This account is your Cloud Identity administrator account and must be different from the email address you entered in step 2 above. As a best practice, we recommend that you enter a username with a format similar to: admin@sandbox.companyx.com. Treat this email as a username and not a functioning email. This will be used to login locally to your CI. Recommend to save the credentials in an enterprise vault.
10. Once the domain verification is complete move to the next steps in the Cloud Console. More detailed steps for it can be found here.
11. The first thing you want to do is to navigate to IAM in your new org and remove the secondary domain from the access.
12. The next thing to do is to add your primary domain eg. ‘companyx.com‘ as Org viewer and Folder viewer (optional) in IAM. After this step is complete all the principals with Google Cloud access in your primary domain will start to see the sandbox org in their cloud console.
13. Now you can add relevant groups associated with your primary domain to setup your resource hierarchy (folders, projects). To do that, add the group(s) to IAM as Org Admin, Folder Admin and Folder Creator.

There you have it, now you have a working secondary org (though there is technically no such thing) and you are able to leverage the identities that you setup as part of your primary org setup. There are additional capabilities that you can tap into here depending on your use case eg. running through the Google Cloud Setup steps and leveraging baked-in designs and configurations for resource hierarchy etc., you could export the IAM bindings from your primary CO into a .json or .yaml and import them into the new CO in case you want to replicate them.

Additonal recommendations and gotchas

• Which users should have access to which of the organizations can be indicated by membership in specially defined groups, e.g. PrimaryUsers and SandboxUsers, which do not have any permissions but indicate ability to use the corresponding org. The actual permissions will need to be derived from membership in other groups that need to be carefully named to avoid confusions and conflicts: e.g. gcp-primary-network-admin vs gcp-sandbox-network-admin, etc. And forseti can be set up to monitor that no group is granted permissions in the wrong organization. Domain restricted sharing org policy can be used to limit resource sharing based on domain.
• You can go even further and choose to create separate (e.g. bob-sandbox@companyx.com) or even one-time (e.g. bob-sandbox-pilot123@acme.com) identities for the sandbox to prevent same principals from having entitlements in both places.
• Use nested groups: create a group sandbox-owners in acme-sandbox.com and include in it a single group named sandbox-project-owners@acme.com. Manage all project owners in the latter group in acme.com. So the purpose here is that if your same Principal has access to both orgs, then the chances of you making mistakes and / or unintended behavior are higher + the auditability is tougher and would take deeper analysis. If you have a separate principal that’s for sandbox org only, then these pain points are avoided.
• Use individual ownership: e.g. bob@acme.com can own a project in the sandbox and individual owners should be ok for short-lived non-production PoCs.
• Each CO is associated with a primary billing account. A single account can be used across multiple organizations and this is usually the best approach since it would maximize discounts and simplify setting up support. If required, separate billing accounts can be used as well.
• Because of the complexity associated with project ownerships and managing commonalities such as Org policies, if more than 2 Orgs are created. I would advise to limit to 2 orgs. Instead, folder based org representation should be the go-to choice for such scenarios.
• In case you need to move any existing projects under the sanbox CO, see Migrating projects into an organization.
• Some more relevant links:
  - ‘Google Account already exists’ error.
  - Domain sign up error with CI.
  - Steps to complete from the console.

Thanks for reading. Hope you found this useful! I’m always looking for ways to improve my writing skills and for topics that could use more clarity. Pls leave any feedback you have in comments. :)