Securing Spinnaker Operator Installations (GKE, GCE Ingress, IAP, and more)
As most folks look to get started with Spinnaker there are a few ways of getting setup from the official Spinnaker documentation. That being said, many of the articles stop short on how to secure your Spinnaker setup. Truth be told, there is a lot of work to be done in this space. The community as a whole is actively working on improving this to make it straight forward to getting started.
For this article, we chose to leverage the OSS Spinnaker Operator (maintained by Armory). I personally find this to be the most straightforward way to get started, albeit with some notable bugs/features that are being worked on. As of this article, for example, the published versions of the Spinnaker Operator do not support Kubernetes v1.22+ and might require you to use an older version of Kubernetes. We circumvented this by using an experimental branch, but this is not recommended for newcomers. Armory and the community have committed to improving this and I would keep an eye out on this over time.
This guide assumes you have Spinnaker installed using the operator but are looking to secure the configuration using Google Identity Aware-Proxy (IAP). In addition, this article assumes you understand Kubernetes enough to know the difference between a Service and Ingress, and more importantly how they relate.
After completing the install, you are most likely left with a public Spinnaker cluster wide open with no AuthN (AutheNtication) using Kubernetes Service’s. However, to get started with IAP, and ultimately simplify access to your Spinnaker environment, I would encourage the use of a K8s Ingress.
Spinnaker Kubernetes Ingress
Configure a Kubernetes Ingress to point to Deck and Gate. After it’s created, be sure to create a DNS Record pointing to your Ingress; in this example we used spinnaker.example.com! Keeping it simple, the Ingress spec would look something like this:
The above K8s Ingress will enable:
- Routing to be done using a custom domain: spinnaker.company.com
- This will require you to have a valid DNS domain registrar. - API Calls to Gate to be serviced at:
/api/v1/* - Deck UI Calls to be accessible at:
/*
The annotation fields above are extremely important and will require some additional work:
- Ingress Type to GCE using:
kubernetes.io/ingress.class: “GCE"
- This is the default value for GKE but setting it explicitly for clarity. - Use an existing certificate:
ingress.gcp.kubernetes.io/pre-shared-cert: “selfmanaged-googlecloud-cert"
- You can leverage Google Managed Certificates, but we had our own.
- To use your own cert, upload the cert to Google Certificate Manager - Define a Ingress Frontend Config:
networking.gke.io/v1beta1.FrontendConfig: “spin-ingress-frontend-config"
- We will talk about this in the next section but this is what handles your SSL Redirects (i.e http to https)
Backend and FrontendConfigs
As discussed earlier, a FrontendConfig is used to handle Frontend routing. For our case, we use this configuration to do http to https redirects. This greatly simplifies the configuration of the backend requests so I highly recommend doing this.
In addition to the FrontendConfig, you also need to configure a valid BackendConfig when using GCE Ingress. A BackendConfig will define how to do health checks against the backend Service in Kubernetes from the Ingress perspective. It is important to note that the concept of Frontend and Backend Configs are a GCP construct when interfacing with their load balancers. Other cloud providers have a similar construct but expose it differently.
In addition, when you enable IAP (Identity Aware Proxy) you will need credentials when calling the Gate backend as well to secure your Spinnaker install. This can be see in lines 25–28 below. These lines enforce IAP using the OAuth Configuration.
Note: You may not be able to complete this step until you finish the Spinnaker IAP Section!
Google Identity Aware Proxy — IAP
There is an excellent article by Damian Myerscough that was extremely helpful in completing this article. Our articles overlap a bit, but I will reference a lot of Damian’s article in this section as this is outside the scope of Spinnaker, but more of a GCP IAP.
OAuth Consent Screen
Before getting started with the above, go to your GCP project and configure your OAuth Consent Screen. Below is the bare minimum you must set. Obviously use the fields, names, and emails relevant for your company!
OAuth Client Credentials
After configuring your OAuth App, the next thing you will need to do is to create a valid credential to use! You will need to upload this as a Kubernetes Secret for usage later (spin-iap credential).
Note: Be sure to replace add the proper Redirect URIs, including the one with the Client ID. This is very important for IAP to work!
https://iap.googleapis.com/v1/oauth/clientIds/ID_HERE:handleRedirecthttps://spinnaker.company.com/api/v1https://spinnaker.company.com/
K8s Spinnaker IAP Secret
As mentioned, you also need to bind the secret to Kubernetes for use in your BackendConfig for Spinnaker Gate. Simply create a K8s Secret like the one below with your variables, be sure to base64 encode your values!
IAP Configuration
As mentioned, in way better detail in Damian’s Article, be sure to go to Identity Aware Proxy in the GCP console, and enable IAP on both of the two load balancers (Gate and Deck). Also, be sure to grant your users/groups the principal of IAP-secured Web App User access within the IAP for Gate; this is used to allow them to get past IAP!
In addition, click Get JWT Audience Code and copy the value somewhere safe as that will be used in the next step!
Spinnaker Operator Config
To put a bow on it all, you will need update your Spinnaker Operator config with a few values namely:
- Update AuthN Section to Enable IAP
- Update Expose Section (leveraging Overrides) to set the backend config to match the ones we created earlier.
- If you followed this guide, they are the same name below.
Note: The below example also shows how to configure Front50 to use GCS and how to wire up a sample service account, but those are out of scope for this article!
Time to test!
After you applied all of the above YAML configs, you should now be able to go to your Spinnaker Ingress you created (and hopefully have mapped to a domain like https://spinnaker.company.com), and you should be prompted to Login with Google. If you granted yourself IAP web secure permissions, you should be able to login and see yourself as an authenticated user in Spinnaker!
