Spinnaker Installation

This is the first of three articles in total on deploying Spinnaker onto Tencent Cloud Kubernetes cluster.

Tencent Cloud is one of the top global public Cloud providers. Tencent Cloud fully supports Cloud Native initiatives, such as Kubernetes for containerization and Terraform as IaC.

Spinnaker is an open-source, multi-cloud continuous delivery platform that helps you release software changes with high velocity and confidence. This article goes through the step-by-step process of installing Spinnaker on Tencent Cloud TKE service using Terraform. After this installation process, you should have a basic Spinnaker running on your cluster that has access to the Kubernetes cluster and TCR you specified and uses COS as storage.

Prerequisites:

1. If you haven’t used Tencent Cloud’s Terraform provider before, please follow this guide to get started.
2. If you don’t have a Tencent Cloud Kubernetes cluster yet, please follow this guide to launch a TKE cluster instance.

Step 1: Install Halyard

Halyard is the tool that installs and manages all Spinnaker features and functions. You can choose to either install it on a local machine or install it on a container in the Cloud. It’s recommended to install Halyard on the container to keep it isolated from other services.

Container Installation

1) Go to our repo folder `$path-to-spinnaker/k8-halyard` and copy out all related Terraform configurations.

2) On `main.tf`, change the `namespace` and `name` field of the container to your desired value.

3) Change other fields’ values like labels, subnet id, etc if it is applicable to you.

4) Initiate and apply the TF configuration.

Local Installation

• Linux system requirements:
  Ubuntu 18.04 or higher
  Debian 10 or higher

1) Download latest Halyard:

curl -O <https://raw.githubusercontent.com/spinnaker/halyard/master/install/debian/InstallHalyard.sh>

2) Install Halyard:

sudo bash InstallHalyard.sh

3) Enable command completion:

. ~/.bashrc

Note: there are other installation methods.

Step 2: Set up Environment Variables

Set up the environment variables that will be used later.

export SPINCONFIG=./Spinnaker_k8s_config_file
export WPCONFIG=./Wordpress_k8s_config_file
export CONTEXT=$(kubectl config current-context --kubeconfig=$SPINCONFIG) #Spinnaker k8s context
export NAMESPACE=spinnaker

export SPINTKE=Spinnaker_TKE_Account_Name
export COSART=Spinnaker_COS_Artifacts_Account_Name
export DRTCR=Spinnaker_TCR_Account_Name
export SPIN_VERSION=1.29.2

Step 3: Configure Spinnaker Accounts and Storage

• Add Spinnaker TKE Cluster Account

1) Use Terraform config files in the repo folder `$path-to-k8-namespace` to create the `spinnaker` namespace in your desired k8s cluster.

2) Use the Terraform config files in the repo folder `$path-to-k8-security` to create a service account, customize the policy, and bind `ClusterRole` to it. The least permission you should set is listed here.

3) Generate Spinnaker K8s config file. Extract the secret token of the created service account:

export TOKEN=$(kubectl get secret --context $CONTEXT --kubeconfig=$SPINCONFIG\
$(kubectl get serviceaccount spinnaker-sa \
--context $CONTEXT \
--kubeconfig=$SPINCONFIG \
-n $NAMESPACE \
-o jsonpath='{.secrets[0].name}') \
-n spinnaker \
-o jsonpath='{.data.token}' | base64 --decode)

4) Set the user entry in original kubeconfig file:

kubectl config set-credentials ${CONTEXT}-token-user --kubeconfig=$SPINCONFIG --token $TOKEN
kubectl config set-context $CONTEXT --kubeconfig=$SPINCONFIG --user ${CONTEXT}-token-user

5). Add Spinnaker K8s account

hal config provider kubernetes enable
hal config provider kubernetes account add $SPINTKE --kubeconfig-file=$SPINCONFIG --context $CONTEXT

The above process is on Kubernetes v1.22. If you work on v1.24+, you need to add one more step to generate a secret and bind it to service account because Kubernetes no longer generates sercets when a service account is created in v1.24. You can uncomment related blocks in `main.tf` to bind the secret automatically.

• Add Spinnaker Managed TCR Account

1) Enable Docker Registry Provider:

hal config provider docker-registry enable

2) Add Spinnaker TCR account:

hal config provider docker-registry account add $DRTCR \
--address <https://TCR_address> \
--repositories TCR_repo \
--username TCR_username \
--password

If you don’t supply the value of — secret-access-key on the command line, you will be prompted to enter the value on STDIN once the command has started running.

• Add COS as Backend Storage

1) Run the following to configure COS storage:

hal config storage s3 edit \
 --access-key-id IKIDToxsh***aIqCOM6bstgGfqm \
 --secret-access-key \
 --endpoint cos.na-siliconvalley.myqcloud.com \
 --bucket your-cos-bucket \
 --root-folder spinnaker

If you don’t supply the value of — secret-access-key on the command line, you will be prompted to enter the value on STDIN once the command has started running.

2) Set the storage source to it:

hal config storage edit --type s3

Please make sure `cos` add-on has been installed into the TKE cluster where Spinnaker is going to install.

Step 4: Deploy Spinnaker

First, you need to set a distributed Spinnaker installation onto one of the Kubernetes cluster accounts:

hal config deploy edit --type distributed --account-name $SPINTKE

Choose Spinnaker Version you are going to install:

hal config version edit --version $SPIN_VERSION

Modify `liveness-probe-initial-delay-seconds` value to to the upper bound of your longest service startup time:

hal config deploy edit --liveness-probe-enabled true --liveness-probe-initial-delay-seconds 180

Deploy Spinnaker:

hal deploy apply

Step 5: Expose Spinnaker Services to a private Load Balancer

Uncomment the Step 5 block in `main.tf` and modify related yaml file in `./settings` and then run `terraform apply` to add private Load Balancer service to Spinnaker.

Configure the URL for Gate and Deck:

#get deck ip
export UI_URL=$(kubectl -n $NAMESPACE get svc spin-deck-private -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

#get gate ip
export API_URL=$(kubectl -n $NAMESPACE get svc spin-gate-private -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

#configure it to Spinnaker
hal config security api edit --override-base-url <http://${API_URL}>
hal config security ui edit --override-base-url <http://${UI_URL}>

Deploy the settings:

hal deploy apply

After all these steps, you should be able to access the Spinnaker UI at the UI_URL you set and see all the accounts you set previously.

To see a basic example of how to deploy a WordPress application to one of your TKE clusters, you can visit this page.

• Upgrade Spinnaker Version

If you want to change Spinnaker versions using Halyard, you can read about supported versions like so:

hal version list

And pick a new version like so:

hal config version edit --version $VERSION
hal deploy apply

• Delete Deployed Spinnaker Service

To delete deployed Spinnaker, run

hal deploy clean

Author: Minze Tao

Minze Tao is a Solution Architect and Product Operation Engineer at Tencent America. He has profound experiences with Tencent Cloud products and architecture. He has dedicated his latest effort in supporting Tencent Cloud’s gaming customers on multi-cloud architecture.