Updating Spinnaker Halyard Releases with Custom Containers

In this post I will be outlining a few different cases where overriding the upstream provided docker containers in a Spinnaker release would be necessary.

Where are the Spinnaker microservice containers stored?

The source URL can be extracted from the Halyard source code, but the URL it reaches out to as of the writing of this post follows the pattern: https://storage.googleapis.com/halconfig/bom/<version>.yml

For Spinnaker 1.5.0 the URL is : https://storage.googleapis.com/halconfig/bom/1.5.0.yml

Inside this file there are versions of spinnaker microservices and where they are stored by artifact type.

artifactSources:
  debianRepository: https://dl.bintray.com/spinnaker-releases/debians
  dockerRegistry: gcr.io/spinnaker-marketplace
  gitPrefix: https://github.com/spinnaker
  googleImageProject: marketplace-spinnaker-release
dependencies:
  consul:
    commit: null
    version: 0.7.5
  redis:
    commit: null
    version: 2:2.8.4-2
  vault:
    commit: null
    version: 0.7.0
hostname: null
services:
  clouddriver:
    commit: 9c517d1f642ee12e8a0ff76b8ca35891ab419123
    version: 1.0.0-20171108151529
  deck:
    commit: b04f25fd5040cfa875bd32d4ee9e622625caeef0
    version: 2.0.0-20171108151529
  echo:
    commit: bf3a4b1d05196d6c1fd993cf0ecf5f767eab4312
    version: 0.7.0-20171108151529
  fiat:
    commit: 55b10104aaed66fa39a5a9f8b68a53635daf055f
    version: 0.4.2-20171108151529
  front50:
    commit: 818557e1b336e0508cef78e920eeb6f40143f98c
    version: 0.8.0-20171108151529
  gate:
    commit: b33909b23f190219891800a3cc122113814c9c04
    version: 0.9.0-20171108151529
  igor:
    commit: b85d8c4453991697c514e11677a78a5335ed73d4
    version: 0.8.0-20171108151529
  monitoring-daemon:
    commit: 6c1f67999bb07c7da1e56e04964b05fe400b0649
    version: 0.4.0-20171108151529
  monitoring-third-party:
    commit: 6c1f67999bb07c7da1e56e04964b05fe400b0649
    version: 0.4.0-20171108151529
  orca:
    commit: 52823d65f6acf79874bd3d3c40e05d530f8c29d7
    version: 0.9.0-20171108151529
  rosco:
    commit: 08c10212818dc71cc4d33cce5cddc11264be6556
    version: 0.4.4-20171108151529
  spinnaker:
    commit: 730297693efd54233fee799542c41fe9c7d50d47
    version: 0.14.0-20171108151529
timestamp: '2017-11-08 18:52:35'
version: 1.5.0

Bolded within this file are the pieces of information that will let us create a Docker container with the Spinnaker microservice that needs customizations. To pull the Clouddriver (from the 1.5.0 release) the docker command would be:

edwin:~/spinnaker$ docker pull gcr.io/spinnaker-marketplace/clouddriver:1.0.0-20171108151529
1.0.0-20171108151529: Pulling from spinnaker-marketplace/clouddriver
709515475419: Pull complete
38a1c0aaa6fd: Pull complete
cd134db5e982: Pull complete
2fef3808d107: Pull complete
e0980e52aba1: Pull complete
Digest: sha256:98a79058b7c5e9561e9174f98ea93e1dc13cee0f496d9e9de72007aceade1340
Status: Downloaded newer image for gcr.io/spinnaker-marketplace/clouddriver:1.0.0-20171108151529

Adding custom certificate chains to upstream containers

There might be times when the Spinnaker microservices need to connect to URLs that are protected with internal certificates that are not signed by a public certificate authority. A way to include those certificates to the service is to package them inside the Docker container and publish it somewhere Halyard can access the container.

An example Dockerfile, CustomClouddriverDockerFile outlines the basic idea

FROM gcr.io/spinnaker-marketplace/clouddriver:1.0.0-20171108151529
COPY cert_chain.crt /
RUN keytool -importcert -keypass changeit -file /cert_chain.crt -keystore /usr/lib/jvm/java-1.8-openjdk/jre/lib/security/cacerts -noprompt -storepass changeit

The Dockerfile can be built and pushed to a Docker registry

edwin:~/spinnaker$ docker build -f CustomClouddriverDockerFile -t edwinavalos/clouddriver:withChain
edwin:~/spinnaker$ docker push edwinavalos/clouddriver:withChain

The last step is making reference to this container within the Halyard configuration utility. This file for clouddriver will most likely be at ~/.hal/default/service-settings/clouddriver.yml

artifactId: edwinavalos/clouddriver:withChain

Once this file is in place, a hal deploy apply will deploy Spinnaker using the customized container for the service. This can be done for any of the other Spinnaker microservice containers.

Creating Microservice Containers off HEAD or a Branch

There are times when there is a need to run code newer than the latest Spinnaker release. This can be accomplished similar to the steps outlined above

1. Fork and clone the microservice, checkout the branch to build

edwin:~/spinnaker$ git clone https://github.com/edwinavalos/clouddriver.git
Cloning into 'clouddriver'...
remote: Counting objects: 91598, done.
remote: Compressing objects: 100% (14/14), done.
remote: Total 91598 (delta 3), reused 2 (delta 2), pack-reused 91582
Receiving objects: 100% (91598/91598), 32.34 MiB | 7.77 MiB/s, done.
Resolving deltas: 100% (35357/35357), done.
Checking connectivity... done.
edwin:~/spinnaker$ cd clouddriver/
edwin:~/spinnaker/clouddriver$ git remote add upstream https://github.com/spinnaker/clouddriver.git
edwin:~/spinnaker/clouddriver$ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
edwin:~/spinnaker/clouddriver$ git pull upstream master
<snipped>

2. Build the microservice via gradle

edwin:~/spinnaker/clouddriver$ ./gradlew clouddriver-web:installDist -x test
<snipped>
BUILD SUCCESSFUL
Total time: 5 mins 46.7 secs
This build could be faster, please consider using the Gradle Daemon: https://docs.gradle.org/2.14.1/userguide/gradle_daemon.html

3. Build the Dockerfile.slim

edwin:~/spinnaker/clouddriver$ docker build -t edwinavalos/clouddriver:master -f Dockerfile.slim .
<snipped>
 ---> Running in 3a04d31ca55d
 ---> a2cacc78829a
Removing intermediate container 3a04d31ca55d
Successfully built a2cacc78829a
Successfully tagged edwinavalos/clouddriver:master
edwin:~/spinnaker/clouddriver$ docker push edwinavalos/clouddriver:master

4. Update the service-settings for the microservice

edwin:~/spinnaker/clouddriver$ cat ~/.hal/default/service-settings/clouddriver.yml
artifactId: edwinavalos/clouddriver:master

5. Run the hal deployment to start using the new container

edwin:~/spinnaker/clouddriver$ hal deploy apply

Other notes

This approach can be used to create customized containers for Spinnaker that contain helper scripts if necessary for debugging down the road.

Halyard together with the custom containers can help speed up the development process of getting rapid feedback on new Spinnaker code!

Let me know if your thoughts on this article and if there is anything that you would like more detail on.