Dave Arlin
Oct 19, 2018 · 14 min read

Background

Our team was looking for a solution to easily view and manage deployments across our different development environments. We set up different GCP projects for each environment with each project hosting its own Cloud SQL instance, Cloud Storage buckets and App Engine services.

Continuous Delivery Needs

  • Environment Variable Management
  • Secret Management
  • Package/Artifact Repository
  • High level view of environment deployment configuration. For instance, a way to see that Dev is running version 1.2.0.5, QA — 1.2.0.4, Prod — 1.1.0.8, etc.
  • Promote and Demote versions of service packages with ease.
  • Same tooling and mechanism for frontend and backend tech stacks.
  • Prefer to keep it simple and not use too many different tools that require their own set up, UIs, user management, integrations with one another, etc.

Tech Stack

Our frontend and backend tech stacks are relatively standard across many App Dev projects.

Infrastructure

GAE (Standard) Services

Solution

Let’s step through the process to get things configured for CI/CD. Although the above GCP products are only used in our configuration, any GCP product accessible via the Google Cloud SDK could be configured this way.

Backend Code

With Spring Boot, most of our configuration is done via application.yml files. This externalizes our configuration which will later allow us to configure different application settings for each environment. A very common setting you might see in an API’s configuration are database configuration settings. Here is the relevant portion of our application-database.yml file.

spring:
datasource:
driverClassName: org.postgresql.Driver
url: jdbc:postgresql://localhost:5432/mydatabase
username: myuser
password: mypassword
<?xml version=”1.0" encoding=”utf-8"?>
<appengine-web-app xmlns=”http://appengine.google.com/ns/1.0">
<service>my-application-api</service>
<threadsafe>true</threadsafe>
<runtime>java8</runtime>
<instance-class>${appengine.instance.class}</instance-class>
<system-properties>
<property name=”spring.datassource.url” value=”${spring.datasource.url}” />
<property name=”spring.datasource.username” value=”${spring.datasource.username}” />
<property name=”spring.datasource.password” value=”${spring.datasource.password}” />
</system-properties>
<properties>
<env>local</env>
<spring.boot.version>2.0.0.RELEASE</spring.boot.version>
<java.version>1.8</java.version>
<maven.compiler.source>${java.version}</maven.compiler.source>
<maven.compiler.target>${java.version}</maven.compiler.target>
<appengine.target.version>1.9.48</appengine.target.version>
<appengine.client.version>1.23.0</appengine.client.version>
<javax.el.version>3.0.0</javax.el.version>
<appengine.instance.class>F1</appengine.instance.class>
<appengine.min.instances>1</appengine.min.instances>
<appengine.max.instances>10</appengine.max.instances>
<spring.datasource.url>jdbc:postgresql://localhost:5432/mydatabase</spring.datasource.url>
<spring.datasource.username>myuser</spring.datasource.username>
<spring.datasource.password>mypassword</spring.datasource.password>
</properties>
<profiles>
<profile>
<id>ci</id>
<properties>
<env>#{SpringProfile}</env>
<appengine.instance.class>#{AppengineInstanceClass}</appengine.instance.class>
<appengine.min.instances>#{AppengineMinInstances}</appengine.min.instances>
<appengine.max.instances>#{AppengineMaxInstances}</appengine.max.instances>
<spring.datasource.username>#{ApplicationDatasourceUrl}</spring.datasource.username>
<spring.datasource.username>#{ApplicationDatasourceUsername}</spring.datasource.username>
<spring.datasource.password>#{ApplicationDatasourcePassword}</spring.datasource.password>
</properties>
</profile>
</profiles>

Frontend Code

We are using Webpack to bundle our code. We are also able to use the StringReplacePlugin to help set our Octopus tokens similar to what we are doing with pom.xml for the backend. Here is the relevant snippet from webpack.config.js

const { resolve } = require(‘path’);
const webpack = require(‘webpack’);
const StringReplacePlugin = require(‘string-replace-webpack-plugin’);
const properties = {
apiBackend: ‘http://localhost:8080/api',
enablePWA: ‘true’
};
const stringReplacementLoader = StringReplacePlugin.replace({
replacements: [
{
pattern: /\#{(.*)}/g,
replacement: function (match, propertyName) {
return properties[propertyName];
}
}
]});

Jenkins Pipeline Configuration

The team set up a Jenkins multibranch pipeline.

API Jenkins Pipeline

For the API, all branches including pull requests are set to be discovered and pulled. A single pipeline file, ci.jenkins is used with all branches.

def OCTOPUS_PROJECT_PREFIX = ‘MyAPI’
def OCTOPUS_PROJECT_ID = ‘Projects-1’
def OCTOPUS_PACKAGE_NAME = ‘’
def OCTOPUS_PACKAGE_VERSION = ‘’
def OCTOPUS_RELEASE_VERSION = ‘’
def MAJOR_MINOR_REVISION_VERSION = ‘1.0.6’
def GCP_PACKAGE_SDK_VERSION = ‘215.0.0’
pipeline {
agent any
parameters {
string(defaultValue: ‘Environments-1’, description: ‘This maps to a GCP project: (Environments-1 = davearlin-dev, Environments-2 = davearlin-qa, Environments-3 = davearlin-uat)’, name: ‘octopus_environment_id’)
}
tools {
maven ‘Maven 3.5.3’
}
stages {
stage(‘Clean’) {
steps {
sh “mvn clean”
}
}
stage(‘Unit Tests’) {
when {
expression {
return env.BRANCH_NAME != ‘develop’ && !env.BRANCH_NAME.toLowerCase().contains(‘release/’);
}
}
steps {
sh “mvn test”
}
}
stage(‘Unit Tests and Stage’) {
when {
expression {
return env.BRANCH_NAME == ‘develop’ || env.BRANCH_NAME.toLowerCase().contains(‘release/’);
}
}
steps {
sh “mvn appengine:stage -Pci”
}
}
stage(‘Package Deployment’) {
when {
expression {
return env.BRANCH_NAME == ‘develop’ || env.BRANCH_NAME.toLowerCase().contains(‘release/’);
}
}
steps {
script {
if (env.BRANCH_NAME == ‘develop’) {
OCTOPUS_PACKAGE_NAME = “${OCTOPUS_PROJECT_PREFIX}.${MAJOR_MINOR_REVISION_VERSION}.${env.BUILD_NUMBER}-SNAPSHOT.zip”;
OCTOPUS_PACKAGE_VERSION = “${MAJOR_MINOR_REVISION_VERSION}.${env.BUILD_NUMBER}-SNAPSHOT”;
OCTOPUS_RELEASE_VERSION = “${MAJOR_MINOR_REVISION_VERSION}.${env.BUILD_NUMBER}”;
} else {
def packageExtension = “-rc.${env.BUILD_NUMBER}”;
OCTOPUS_PACKAGE_NAME = “${OCTOPUS_PROJECT_PREFIX}.${MAJOR_MINOR_REVISION_VERSION}${packageExtension}.zip”;
OCTOPUS_PACKAGE_VERSION = “${MAJOR_MINOR_REVISION_VERSION}${packageExtension}”;
OCTOPUS_RELEASE_VERSION = “${MAJOR_MINOR_REVISION_VERSION}${packageExtension}”;
} }
echo “Package Name: ${OCTOPUS_PACKAGE_NAME}”
dir(‘target/appengine-staging’) {
sh “zip -r ${OCTOPUS_PACKAGE_NAME} ./”
}
}
}
stage(‘Octopus — Send Package’) {
when {
expression {
return env.BRANCH_NAME == ‘develop’ || env.BRANCH_NAME.toLowerCase().contains(‘release/’);
}
}
steps {
dir(‘target/appengine-staging’) {
sh “/opt/bitnami/common/bin/curl -X POST ${env.OctopusBaseUrl}/api/packages/raw -H \”${env.OctopusApiKeyHeader}\” -F \”data=@${OCTOPUS_PACKAGE_NAME}\””
}
}
}
stage(‘Octopus — Create Release’) {
when {
expression {
return env.BRANCH_NAME == ‘develop’ || env.BRANCH_NAME.toLowerCase().contains(‘release/’);
}
}
steps {
dir(‘target/appengine-staging’) {
script {
def responseText = sh(returnStdout: true, script: “/opt/bitnami/common/bin/curl — fail -X POST ${env.OctopusBaseUrl}/api/releases -H \”${env.OctopusApiKeyHeader}\” -H \”Content-Type: application/json\” -d \”{ \\\”ProjectId\\\”: \\\”${OCTOPUS_PROJECT_ID}\\\”, \\\”SelectedPackages\\\”: [ { \\\”StepName\\\”: \\\”Token Replace and Deploy to GCP\\\”, \\\”Version\\\”: \\\”${OCTOPUS_PACKAGE_VERSION}\\\”, \\\”ActionName\\\”: \\\”Token Replace and Deploy to GCP\\\”, \\\”PackageReferenceName\\\”: \\\”${OCTOPUS_PROJECT_PREFIX}\\\” }, { \\\”StepName\\\”: \\\”Token Replace and Deploy to GCP\\\”, \\\”Version\\\”: \\\”${GCP_PACKAGE_SDK_VERSION}\\\”, \\\”ActionName\\\”: \\\”Token Replace and Deploy to GCP\\\”, \\\”PackageReferenceName\\\”: \\\”GoogleCloudSDK\\\” } ], \\\”Version\\\”: \\\”${OCTOPUS_RELEASE_VERSION}\\\” }\””).trim()
def releaseInfo = readJSON text: responseText;
if (env.BRANCH_NAME == ‘develop’) {
sh(script: “/opt/bitnami/common/bin/curl — fail -X POST ${env.OctopusBaseUrl}/api/deployments -H \”${env.OctopusApiKeyHeader}\” -H \”Content-Type: application/json\” -d \”{ \\\”ProjectId\\\”: \\\”${OCTOPUS_PROJECT_ID}\\\”, \\\”EnvironmentId\\\”: \\\”${params.octopus_environment_id}\\\”, \\\”ReleaseId\\\”: \\\”${releaseInfo.Id}\\\”, \\\”UseGuidedFailure\\\”: false }\””)
}
}
}
}
}
stage(‘Set Build Success’) {
steps {
script {
currentBuild.result = ‘SUCCESS’
}
}
}
}
post {
success {
echo ‘Build succeeded’
}
failure {
echo ‘Build failed’
slackSend “@here — Build Failed: `${env.JOB_NAME} [${env.BUILD_NUMBER}]`\n${env.BUILD_URL}\n```Last Commit: ${env.GIT_COMMIT}```”
}
changed {
echo ‘Build result changed’
script {
if (currentBuild.result == ‘SUCCESS’) {
slackSend “Build Fixed: `${env.JOB_NAME} [${env.BUILD_NUMBER}]` \n${env.BUILD_URL}”
}
}
}
}
}
  1. Package Deployment — This zips the contents of the /target/appengine-staging folder to a file named appropriately to match Octopus’ supported versioning structure. Octopus supports both Semantic Versioning and Maven Versioning. We chose to go with Maven Versioning.
  2. Octopus — Send Package — This invokes a curl POST to the Octopus API “api/packages/raw” endpoint sending the zip to Octopus’ package repository.
  3. Octopus — Create Release — This invokes a curl POST to the Octopus API “api/releases” endpoint telling Octopus to create a new release. Releases in Octopus represent a snapshot of the packages that are associated with them. They also contain a project which in our case is “Project-1”. This is the ID in Octopus that maps the more readable “My API” project. The appropriate project ID can be found via the Octopus API, since this doesn’t change, we hard code it in the pipeline script.

Frontend Jenkins Pipeline

The frontend works similar to the API. Again, all branches including pull requests are set to be discovered and pulled. A single pipeline file, ci.jenkins is used with all branches.

def OCTOPUS_PROJECT_PREFIX = ‘MyWebSite’
def OCTOPUS_PROJECT_ID = ‘Projects-2’
def OCTOPUS_PACKAGE_NAME = ‘’
def OCTOPUS_PACKAGE_VERSION = ‘’
def OCTOPUS_RELEASE_VERSION = ‘’
def MAJOR_MINOR_REVISION_VERSION = ‘1.0.6’
def GCP_PACKAGE_SDK_VERSION = ‘215.0.0’
pipeline {
agent any
parameters {
string(defaultValue: ‘Environments-1’, description: ‘This maps to a GCP project: (Environments-1 = davearlin-dev, Environments-2 = davearlin-qa, Environments-3 = davearlin-uat)’, name: ‘octopus_environment_id’)
}
tools {
nodejs ‘NodeJS 8.11.1’
}
stages {
stage(‘NPM’) {
steps {
sh “npm install”
sh “npm run production”
}
}
stage(‘Set up Staging Folder’) {
when {
expression {
return env.BRANCH_NAME == ‘develop’ || env.BRANCH_NAME.toLowerCase().contains(‘release/’);
}
}
steps {
sh “rm -rf stage”
sh “mkdir stage”
sh “cp -r dist/ stage/”
sh “cp app.yaml stage/”
sh “cp -r dispatch/ stage/”
}
}
stage(‘Package Deployment’) {
when {
expression {
return env.BRANCH_NAME == ‘develop’ || env.BRANCH_NAME.toLowerCase().contains(‘release/’);
}
}
steps {
script {
if (env.BRANCH_NAME == ‘develop’) {
OCTOPUS_PACKAGE_NAME = “${OCTOPUS_PROJECT_PREFIX}.${MAJOR_MINOR_REVISION_VERSION}.${env.BUILD_NUMBER}-SNAPSHOT.zip”;
OCTOPUS_PACKAGE_VERSION =”${MAJOR_MINOR_REVISION_VERSION}.${env.BUILD_NUMBER}-SNAPSHOT”;
OCTOPUS_RELEASE_VERSION = “${MAJOR_MINOR_REVISION_VERSION}.${env.BUILD_NUMBER}”;
} else {
def packageExtension = “-rc.${env.BUILD_NUMBER}”;
OCTOPUS_PACKAGE_NAME = “${OCTOPUS_PROJECT_PREFIX}.${MAJOR_MINOR_REVISION_VERSION}${packageExtension}.zip”;
OCTOPUS_PACKAGE_VERSION = “${MAJOR_MINOR_REVISION_VERSION}${packageExtension}”;
OCTOPUS_RELEASE_VERSION = “${MAJOR_MINOR_REVISION_VERSION}${packageExtension}”;
}
}
echo “Package Name: ${OCTOPUS_PACKAGE_NAME}”
dir(‘stage’) {
sh “zip -r ${OCTOPUS_PACKAGE_NAME} ./”
}
}
}
stage(‘Octopus — Send Package’) {
when {
expression {
return env.BRANCH_NAME == ‘develop’ || env.BRANCH_NAME.toLowerCase().contains(‘release/’);
}
}
steps {
dir(‘stage’) {
sh “/opt/bitnami/common/bin/curl -X POST ${env.OctopusBaseUrl}/api/packages/raw -H \”${env.OctopusApiKeyHeader}\” -F \”data=@${OCTOPUS_PACKAGE_NAME}\””
}
}
}
stage (‘Octopus — Create Release’) {
when {
expression {
return env.BRANCH_NAME == ‘develop’ || env.BRANCH_NAME.toLowerCase().contains(‘release/’);
}
}
steps {
dir(‘stage’) {
script {
def responseText = sh (returnStdout: true, script: “/opt/bitnami/common/bin/curl — fail -X POST ${env.OctopusBaseUrl}/api/releases -H \”${env.OctopusApiKeyHeader}\” -H \”Content-Type: application/json\” -d \”{ \\\”ProjectId\\\”: \\\”${OCTOPUS_PROJECT_ID}\\\”, \\\”SelectedPackages\\\”: [ { \\\”StepName\\\”: \\\”Token Replace and Deploy to GCP\\\”, \\\”Version\\\”: \\\”${OCTOPUS_PACKAGE_VERSION}\\\”, \\\”ActionName\\\”: \\\”Token Replace and Deploy to GCP\\\”, \\\”PackageReferenceName\\\”: \\\”${OCTOPUS_PROJECT_PREFIX}\\\” }, { \\\”StepName\\\”: \\\”Token Replace and Deploy to GCP\\\”, \\\”Version\\\”: \\\”${GCP_PACKAGE_SDK_VERSION}\\\”, \\\”ActionName\\\”: \\\”Token Replace and Deploy to GCP\\\”, \\\”PackageReferenceName\\\”: \\\”GoogleCloudSDK\\\” } ], \\\”Version\\\”: \\\”${OCTOPUS_RELEASE_VERSION}\\\” }\””).trim()
def releaseInfo = readJSON text: responseText;
if (env.BRANCH_NAME == ‘develop’) {
sh (script: “/opt/bitnami/common/bin/curl — fail -X POST ${env.OctopusBaseUrl}/api/deployments -H \”${env.OctopusApiKeyHeader}\” -H \”Content-Type: application/json\” -d \”{ \\\”ProjectId\\\”: \\\”${OCTOPUS_PROJECT_ID}\\\”, \\\”EnvironmentId\\\”: \\\”${params.octopus_environment_id}\\\”, \\\”ReleaseId\\\”: \\\”${releaseInfo.Id}\\\”, \\\”UseGuidedFailure\\\”: false }\””)
}
}
}
}
}
stage(‘Set Build Success’){
steps {
script {
currentBuild.result = ‘SUCCESS’
}
}
}
}
post {
success {
echo ‘Build succeeded’
}
failure {
slackSend “@here — Build Failed: `${env.JOB_NAME} [${env.BUILD_NUMBER}]`\n${env.BUILD_URL}\n```Last Commit: ${env.GIT_COMMIT} by ${env.GIT_COMMITTER_NAME}\nPrevious Successful Commit: ${env.GIT_PREVIOUS_SUCCESSFUL_COMMIT}```”
}
changed {
echo ‘Build result changed’
script {
if(currentBuild.result == ‘SUCCESS’) {
slackSend “Build Fixed: `${env.JOB_NAME} [${env.BUILD_NUMBER}]` \n${env.BUILD_URL}”
}
}
}
}
}
  1. Package Deployment — This zips the contents of the /stage folder to a file set to Maven Versioning.
  2. Octopus — Send Package — Same as the API pipeline. This invokes a curl POST to the Octopus API “api/packages/raw” endpoint sending the zip to Octopus’ package repository.
  3. Octopus — Create Release — Same as the API pipeline. This invokes a curl POST to the Octopus API “api/releases” endpoint telling Octopus to create a new release. Releases in Octopus represent a snapshot of the packages that are associated with them, a project, in our case the “My API” project as well as the current set of deployment steps assigned to the project.

Octopus Configuration

Our goal is to have an Octopus Dashboard set up like this:

$extractedpath = $OctopusParameters[“Octopus.Action.Package[MyAPI].ExtractedPath”]
Write-Host “Extracted Path: $extractedpath”
$content = Get-Content -Path “$extractedpath\WEB-INF\appengine-web.xml”;
Write-Host “Content: $content”
$serviceAccountJson = ‘#{MyServiceAccountKey}’
Write-Host “Serivce Account: $serviceAccountJson”
$serviceAccountJsonFilePath = “$extractedpath\serviceAccount.json”
Set-Content -Path “$serviceAccountJsonFilePath” -Value “$serviceAccountJson”
$gcloudExtractedPath = $OctopusParameters[“Octopus.Action.Package[GoogleCloudSDK].ExtractedPath”]
Write-Host “Extracted GCP Path: $gcloudExtractedPath”
$CMD = “$gcloudExtractedPath\google-cloud-sdk\bin\gcloud.cmd”
$arg1 = “auth”
$arg2 = “activate-service-account”
$arg3 = “ — key-file=$serviceAccountJsonFilePath”
& $CMD $arg1 $arg2 $arg3$appEngineServiceVersion = $OctopusParameters[“Octopus.Action.Package[MyAPI].PackageVersion”]
$appEngineServiceVersion = $appEngineServiceVersion.ToLower();
If ($appEngineServiceVersion.IndexOf(“snapshot”) -ge 0) {
$appEngineServiceVersion = $appEngineServiceVersion.Substring(0, $appEngineServiceVersion.LastIndexOf(“.”)) + $appEngineServiceVersion.Substring($appEngineServiceVersion.IndexOf(“-”))
}
Else {
$appEngineServiceVersion = $appEngineServiceVersion.Substring(0, $appEngineServiceVersion.IndexOf(“-”) + 1) + “release”;
}
$appEngineServiceVersion = $appEngineServiceVersion.Replace(“.”,”-”);$arg1 = “app”
$arg2 = “deploy”
$arg3 = “ — project=#{MyGCPProjectName}”
$arg4 = “ — version=$appEngineServiceVersion”
$arg5 = “ — quiet”
Set-Location $extractedpath
& $CMD $arg1 $arg2 $arg3 $arg4 $arg5
$cronArg = “$extractedpath\WEB-INF\appengine-generated\cron.yaml”
& $CMD $arg1 $arg2 $cronArg $arg3 $arg5
$extractedpath = $OctopusParameters[“Octopus.Action.Package[MyWebsite].ExtractedPath”]
Write-Host “Extracted Path: $extractedpath”

$serviceAccountJson = ‘#{MyServiceAccountKey}’
Write-Host “Serivce Account: $serviceAccountJson”
$serviceAccountJsonFilePath = “$extractedpath\serviceAccount.json”
Set-Content -Path “$serviceAccountJsonFilePath” -Value “$serviceAccountJson”
$gcloudExtractedPath = $OctopusParameters[“Octopus.Action.Package[GoogleCloudSDK].ExtractedPath”]
Write-Host “Extracted GCP Path: $gcloudExtractedPath”
$CMD = “$gcloudExtractedPath\google-cloud-sdk\bin\gcloud.cmd”
$arg1 = “auth”
$arg2 = “activate-service-account”
$arg3 = “ — key-file=$serviceAccountJsonFilePath”
& $CMD $arg1 $arg2 $arg3$appEngineServiceVersion = $OctopusParameters[“Octopus.Action.Package[MyWebsite].PackageVersion”]
$appEngineServiceVersion = $appEngineServiceVersion.ToLower();
If ($appEngineServiceVersion.IndexOf(“snapshot”) -ge 0) {
$appEngineServiceVersion = $appEngineServiceVersion.Substring(0, $appEngineServiceVersion.LastIndexOf(“.”)) + $appEngineServiceVersion.Substring($appEngineServiceVersion.IndexOf(“-”))
}
Else {
$appEngineServiceVersion = $appEngineServiceVersion.Substring(0, $appEngineServiceVersion.IndexOf(“-”) + 1) + “release”;
}
$appEngineServiceVersion = $appEngineServiceVersion.Replace(“.”,”-”);#Deploy app to App Engine
$arg1 = “app”
$arg2 = “deploy”
$arg3 = “ — project=#{MyGCPProjectName}”
$arg4 = “ — version=$appEngineServiceVersion”
$arg5 = “ — quiet”
Set-Location $extractedpath
& $CMD $arg1 $arg2 $arg3 $arg4 $arg5
#Deploy appropriate environment dispatch.yaml
$dispatchYamlArg = “.\dispatch\#{Dispatch_Yaml_Folder}\dispatch.yaml”
& $CMD $arg1 $arg2 $arg3 $arg5 $dispatchYamlArg

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade