Microservice deployments made super easy

Published in

comsystoreply

9 min readSep 10, 2019

GitHub Actions, AWS Elastic Beanstalk and Dockerized Deployments at ease.

I have been using many deployment mechanisms in many projects and of course, the nicest thing out there is stuff like Kubernetes, CloudFoundry a.s.o

But for those big things you need DevOps people to manage those huge installations. I was recently updating some old microservices and needed to move them to their dedicated AWS account.

We just had been accepted for the GitHub Actions Beta on GitHub and I thought “Now there really should be an easy way to deploy stuff!”.

1. AWS Elastic Beanstalk Project Setup

So these miroservices are either NodeJS or Java. But I wanted a common way to deploy this stuff. Therefore I decided to use the Docker Single Instance approach.

Basically this means for my NodeJS apps to provide this directory structure:

/Dockerrun.aws.json
/Dockerfile
/.ebextensions/*.config
/.elasticbeanstalk/config.yml
/package/app.server.js
/package/package.json

The Dockerrun.aws.json only tells AWS EB which version to use.

Dockerrun.aws.json

The .ebextensions/proxy.config tells the NGINX that runs as a thin layer around our docker container to adjust some settings. Make sure it has *.config as extensions and that you include .ebextensions in the deploy.zip. In our case we wanted to increase the client_max_body_sizeto avoid HTTP Errors like 413 Request Entity Too Large.

.ebextensions/proxy.config

UPDATE 2020–06: I had some problems in production with the container_commands failing and bringing the service down. So I had to disable the nginx proxy completely. Which can be done under configuration -> software.

The Dockerfile tells AWS EB how the app should be build and which ports are exposed. The cool thing: You do not need to build the docker image yourself and deploy it to e.g. AWS ACR — the EB does this on the fly for you.

Dockerfile

The Environment variables used are just defaults, we will override them later in the Configuration of our EB Environments.

Our NodeJS build should simply build the app and put the package.json and the compiled app JS files into the /package folder. During the build of the Docker image the dependencies are installed via npm install.

Lastly we need to have a config file .elasticbeanstalk/config.yml for EB to identify the app we want to deploy to.

.elasticbeanstalk/config.yml

This file contains instructions on how the EB CLI should deploy the app.
Especially the line deploy: artifact: deploy.zip is important. I decided to always use this approach and a small shell script creates the deploy.zip for me.

.github/build-package-and-deploy-zip.sh

.github/build-package-and-deploy-zip.sh

Ok now comes the cool thing, we can let EB CLI build and start the docker image locally as it would on AWS.

Therefore we just build the app, create the deploy.zip and run the eb local run command. Of course you need a ~/.aws/config file containing your AccessKey and AccessSecretKey.

Now we can open http://localhost:10444 and see our app.

2. AWS Elastic Beanstalk initial App Setup

I have to say that I just want these environments without blueGreen deployment because it is ok if during redeploy the app is offline for some seconds. If you want blueGreen deployment you can do this as well, but we will simply do a normal deploy.

I decided for these Environment conventions

git branch develop deploys to sandbox
git branch master deploys to live

2.1 Create App and Environments

We create our app3000 via the GUI — we could via the cli too but we want to see all possibilities.

We select the AWS Elastic Beanstalk Service and click ‘Create New Application’

Create sandbox Environment and select Docker

Now we click on ‘Configure more options’ — if we do not do this now you will not be able to e.g. add a loadbalancer later.

Select ‘High availibility’ to add Loadbalancer

In our case we select ‘High availability’ to get an Application Loadbalancer ALB. We will add HTTPS later via the ALB and have AWS Certificate Manager issue a certificate. Finally Click on ‘Create environment’ and wait until it is created.

It takes some time and then your sandbox environment is ready.

We can visit http://app3000-sandbox.eu-central-1.elasticbeanstalk.com now. We repeat this for the live environment too and then we have our app setup ready for automatic deployment.

2.2 Add a database to the Environment

Should your app need a database we could also click on ‘Configuration’ and add it like so.

This will add a Managed Database and also provides ENV variables so that this is zero config for you.

If you happen to use Java and Spring Data JPA just put this into your config file and it will work out of the box:

Spring Data JPA RDS config

If you use NodeJS and TypeORM then you could use this config.

TypeORM config

This is really useful when having different environments but not needing a special config per environment. Simply use the ENV variables provided and all is fine.

2.3 Providing custom ENV variables to your app

You have seen the ENV vars I defined in the Dockerfile and here is how you can set them for your EB Environment.

2.4 Deployment User with limited Access (IAM)

We want to deploy to AWS EB via GitHub Actions and do not want to use our main IAM user account. Therefore we create a deployment user with limited access (you can even limit it some more, but for me this is fine).

Create the user with the plocy AWSElasticBeanstalkService . Store AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in your password manager for later usage.

3. Deploy on code push via GitHub Actions

GitHub Actions is currently beta only but will soon be available. And since I am a big GitHub fanboy this is like christmas for me :)

So what does GitHub Actions do?

On code push run a build and possibly deploy
On Issue event run action
Run builds on PullRequests and inform if PR has build errors
Provide pre-build actions and docker environments

We simply want something to happen on code push — You can have multiple files for different events. An Action for a Java 11 Gradle build looks like so:

Buildfile .github/workflows/main.yml

So what happens in this file

Only run if branch master or develop has code push
Setup Java 11
Install the Elastic Beanstalk CLI
Build the Java App with Gradle
Write AWS Config from GitHub Secrets
Conditionally deploy to environment based on branch

Ok this is basically a no brainer but two things are really fun and I will detail them out later. But first let’s see how a build looks:

Successful AWS EB Deployment via GitHub Actions

Beautiful and Awesome :) And if your build should fail you are informed via E-Mail as well. Really nice.

3.1 Using Secrets as ENV variables

We can put our precious passwords into GitHub Secrets and use them as ENV variables during our build. This is needed for us to store AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. Simply put the values into secrets under the projects settings.

During our builds we can access the secrets like so.

I wrote myself a simple bash script create-aws-config.sh to write the stuff needed for EB CLI to work.

NOTE: Alternatively you can also use predefined Actions:
https://github.com/marketplace/actions/beanstalk-deploy to deploy to EB directly from the action. Here you do not need the eb/config.yml and do not need the create-aws-config.sh script.

3.2 Conditional Deployment

GitHub Actions provides a lot of information to use during the build.

We can access the github object and get useful stuff like the branch name out of it. Conditionally we deploy the master branch to live environment. The ENV var $GITHUB_SHA contains the commit hash and is used during deployment as the label/version that is deployed on AWS. After such a deployment the GitHub commit hash is visible here:

3.3 A NodeJS Action

I just showed you my Java 11 Action to build something with Gradle. And here is my NodeJS Action:

As you can see it works exactly like the Java one.

4. Provide HTTPS via ALB

Ok finally we want to have some SSL certificates. What do we need?

Register a domain via AWS Route53 (or use your existing one)
Issue SSL Certificates through AWS Certificate Manager (ACM).

4.1 Request Certificate through ACM

We go to AWS ACM and reuqst a certificate. Enter both environment subdomains (=One certificate for both domains).

Once you have added the needed DNS entries in order for ACM to do the Domain verficiation your certificate will soon be issued.

4.2 Add DNS CNAME entries for your app subdomains

In order for our certificates to work we need to create the subdomains in Route 53 and point them to our app domains.

app-sandbox.mycompany.foo
→ app3000-sandbox.eu-central-1.elasticbeanstalk.com
app-live.mycompany.foo
→ app3000-live.eu-central-1.elasticbeanstalk.com

Do this by creating a CNAME record per subdomain.

4.3 Add HTTPS Listener to App Environments

Now we can use the SSL Certificate and modify our app environment (do this for live and sandbox) to have a port 443 HTTPS listener.

Add the HTTPS listener on Port 443 and select the certificate.

The config should look like so now.

Once you updated the config you can access your app through https://app3000-sandbox.mycompany.foo and ACM will always provide a valid SSL certificate for you. Nore more expiring and reordering of certificates :)

X. Final Thoughts

This was fun. I like the simplicity of this approach and that not so many things are involved. Basically your code is hosted on GitHub. GitHub Actions deploy your code to AWS. So only two things to know about. I like it very much and think this way deployments for our microservices are now on a modern and stable basis. I like the most that I do not have to build and push the docker image myself :)

Passion, friendship, honesty, curiosity. If this appeals to you, Comsysto may well be your future. Apply now to join our team!

SHOW ME THE JOBS!

This blogpost is published by Comsysto Reply GmbH