For some time already we delivered several projects to our clients that consisted of architecture that was very similar to micro-services. We used separation primarily between presentation layer (web site), backend system (REST API) and database.

We were treating every part as an individual unit — website contains only presentation layer, REST API (sometimes several REST APIs) are handling all business logic. Database — sometimes one, sometimes many (depending on how we executed separation of concerns).

During development and testing we are using Docker. Docker is giving us confidence when testing that we are on the same “environment” as on production.

For deployments to our central TEST environment we are using docker-compose to quickly start up whole environment with all services. This method is pretty much straight forward — we specify what services to run (images, mapping volumes, etc.) and we define what ports to expose from docker container to the host system. Then we map it in our internal DNS and we have nice working access to our TEST environment.

Multiple sub-domains with different deployed versions

On our latest project, requirement was to have (after first phase):

- multiple environments running with different versions;
- every environment running on its own sub-domains;
- all of those environments are supposed to be covered with proper SSL/TLS certificate.

For example: env1.coolapp.com would contain release from sprint 10, but env2.coolapp.com would contain release from sprint 11. At one point in time env1 would be updated to sprint11 (or even 12, depending on the need).

Step 1: docker-compose with .env file

When using docker-compose you can create .env file (that is of course just default name, you can use different but then you have to specify it when executing docker-compose command).

.env file can contain “variables” that will be loaded in docker-compose (very similar to “environment” variables).

This gives us an option to have the same docker-compose file but just having different .env file we can point to different image version and give different name to the environment.

Step 2: Reverse proxy

We needed some kind of reverse proxy that will accept traffic coming to specific sub-domain and route that traffic to its appropriate docker environment.

Reverse proxy can be executed in many ways, we can make custom service, we can use NginX, but it would be really cool if there would be some kind of already existing tool that can give us easy configuration, dynamic discovery of new subdomains and if it could also automatically obtain SSL/TLS for every new environment … oh, and when I am writing my wish list, I would also like to include — load balancing. Yeah, that would be nice…

Traefik

Traefik (traefik.io) is wonderful piece of software written in Go language that gives everything that we need, and it can do much much more.

In its essence it is dynamic reverse proxy. It can connect to many popular deployment platforms (docker, swarm, mezos, kubernetes, etc.) and obtain information about services (containers). It is using .toml file (simple text config file) for configuration.

Traefik is composed of rules that are used to connect “Frontend” with “Backend” In terms of Traefik — “Frontend” is internet domain like api.myapp.com. On the other hand “Backend” is our deployed web service. In this case — we can set a rule in Traefik that for Host:api.myapp.com traffic should be routed to our api service container.

All those rules can be set in .toml file. But here comes a very interesting part — they don’t have to be! Rules can be defined in labels on docker containers and Traefik will pick them up dynamically.

Set docker service labels to “push” rule into Traefik

In our case we have a service defined in our docker-compose and that service will have a docker labels with content like (there are lot more labels, this is just a small sample):

labels:
      - "traefik.backend=restapi"
      - "traefik.frontend.rule=Host:api.coolapp.com"
      - "traefik.enable=true"
      - "traefik.port=8080"

This will mark this docker service as a backend with name “restapi”, it will create a rule in Traefik that all traffic coming from api.coolapp.com should be redirected to this docker service to port 8080.

In some cases you don’t need to expose some docker services to Traefik (for example you can have some backend api that doesn’t need to be exposed) then you just omit “enable” label.

Start up Traefik

We start up Traefik as separate docker container from our environments. When we start it up we map docker.sock so Traefik can communicate with Docker, but also we give it very simple .toml file:

logLevel = "DEBUG"
defaultEntryPoints = ["http", "https"]

# WEB interface of Traefik - it will show web page with overview of frontend and backend configurations 
[web]
address = ":8080"

# Connection to docker host system (docker.sock)
[docker]
domain = "mycoolapp.com"
watch = true
# This will hide all docker containers that don't have explicitly  
# set label to "enable"
exposedbydefault = false

# Force HTTPS
[entryPoints]
  [entryPoints.http]
  address = ":80"
    [entryPoints.http.redirect]
    entryPoint = "https"
  [entryPoints.https]
  address = ":443"
    [entryPoints.https.tls]
  
# Let's encrypt configuration
[acme]
  email="email@mycoolapp.com"
  storage="/etc/traefik/acme.json"
  entryPoint="https"
  acmeLogging=true
  onDemand=true
  OnHostRule=true

Beginning of the file is mostly self explanatory — it is setting log level, enabling both http and https, couple of lines that makes it connect to docker host and finally “force to https” (all traffic that comes to port 80 — redirect to 443)

But last part is really neat — acme. This is connection to Let’s encrypt service, but the best part of all — it is completely dynamic. For every Host rule (domain/sub-domain) that appears in Traefik — it will go to Let’s encrypt and obtain key and certificate (store them in acme.json file) for that host configuration.

As you can see here — .toml configuration doesn’t have any configuration about our sub-domains and docker-compose environments. It just simply “waits” for any incoming rule that will be “pushed” when docker-compose starts up new environment. Then it will generate all rules “on-the-fly” and (if necessary) obtain SSL/TLS certificate for every environment.

Load balancing

This is simple — it is just there “out-of-the-box” load balancing works, you can specify different load balancing algorithm to be used, but if you don’t specify anything and you scale up your docker-compose service, load balancing is up and running.

Instead of conclusion

Traefik is solving our problems really good. It is built with dynamic environments in mind, so lots of things just work out of the box without configuring too much. Other features like let’s encrypt, load balancing, circuit breakers etc. just makes it even more appealing.

We still have a lots of ways to go, to explore more features, try it out in docker swarm environment, etc. but for now we are very satisfied.

Give it a try, if nothing else, at least because of cute logo :)

Having proper continuous integration flow is the key of reducing regressions, finding bugs as soon as possible, keeping main code base clean etc.

But before we start, lets try to answer a question:

What is continuous integration?

Continuous integration as a term was coined in early 1990s to mark idea that developers should often merge their working code with main code base.

It was introduced as a way to avoid long running development jobs without proper checking against main code base that can lead to hair-pulling merges that can produce lots of build fails and long fixing times on something that is already done.

Simple idea, but it has big implications

If you want to perform successful continuous integration, you need to cover several areas

Fast development time

Development tasks (features) should be defines/split in a way that development of one feature takes least possible time. This will ensure short development times and often merges to main code base. Potential problems/regressions are reduced because new items are just small chunks of work.

Automate tests

It is difficult to cover 100% of any application in completely automated tests that can cover all possible and impossible combinations. But you should always aim to cover as much as possible. There are lots of different testing frameworks that can help you cover back-end and front-end parts of the application.

Test environment should be as similar as possible to production

Test environments should be configured in a way to mimic production environments as much as possible. Same version of OS, hardware platform, pre-installed software, configurations, etc. This step should eliminate lots of cases of “It works on my machine…”

Automate builds

This step ensures continuous part of term Continuous Integration. There are several software platforms that perform automated builds. One of most opensource platform is Jenkins. Job for CI platform is to detect changes in code base and automatically perform build and testing actions.

INFOdation CI flow

Everything start from job planning

At INFOdation we are aiming to split development jobs into tickets that shouldn’t take more than 2 days to complete. All automated tests are defined at the beginning, so developer knows what kind of tests he need to create to cover business logic parts.

For job planning we are using JIRA and for development documentation we are using Confluence.

GIT for successful code management

For code management we are using GIT, specifically Bitbucket where we keep our central code repositories.

In our everyday development flow we tend to use popular GitFlow method.

Quick word about GitFlow

We are using 2 main branches: master and develop.

Develop branch is used in everyday work. Master branch contains only finished versions that will be published/deployed to end customer.

• When developer picks up ticket to work on, he starts from latest develop branch and creates feature branch with specific naming pattern: feature/<ticket_name_from_jira>. This ensures easy tracking on what is in progress when we look at repository.
• Work on feature is finished, code from feature branch is merged with develop branch. Any eventual merge conflicts are resolved on feature branch before merging.
• When sprint is close to the end, we create release branch where we perform final tuning, stress testing, eventual bug fixing etc.
• After release is successfully tested, it is tagged and merged to master and develop branch.

Jenkins for automated builds

We are using webhook mechanism of Bitbucket that posts information to our Jenkins CI system that new code is available on central repository.

Jenkins is set up to take latest code from Bitbucket and perform build and test. Specific configurations differ from project to project, depending on requirements and environments.

Usual configuration is that Jenkins will build and deploy from develop and release branches for additional manual user testing, but build jobs that are building feature branches are configured in a way that they will only perform build and automated tests (no deployment).

Code reviews on pull request

Pull request is very important task of keeping the main code base (develop branch) clean.

Developer will do his development on feature branch, perform testing on his development platform, also Jenkins will execute build and automated tests.

But developer cannot merge directly to develop branch. Instead he must open Pull request. This is notification that development is tested and ready to be merged. This pull request will be sent as notification to set of users who are defined as reviewers. It is their job to do last minute check on the code to make sure there are not some potential issues.

Getting everything together

Merging (approving) pull requests can be tiresome job if you don’t have any good help with valid insights. That is why having several useful plugins in Jenkins platform can help reviewer with information if build is good and how many tests are passing before reviewer even have to “dive into” the code.

Also there are useful integrations among different platforms that can make progress tracking much easier and give confidence that all code is reviewed.

Jenkins plugins for test results and code coverage

There are multiple plugins that can collect test execution results and code coverage reports.

On this picture you can see sample JAVA application that is using JUnit plugin to show test execution and JaCoCo plugin to show code coverage:

Jenkins Bitbucket build notification plugin

With this plugin, Jenkins results are pushed to Bitbucket and provide information if specific git commit is good or bad.

Here you can see example of pull request page where build status is shown from Jenkins:

If you click on that link it will show you details of build. Here you can see which job is showing status with total number of passed tests. You can follow link to open direct Jenkins page of the build job:

Beside showing build status on the pull request, this status is also shown on list of active branches and commits, so with just a quick glance on the repository you can easy see if development is on track:

Bitbucket and JIRA integration

Because both Bitbucket and JIRA are products of the same company (Atlassian) it is expected to have some form of integration.

On Bitbucket it is shown in form of linking JIRA tickets if you properly reference them in commit message (seen on previous picture as links in commit messages). This way when you click on that link you will be transferred to details of that JIRA ticket

On the other hand, on JIRA on every ticket details there is section Development that shows references to specific ticket (number of commits, branches and pull requests).

This also transfers to Releases section on JIRA where tickets are grouped in their versions and gives good overview progress per different version. There you can see very important Warnings section that shows tickets that are mentioned in Bitbucket, but are not part of any Pull Request. Those tickets can pose major stability issues, because if they were not part of a Pull Request, then they are probably not properly tested and/or reviewed.

Potential issues

CI is not of much use if it code is not covered with automated tests. Tests can be very simple, but also very complicated and take lot of development time.

Also, CI must be configured in a way to perform tests on a platform that is close to production as possible. Those configurations can be very difficult to create and maintain.

Conclusion

CI systems are vital in ensuring that code base is kept good after new development tasks.

Good CI systems must be able to properly integrate into development workflow. At INFOdation we found very good combination of JIRA, Bitbucket and Jenkins. There are lots of useful plugins in Jenkins that can ensure build results on every code change and show status back on Bitbucket.

Be aware that you will need to spend some time in devising proper testing strategies and also time will need to be invested in proper CI configuration and maintenance.

If you overcome those obstacles, your everyday development process (and tracking) will become significantly easier and you will be able to focus on actual problem solving.