What a Developer Should Know About DevOps?

Published in

SDA SE Open Industry Solutions

10 min readJun 9, 2022

Imagine a world …

Imagine a world where you can change essential pillars of your life, try new things out or fix important issues without accidentally messing everything up. An expert is having a close eye on what you have done before this actually hits real life. Wouldn’t that be cool? Unfortunately, this is not (yet?) possible.

However, in software development, this is what we are doing to our codebase almost every day. We are proposing changes and having them reviewed by experts to verify them, running automatic tests to have another layer of security & validation, and having them smoothly implemented into our codebase. Why not have the same safety net for your infrastructure? This is what the GitOps concept allows us to do.

What additional benefits does it entail? Why should you care? And what should a developer know about DevOps? Let’s find out!

Credits: A huge thanks to Nana Janashia and her team. They have built-up an awesome Youtube channel called TechWorld with Nana to make DevOps approachable for everyone!

The Infrastructure as Code concept

Before the concept of Infrastructure as Code (IaC), System Administrators were essentially responsible for manually setting up servers, configuring networking, creating route tables, installing software, configuring this software, installing the database, etc. in order to enable an application to run on the server. Additionally, if you had multiple environments like test and production, the workload grew respectively without even counting in the actual maintenance of these environments.

With IaC, it is possible to automate all of these tasks. Writing and executing infrastructure specifications as scripts will do the trick, as the name Infrastructure as Code already suggests.

Automating these tasks results in numerous benefits like configuration consistency, faster deployments, decreasing risks of human error, and cost savings just to name a few.

The concept can be broken down into three main categories:

Infrastructure provisioning
Configuration of provisioned infrastructure
Deployment of application

Later in this article, we will have a brief introduction to tools that cover each of the categories.

How not to work with IaC?

It already seems to be a giant leap forward using this concept. However, without a proper process, IaC can also have several pitfalls.

Imagine creating and executing the scripts required for infrastructure from your local machine. This will have obvious flaws like lack of version control, possibly no transparent source of truth, collaboration issues et cetera.

Another example is using Git as a centrally accessible place to provide a single source of truth for the code base, but not using it to its fullest potential by skipping a review process, collaboration possibilities, and automated tests. Instead, you are pushing changes directly to the main branch. In addition, the updates on the infrastructure still have to be applied manually. Down the line, this can also lead to considerable negative impacts on the infrastructure’s stability and raise security concerns, since the infrastructure needs to be accessed through the local machine.

This is where the GitOps concept is coming to your rescue. It’s making use of the best practices for applying Infrastructure as Code in your organization or project and as you may have guessed already, it’s addressing all the issues mentioned above.

The GitOps concept

In GitOps, we treat the infrastructure code the same way as the application code. This involves a separate Git repository for the infrastructure code and a full DevOps CI/CD pipeline.

A visualization of the GitOps flow represented by two branches that are merged at the end. — The GitOps flow.

The illustration above refers to the GitOps flow. It demonstrates the different stages a change is going through. At first, the changes are made to the code according to the desired outcome and a pull request is created. Subsequently, the CI pipeline validates configuration files and runs automated tests. A review by an expert will follow to check code quality and consequently approve them. Lastly, the CD pipeline implements the changes to the deployment environment. This looks familiar, doesn’t it? The benefits of such a comprehensive flow comprise:

Having a version control system
Having a single source of truth
Enabling an easy rollback of changes
Having well-tested and reviewed changes
Enabling collaboration
Providing empowerment — so even Junior Developers can safely propose changes to the infrastructure
Providing transparency
Providing automation through a fully-fledged CI/CD pipeline

Together with a modern CD pipeline, this concept can be a real blessing for every organization.

CD pipeline

In IaC environments, we have two options to automatically apply changes to the infrastructure — the push and the pull based deployments.

In push based deployments, the pipeline is building and pushing the new version to the deployment environment. This means that you have to set up your pipeline using tools like kubectl (the cli for Kubernetes), configuring access to the cluster and to the cloud platform, which means the secrets are stored outside of the cluster providing an additional hypothetical point of attack. Furthermore, you have no visibility of the deployment status.

In pull based deployments, an agent is installed in the deployment environment (e.g. Kubernetes cluster) and pulls the new version from the Git repository. This is done through constant monitoring and comparing the desired state in the Git repository with the actual state in the deployment environment. As soon as changes are detected, the agent is taking care for ensuring the actual state is equal to the desired state. An example agent in this context is Argo CD. With this approach, only Argo CD needs to be configured to apply changes, which greatly enhances the environment’s security and reduces the permissions to manage. Moreover, Argo CD provides constant visibility of the deployment status which comes in handy when e.g. debugging or monitoring your environment and also detecting required changes.

To put it into a nutshell, the pull-based deployments offer multiple benefits to facilitate the management of your deployment environments and are a key concept for a secure, stable, and highly available CD pipeline.

Common tools to implement the IaC concept

In order to embrace the power of the GitOps concept, we require the help of some tools that are taking care of different steps within this flow. In the following, I will provide you with a brief introduction to some of the most common ones (non-exhaustive). However, there are also well-designed alternatives and more tools for a wider range of functionality.

Docker

If you have already heard of containerization in the field of software development, you have most certainly also heard of Docker. It is the most common and widely-used tool to containerize your application. But what is a container and why is it important to us?

A container is a way to wrap applications with all necessary dependencies and configurations into a package. In Docker, it’s based on a blueprint called container image, which is an artifact that is portable and thus can be easily shared. This significantly increases efficiency, since the application works on all machines similarly. It provides an isolated environment with no configuration on the server needed.

As mentioned, Docker containers are based on images. An image is a read-only template that is generated from a so-called Dockerfile and contains all information necessary for the container to run like the application itself, dependencies, and processes when running.

If you are new to the concept and want to learn more about it, I can recommend great resources like the article “A Beginner-Friendly Introduction to Containers, VMs and Docker” from Preethi Kasireddy or “Docker Tutorial for Beginners [FULL COURSE in 3 HOURS]” YouTube video from TechWorld with Nana covering this topic in detail and providing beginner-friendly hands-on examples.

Container images are stored in a special container image registry. These can be private registries as often used by any size of an organization and private users or public registries like Docker Hub which provides official application images like node, jenkins, mongoDB and a lot more.

To break down the difference between an image and a container, you can have a look at the sections below.

Image

Actual package of dependencies, configs, etc.
Artifact that can be moved around
Blueprint for the container
Not running

Container

Start and run the application
Container environment is created and runs from an image

Images consist of one or more layers which then result in the final image you are building. Each layer represents an image. These images are based upon each other. This means that when you want to containerize for example a Node.js application with Docker, you will probably use a lightweight version of node like node:lts-alpine as the base layer for your application image. Layering allows us to avoid redundant information transfer and increases build step efficiency, since only not available or changed layers have to be updated.

After you have worked your way through the resources mentioned above, you can follow this helpful guide written by snyk on how to create a secure Docker image with all the best practices.

To better grasp what is happening until the containerized application is actually deployed, you can find an illustration below.

Process visualization showing the steps of the Docker flow until deployment of a containerized application. — The Docker flow.

After you have built your Node.js application or adjusted it, the CI pipeline builds the app and creates a Docker image. It contains all necessary dependencies, the application itself, and instructions on what to run in the container. The image is based on the provided Dockerfile that lives inside your app repository. It is then pushed to your image repository where it gets pulled from the production server. Let’s say you additionally have configured a mongoDB database. As mentioned above, these kinds of official application images will be pulled from Docker Hub. As you can imagine, the switch to a previous version of your application is just an image reference away. This is another reason why Docker really is loved among the Software Development community and you should have at least a basic understanding of how Docker works and how to containerize applications.

The trend from monolithic to microservices architectures was a driving factor to focus more on the concept of containerization. But what happens when you have to manage dozens of containers in your production environment? When one of your containers does fail? Or when the containers have to communicate with each other reliably and on a regular basis? This is where Kubernetes comes into play.

Kubernetes

Kubernetes, also known as K8s, is an open-source system for automating deployment, scaling, and management of containerized applications.
— Kubernetes’ official website

K8s offers high availability, scalability, high performance, and disaster recovery. That sounds astonishing and it is. This is achieved by several cool features like self-healing containers, load balancing, secret and configuration management, storage orchestration, and many more.

A running Kubernetes system is called a cluster. The cluster is made up of nodes that run containerized applications. Each cluster has at least one master node named control plane that manages the worker nodes and pods of the cluster and is your point of entry to manage the cluster. If a node is added or removed, the load will be distributed across the nodes.

A namespace is used to group resources within a cluster. These groups are isolated from each other. In a sense, they create a virtual cluster within the cluster. Among others, this enables multiple teams to use the same cluster without having to deal with e.g. naming conventions and keeps the cluster organized.

A node represents a single physical or virtual machine located on-premises or hosted by a cloud provider. It runs one or mode pods, has dedicated CPU and RAM resources, and serves as a layer of abstraction to not be concerned about the location or other characteristics. This allows any node to substitute another. As mentioned above, you have worker nodes that do the actual work and master nodes that manage the worker nodes, balance the load, check the health status, manage information about the complete cluster, et cetera. You can probably already guess that they are quite important. Thus, you normally don’t run a cluster with just one master node, but at least two.

A pod is a containerized application. It is also defined as an abstraction layer to not only focus on Docker as containerization technology but to make the design agnostic.

To support your imagination with some visual content, you can find a very simplistic illustration of a K8s cluster below.

Kubernetes consists of many more components with a wide variety of functions. For a detailed overview, I highly recommend the “Kubernetes Crash Course for Absolute Beginners” from TechWorld with Nana.

As a Software Developer, you will certainly benefit from having a basic knowledge of the main components and functions of k8s, if your organization is using it together with the GitOps concept.

Terraform

To complete the picture, I will briefly introduce you to a neat open-source tool that is concerned with the first two IaC categories “Infrastructure provisioning” and “configuration”. It is called Terraform and is used to create, change and replicate infrastructure. With more than 1700 dedicated providers, the tool is able to interact with an enormous range of infrastructure services and resources and thus, is able to automate a large variety of infrastructure provisioning and configuration use cases.

Most commonly, Terraform works in three phases:

write - define your resources declaratively
plan - create an execution plan (determines what actions are necessary to achieve the desired state)
apply - execute the plan

To learn more about Terraform, the creator organization HashiCorp provides well-designed tutorials and use-cases on the official Terraform website Terraform Tutorials — HashiCorp Learn.

It’s really helpful to know about this tool and eventually checkout some use-cases, but as a Software Developer and depending on your organization, you will rarely get in touch with Terraform directly.

I hope that this article has given you useful tips and insights into the world of DevOps that you can apply to your development process. If you’re interested in working with the tools and techniques presented, check out our job postings on Home | Dein Job bei SDA SE Open Industry Solutions and join the SDA SE team!