Continuous Deployment — vehicle that drives the technological scaling of new-age companies

Our thoughts on continuous deployment, emerging trends and opportunities in the space

The developer-led market has been growing at a tremendous pace with over 1000 companies building more than 1250 products and raking in ~$49B in ARR [1] in 2021. Around 117 companies were born in 2021 alone and raised ~$1B in the same year. In fact, the private capital flowing into the market also expanded from $21B in 2020 to $37B in 2021. The value realization also followed into the public markets with successful IPOs of Gitlab, Hashicorp, Confluent, Digital Ocean, Agora, and JFrog. With the broader trend of digitization and all companies adopting the tech-first approach to improve their operations, processes, and customer experience, the developer-led market is only beginning its growth journey.

Software development has largely evolved through the years — from developers working on a single code base, converting into an executable and deploying on a computer/server (waterfall methodology) — developers working on different modules in parallel, converting the multiple code files into an executable for the users (agile methodology) — developers building their own modules in parallel and owning the deployment (DevOps methodology). The individual modules in the DevOps methodology are known as microservices.

Breaking down an application into microservices is a necessity to deal with the increasing complexity and the temporal variation in traffic across different functions in an app (optimizing server costs). Furthermore, each piece of functionality can follow its own development, testing and deployment cycles. And in case of a bug, the problem can be addressed by rolling back a single microservice as opposed to taking down the entire application. However, microservices can be built in different languages, requiring distinct libraries, and having varied dependencies. One approach is to use virtual machines to install multiple OS on a single server — which can be an inefficient (each OS will have its own memory, CPU requirements, etc.). Furthermore, in case of a microservice update requiring a new version of libraries, each server’s VM will have to be updated (a tedious task prone to errors and leakage).

Containerization addresses the problems associated with VMs, wherein an app and its required environment, libraries, and dependencies are isolated in containers which can be deployed anywhere having a standard OS and Docker. Therefore, the microservices are isolated from the infrastructure, commoditizing it in the process and the need to update servers becomes minimal. The development, testing and deployment cycles of microservices (and containers) are brought together by an underlying CI/CD pipeline infrastructure. The methodology is envisioned to automate all the steps following a code commit right up to deployment (service going live for the customers).

At Pravega, we have been closely following the developments in the microservices/containerization area and its implications on CI/CD. This note is the second in the DevOps series (check out our previous note on Testing in the CD Pipeline [2]) — sharing our observations on the modern-day software deployment processes and the emerging trends.

There are 7 pillars of modern software deployment (CD) — Provisioning infrastructure, configuring infrastructure, App image creation (preparation for containerization), Management of packages/images, CD Pipeline automation and monitoring, Application configuration management (container orchestration), Deployment & Rollbacks.

• Before deployment, a given server hardware (cloud or on-premise) needs an OS to be installed, a private network (or VPC) to be setup. Any required software like Java, Docker, or Kubernetes needs to be installed. Security rules, firewalls, users and permissions need to be created. All of these processes come under the umbrella of Infrastructure Provisioning.
• Configuring infrastructure includes taking decisions on distributing microservices, worker nodes (in case of Kubernetes) on servers (managing the core availability, RAM requirements, etc.). Apps are also deployed (installed) on the servers in this step.
• Creation of an app image inclusive of all dependencies which can be linked to a container and is deployable on generic infrastructure.
• Direct the created image into a repository to manage their life cycle — Manage the versions, copies of the images, act as a central directory for containers to point and download the images, monitor their status. App updates can be made by simply replacing an older version with a newer image version and images can be deleted (along with corresponding container deployment yaml files) to remove certain functionality from the app.
• Managing and automating the workflows post integration until the app is deployed in the production environment.
• Manage the deployed containers hosting the relevant app images and the underlying infra — versions deployed, number of replicas, internal services network, traffic distribution, etc.
• The process of releasing or updating an app using a deployment strategy — Rolling deployment, Blue-Green deployment, Canary deployment.

The three emerging trends that we observe in the deployment space are discussed below:

Abstracting Infrastructure:

Companies historically owned the infrastructure (server hardware) and hired systems engineering teams to provision and configure infrastructure based on the deployment cycles. With cloud computing (AWS, Azure, GCP, etc.), the server hardware and the first layer of software (OS, VPC, etc.) was abstracted. The next generation of tools (Infrastructure-as-code) abstract the higher layers of installing relevant libraries with declarative commands to configure cloud servers based on the requirements.

• As the complexity of applications increases (number of microservices), so does the underlying infrastructure. Managing the multiple Terraform, Ansible, etc. scripts alongside the development in the app as the team size increases can result in mismatch across the staging and production environments.
• Infrastructure-as-code tools are stateful in nature. Therefore, any partial updates/changes can create a configuration drift.
• Erosion of infra systems — server disks filling up with log files, failure of underlying hardware, etc. causing applications to crash
• As the adoption of cloud increases and layers of infrastructure abstraction are added, the physical disconnect between a company and its deployed servers has been growing. Application deployment, monitoring, update, etc. also happens wirelessly through the internet posing an associated security and compliance risk (leakage of data, cost scaling, bugs to drive attacks, etc.).

The upcoming tools create the next layer of infrastructure abstraction:

• Infrastructure-as-code management platforms that integrates and executes workflows across IaC tools (Terraform, Ansible, Pulumi, CloudFormation, etc.). An added layer can also create scripts to automatically generate infrastructure involving Containers, RDS, Object Storage, API Gateway, Web apps, etc. [Spacelift, Digger.dev, etc.]
• Infrastructure abstraction to allow developers to deploy apps without prior knowledge of DevOps [Railway.app, Shipa.io, Jelastic, etc.]
• Templatization of infrastructure (elements can also be added/removed like legos) based on the type of app being developed (blogs, chat bots, etc.)
• X-as-a-service (X — Kubernetes, Container, Application, Database) and Application-as-code across multi-cloud (hybrid, private, public, etc.) to abstract the application from the infrastructure. In such solutions, the developer experience remains the same irrespective of the underlying deployment infrastructure.
• Tools to automate the security and compliance of Infrastructure-as-Code (IaC) scripts by performing static and dynamic analysis, policy and cost checks before scaling. Furthermore, a digital twin of the cloud configuration can be created to simulate any infrastructure change and verify its impact before deployment in the production environment. [Soluble, Lightlytics]
• Tools to identify security design gaps by reading the IaC files. Any configuration drift can be identified and sorted in a pre-production deployment. [Oak9]

Kubernetes:

The open-source container orchestration tool was released in 2015. It’s been ~7 years and yet the adoption of the tool has only been gradual even when the tool is widely agreed to be the state-of-the-art at scale. Speaking to 20+ tech-first companies, we have found two major bottlenecks in the adoption of Kubernetes:

• The sheer complexity of creating, managing, and operating in a Kubernetes system requires teams to work on a bunch of modules (Ingress, Proxy, Service Mesh, Secrets, etc.) which is generally irrelevant to the core business of a company.
• The steep learning curve of Kubernetes that teams need to invest in to shift from auto-scaling groups or manually managing containers to an automated container orchestration solution.

The upcoming tools aim to address the above listed problems by handling certain aspects of the Kubernetes system. While solutions exist today providing managed Kubernetes services (Amazon EKS, Azure KS, Google KE, etc.), they also have the issue of vendor lock-in. Moreover, enterprises are adopting a multi-cloud hybrid approach (public + private servers) and require a flexible Kubernetes system. [Akuity, Space-cloud, RedKubes — Otomi, etc.]

• Tools to manage the CD of applications in a Kubernetes system and abstract all configurations (from the control panel as opposed to individual YAML files), and rollouts (blue-green, canary deployment strategies, etc.). Furthermore, workflows with parallel jobs on Kubernetes can be created (UI based) and extended to event-driven triggers to create Kubernetes resources, send notifications, etc.
• A Kubernetes system can use multiple secondary apps to handle parts of the orchestration process — the likes of Kong, Vault, Istio, Harbor, Prometheus, Grafana, etc. A secondary layer of tools to manage and control the entire orchestration system from a single place with role-based access control.

CI/CD Orchestration

CI/CD has come a long way both in terms of the offerings and the adoption. The gaps and pain points associated with the existing CI/CD tooling are as follows:

• Various environments (development, staging, production) need to be created, managed, and updated as the application evolves. While the CI/CD pipeline automates the lifecycle of the application and infrastructure itself, the environment files need to be manually created and maintained.
• With the number of tools available across the CI/CD pipeline growing exponentially, choosing, and integrating the most-suited tools (to create the tech stack) is a frictional process and requires iterations — can be a costly affair especially in terms of the time spent.

The upcoming tools expand the scope of CI/CD or build a secondary layer to orchestrate the CI/CD pipeline itself [ReleaseHub, Bunnyshell, Opsera, ReleaseIQ, Architect, Argonaut, etc.].

• Environment-as-a-service to create on-demand development, staging and production environments. The environments are auto generated by taking the app repository as the input and can be integrated with the CI/CD pipeline to create ephemeral environments as and when required.
• Creation of a CI/CD pipeline (tools for configuration management, CI, repository, security, log, QA, CD, monitoring, etc.) through drag-and-drop to automatically integrate the selections and build the tech stack. The secondary layer also allows monitoring and control over the entire pipeline from a single point.

A tech-first company’s growth comes along with increased complexity in their applications and correspondingly in their tech stack. Some of the frictional transitions include moving to a microservice architecture from monoliths, followed by containerization and the adoption of container orchestration tools for scaling. Developments in the deployment domain along the lines discussed above are easing the technology transitions in companies allowing them to focus primarily on scaling their core business.

If you are a founder building in the DevOps/SaaS space, we would love to hear from you. Please feel free to reach out to Kanishk to brainstorm, discuss developments or opinions at kanishk@pravegavc.com

References

1. Developer-led landscape 2021 (Link)
2. Testing in the CD Pipeline (Link)