Harmonizing the Cloud: Unveiling the Secrets of Platform Engineering and CNOE

Introduction

Nowadays, in the most complex corporate organizations, composed of various divisions and development teams, the need for a standardized approach to software development is increasingly growing. It is unthinkable for thousands of developers to each develop using their own model. This would complicate the delivery of applications, and the time to market would become significantly prolonged. The discipline of Platform Engineering emerged to address this issue.

What is Platform Engineering?

Platform Engineering is the discipline focused on defining standards and workflows for software release and managing its entire lifecycle. In short, it is the discipline responsible for defining the design principles of an Internal Developer Platform (IDP), a self-service development platform made available to all developers within a corporate organization.

Key features of an Internal Developer Platform

In this section, we will analyze the key features that an IDP (Internal Developer Platform) brings:

• Centralized Tooling and Services​: An IDP provides a set of centralized tools and services, allowing developers to easily access all necessary resources from a single platform. This reduces tool fragmentation and simplifies the development process, improving operational efficiency and reducing the time needed for resource setup and management.
• Self-Service Capabilities:​ One of the key features of an IDP is the ability to offer self-service to developers. This means that teams can independently create, configure, and manage their development, testing, and production environments without waiting for operational team intervention. This approach accelerates the development cycle and increases team autonomy, enabling faster application releases.
• Automation:​ Automation is a crucial element of an IDP, enabling the automation of repetitive and complex processes such as infrastructure provisioning, configuration management, and application deployment. Automation reduces the risk of human error, ensures consistency, and frees developers from manual tasks, allowing them to focus on higher-value activities.
• Security and Compliance​: An IDP integrates security and compliance measures from the start, ensuring that applications are developed and deployed in accordance with corporate and legal regulations. The platform provides centralized access controls, identity management, data encryption, and continuous vulnerability monitoring, protecting data integrity and ensuring that all operations are traceable and secure.
• Observability and Monitoring​: The platform offers advanced observability and monitoring tools that allow teams to have a complete view of the performance of applications and the underlying infrastructure. These tools facilitate timely detection of issues, root cause analysis, and proactive resolution, ensuring that applications are always available and performing well.
• Collaboration and Communication​: An IDP facilitates collaboration and communication between different development, operations, and security teams. By integrating work management tools, knowledge sharing, and real-time communication, the platform promotes a collaborative environment where teams can work synergistically, reducing barriers and improving the overall efficiency of the development process.
• Scalability and Flexibility: An IDP is designed to be scalable and flexible, supporting organizational growth and the evolution of business needs. The platform allows resources to scale dynamically, both vertically and horizontally, and to quickly adapt to new technologies and development methodologies, ensuring that the infrastructure remains aligned with business requirements.

A framework for an Open Source Internal Developer Platform: CNOE

Now, in this paragraph let’see an Internal developer Platform Reference Architecture and its components.

Fig 1. Platform Reference Architecture

Main components of the architecture:

• Developer Control Plane: It is the layer dedicated to developers. It includes, among other things, the dev portal (through which developers can provision their cloud-native environment in just a few clicks) and the Git repositories.
• Integration & Delivery Plan: This plane is about building and storing images and artifact. This layer is composed by CI/CD pipeline, a Container Registry and a Platform Orchestrator.
• Monitoring & Logging Plane: Provides real-time metrics, traces, and logs for both applications and infrastructure.
• Security Plane: This layer is responsible for platform identity and access management, as well as securely storing sensitive data in secrets.
• Resource Plane: It encompasses all the resources where application delivery is performed. This includes the Kubernetes engine, cloud provider objects created through Infrastructure as Code. It also includes the data domain and all the network infrastructure necessary for the operation of the applications.

Cloud Native Operational Excellence (CNOE) is an approach to the management and operation of cloud-based systems designed using cloud-native principles. It leverages cloud services, containerization, and orchestration tools like Kubernetes to optimize resource utilization, scalability, and application reliability. Key advantages of CNOE include enhanced operational efficiency, improved system reliability, and scalability. By incorporating practices such as thorough monitoring, automation, continuous integration/continuous deployment (CI/CD), and change management, CNOE aims to maximize the performance and availability of applications in a cloud-native environment. This approach enables organizations to harness the full potential of cloud infrastructure, ensuring a seamless and efficient operation of their systems.

Let’s see a reference architecture of an IDP based on CNOE:

Fig 2. Reference architecture based on CNOE and Open source tools

CNOE reference architecture is based on Open-Source technologies. In this table you can see the mapping between capability and technologies in CNOE.

Table 1. Capabilities and Tools/Technologies

Thanks to the open-source and flexible nature of CNOE, you can leverage any tools or technologies to build your own Platform Reference Architecture.

Tenets of CNOE

• Open Source First: use of open source technology is prioritized over proprietary technology for each of the technology verticals discussed later in the doc. This helps ensure alignment across all the participating members by allowing them to coordinate on collaborations while having the freedom to update and modify a given technology to their needs.
• Tools & Not Practices: CNOE offers suggestions on which tools to use and with what configurations. What practices a given company builds around and above those tools is out of scope for CNOE.
• Standardized & Customizable: CNOE aims at addressing the usability requirements of its stakeholders. While the requirements of the platform could be enforced by the security engineers and infrastructure operators, the usability of it needs to be guaranteed by platform operators and application developers.
• Community Driven: Decisions on the direction of the working group is driven by the community and its governing body. This involves the selection of technologies, level of commitment, and level of contribution.
• Powered by Kubernetes & CNCF: The CNOE working group relies heavily on the success of the CNCF community to choose technologies that are deemed useful to the type of operations required by the community. As such, Kubernetes is considered the de-facto environment to operate CNOE tooling. However, choosing of Kubernetes as the operating environment, does not require for it to be the environment to orchestrate against. Using the right infrastructure as code tooling, the CNOE community can choose to orchestrate against any compute platform of their choice.
• Build to be Shared: All CNOE deliverables including the reference architecture and the deployment packages will be developed out in the open and by collaboration of all its participating members and with the goal of making it sharable and usable by the larger open source community of interest.

Conclusion

Platform Engineering represents a natural evolution of other software lifecycle management paradigms, including DevOps. With an Internal Developer Platform, a corporate organization can establish a development standard for all its developers. The benefits of an IDP are numerous, including increased development team productivity, consistency, faster time to market, improved quality, and cost efficiency.