OpenShift Container Platform: A Holistic Approach to Container Security

Buckle Up!

Secrets as a Feature

Docker Inc’s introduction of secrets into Docker Datacenter is a welcome and expected development. The Kubernetes community has had this capability for years and it has helped propel Red Hat’s Enterprise Kubernetes distribution, the OpenShift Container Platform, further into many mission-critical use cases and deployments.

In general, secrets are a mechanism by which application deployment lifecycles can live independently of any sensitive data that might be required for calling out to secured resources. For example, access to a secured repository or a database can be stored within a secret that contains the requisite credentials needed by the application or service. At runtime the secret is injected in such a way that the application can use it to successfully make the request.

While secrets are a basic and fundamental construct for decoupling sensitive data from the development and deployment (think in terms of portability across environments) of an application, and while they can be used to help secure containers, they are just one small component of the overall security of a container platform. In fact, secrets are arguably the least critical feature in securing your containers and the platform upon which they run. Red Hat and the OpenShift Container Platform take a much more holistic, multi-layered approach to security thereby making it a true pillar of strategy. What follows is an overview of Red Hat’s approach to container security beginning with the underlying platform OS, running through to the over-arching management and enforcement layer, and everything in between.

Securing Containers Begins With The Platform

Any discussion around securing containers unequivocally must begin with a discussion around securing the platform upon which they are deployed. In this space, Red Hat predicates the deployment of OpenShift on Red Hat Enterprise Linux (RHEL) 7+. Red Hat has a rich, proven history of securing the OS platform and is considered a leader in the industry for running mission-critical workloads. This is due in large part to the integration of SELinux** and its proven capabilities of locking down platform processes and resources to prevent malicious behavior on a host.

It is also due to Red Hat investing heavily in ensuring RHEL has the requisite certifications demanded by the most stringent security use cases. Without this foundational layer of platform security, an organization risks making vulnerable the workloads (inclusive of any containers) that are deployed within it. The selection of a solid foundation at the beginning is often overlooked yet it is a defining factor for the security posture of the rest of the container infrastructure. This lends itself directly into what follows here.

** SELinux is a Linux kernel module, co-developed with the NSA for implementing a layer of mandatory access control on a host.

Securing Containers, Not Just What’s Inside

As mentioned, while secrets are great at securing applications while also decoupling them from sensitive information, note that this has little to do with the fact that a container is a running process on a Linux host that is “contained” via a collection of applied namespaces and cgroups which define what resources the container (process) can see and use. Remember, this means that a container inherently shares kernel space with the host. As a result, while containers provide a plethora of advantages over traditional virtualized deployment environments, they do represent a different type of attack vector that needs to be holistically addressed.

As just one example, a very recent CVE [1] is a zero-day vulnerability found in the docker engine that allows an external process (with an open handle to the host) coming into a container to be hijacked and used to piggy-back out of the container into the host system. This vulnerability has existed for some time and was only recently discovered, hence its zero-day designation. Thanks to SELinux integration, RHEL7 hosts running containers (for example in OpenShift) were automatically protected against this as a function of how SELinux operates. This massive vulnerability is a clear example of where Red Hat and OpenShift provide security coverage and protection against seen and unforeseen attacks and attack vectors.

Beyond all of this is the fact that Red Hat Enterprise Linux upon which the OpenShift Container Platform is deployed is Common Criteria Certified, will comply with DOD/DISA STIGs, and is used heavily in the most demanding production environments both commercially and in the federal space.

Running as Root

Because containers share kernel space with the host Operating System (OS) and are simply processes running in the OS, allowing a container to run privileged is a clear security concern as that would allow whatever is running in the container unfettered access to the host system. While in the majority of use cases a container should never need to run as root (barring some very niche requirements that would necessitate exposing that vulnerability), you will find many images in the wild that do precisely that. The OpenShift Container Platform protects against this exposure by disallowing containers to run privileged or as root. This is done in addition to the protections that SELinux provides through the use of OpenShift security constructs, known as Security Context Constraints (SCC) [2].

Using SCC’s **, administrators can define at a precise level the capabilities of a running container within the host OS, what it can see, what it can do, and even how it is allowed to run in conjunction with an SELinux context. An administrator can additionally define custom SCC’s to suit the needs of their organization’s policies and procedures for securing workloads. In doing so, they are assured that their containers are securely running and interacting with the kernel precisely how they desire and see fit.

** SCC’s also control user and group level interactions via service accounts which serve to further secure the platform, however that is outside the scope of this document.

Continuous Security

Running a container securely is foremost as discussed above, but what about continuous security as a function of development and operations management of those containers once they are running? The following is a list of security considerations an organization must think through before deploying mission-critical workloads into a container platform, and we will cover each here:

  • Software Supply Chain / Image Provenance [3]
  • Patching Your Deployments [4]
  • Security Scanning and Policy-based monitoring

Software Supply Chain / Image Provenance

As container images rely on a base OS layer, it is crucial to utilize a source registry that can be trusted to provide secured, patched, and up-to-date images for use in one’s container platform. Red Hat provides a registry populated with certified and validated images from which a vast variety of workloads can be built. This includes a wide array of languages and frameworks, databases and middleware services, as well as images from vendors who have containerized their products and tools for deployment into OpenShift.

Starting with this certified base layer is crucial in ensuring the integrity and security of an organization’s software supply chain as workloads are deployed and mature over time to include more in their stack. The ability to attest where container images originated, what they are running, and how they are running directly inform the security posture of the deployed workloads.

Patching Your Deployments

Red Hat is widely known for its ability to deliver security patches for its enterprise products in a timely and efficient manner. This is true as well for its container images. For example, when a patch is released for RHEL 7, a new patched image is also released in the Red Hat image registry such that once the new patched image is imported by an administrator (or scheduled job), OpenShift will automatically detect the new image version. This automatic detection is tied to each and every deployment in OpenShift such that any deployment that relies on the updated image as its base will be rebuilt and redeployed automatically, all in a rolling deploy strategy unless otherwise specified by the deployment configuration.

In this way patching is fundamentally changed to be more efficient and less time consuming, and continuous security (CS) becomes part of the CI/CD model turning it into CI/CD/CS. This provides a complete pipeline that is also secure end-to-end from vulnerabilities at every step of the deployment and operations lifecycle.

Options outside of OpenShift include patching containers on your own, or dealing with downtime… not the most fun you’ve had on any given night.

Security Scanning and Policy-based monitoring

With the inclusion of the CloudForms Cloud Management Platform, OpenSCAP-based security scanning of images is made possible for an added layer of image provenance to ensure the integrity and security of the images being used in the OpenShift Container Platform. The native integration between CloudForms and OpenShift provides easy intelligence into precisely what is running, how much of it is running, where it is deployed in the infrastructure, and what vulnerabilities may exist. Reports resulting from the OpenSCAP scans are automatically generated and can be exported with result filtering.

In addition, CloudForms provides real-time monitoring capabilities that allow enforcement of custom policies such that no container can be spun up in OpenShift that does not comply and conform with those organizationally-defined policies. These policies can be formed from any number of data points associated with a container from specific versions of libraries that are expected to vulnerabilities that can be detected. This, above and beyond what has been covered above (SELinux, SCC, Software Supply Chain), provides yet another layer of security ensuring that the only containers that run are those that are running secured, have been validated, and vetted by the organization.

Concluding

While secrets are an important component for securing applications and application access from development to production, securing containers requires a multi-layered, multi-pronged approach that begins at the platform operating system, runs through the container platform itself, and permeates up to cluster and infrastructure monitoring and management. Security is a foundational pillar at Red Hat and in the OpenShift container Platform. Red Hat invests heavily in ensuring the minimization of attack vectors as well as zero-day vulnerabilities, all towards the goal of making the OpenShift Container Platform enterprise-ready and fully capable for mission-critical workloads.

[1] — http://red.ht/2kkDftK & http://red.ht/2kkHgON

[2] — http://red.ht/1XwF0Pi

[3] — http://red.ht/1TEaII7

[4] — http://red.ht/1TEa0dJ