Cloud Foundry Container Runtime 0.21

Oleksandr Slynko
Sep 6, 2018 · 3 min read

We keep releasing new versions and getting better. Since I wrote last time we have released four versions: 0.18, 0.19, 0.20 and 0.21.

In the 0.18 release, we have focused on certificates. Kubernetes has several components and they communicate via TLS. Often it is suggested to use mTLS to authenticate them. This leads to generating and managing multiple certificates. We are gaining the knowledge of them, so end-users don’t have to. As part of the 0.18 release, we explicitly set several more certificates to get the trust between components.

0.19 was all about providing custom CIDRs for pods and services. We got an external pull request and worked on it. We have fixed several important bugs – upgrade of a big cluster with IP addresses that can be part of other addresses (i.e 192.168.1.2 and 192.168.1.20), usernames in vSphere now can be NTLM-formatted, and overlay2 is enabled for Docker. The last one is the most annoying, we have overridden too many properties in our bosh manifest, now it is hard to change defaults.

0.20 we upgraded to Kubernetes 1.11 and enabled support of xenial stemcell. Our existing upgrade pipelines caught several issues with that bump, so we quite confident that this version works fine. Unfortunately, we have found that there was a bug in our tests when we were deploying the cluster in internetless environments. We have fixed it in 0.21.

But that is not only what we have done. There are several things underneath. The biggest internal track that the team is working on is “Consolidation”.

It has multiple things under the cover. The most important is removing parts that we no longer want to support. Some are just wrong, such as ABAC which we dropped, some are things that no one uses in big environments such as reusing Cloud Foundry Application Runtime routing, some are too complicated to support such as scripts. We have dropped ABAC in 0.18, we deprecated CFAR routing. We are deprecating the scripts, they are still there but we are not going to fix any bugs in the scripts and going to remove them as soon as possible. Another side of consolidation is the documentation. Everything is going to change. We plan to remove sections on how to deploy Bosh. We expect it to be handled by bosh-bootloader. It already has Terraform files written specifically for CFCR and we plan to support them. As result, all that documentation will contain is how to deploy using Bosh, list of Bosh operation files and instructions on connecting to the cluster.

As part of the consolidation, we have started working on improving our acceptance test suite. We have a customisable set of tests that will allow everyone to verify their patches and integrations for Cloud Foundry Container Runtime. They are placed at https://github.com/cloudfoundry-incubator/kubo-ci/tree/master/src/tests/integration-tests and anyone can try to run them using ginkgo with setting GOPATH to the root of the CI repo. Feel free to add new issues if the tests are not covering some feature.

Another big thing that is not visible outside – is getting more understanding of Kubernetes internals. There are many features that we don’t support yet. And sometimes we don’t support them not because they are complicated or hard to implement, but rather we don’t know about them or don’t know how they work with our current Bosh release. We are changing that so you will get next better versions of CFCR.

In next versions, we plan to expose more Kubernetes features while keeping CFCR opinionated.

Oleksandr Slynko

Written by

Reading code for a long time, writing code for even longer.