Kubernetes with Cloud Foundry

As I mentioned before I don’t see lots of benefits of migrating from Diego to Kubernetes. 
I can summarize them in a way of an old joke:

If I were Rothschild, I’d be even richer than Rothschild, I’d do a little teaching on a side.

It seems that people want to add Kubernetes for the same reason, they want to run something on a side.

However, project Eirini has one super valid point. Kubernetes is quite far from Cloud Foundry Application Runtime from UX point of view.

There is nothing even close to in cf push Kubernetes world.

K8s 20–20 vision — have `cf push`

I heard many times that in order to start using Kubernetes one has to first learn how to deploy it and study its internals. Developers are intended to become operators. But this is not the best way to start using it.

Why? There are too many moving parts and things cluster operator has to do.

  • There is no easy way to check security in default Kubernetes. Yes, it is possible and relatively easy to add more components to the running cluster, such as container registry with Notary support, security vulnerability checks and admission controllers.
  • It is quite hard to connect vanilla Kubernetes with LDAP or SSO services. The logging in solutions is again up to each operator and there is no easy way to connect them. Dex provides relatively easy authentication, but does not with authorization and leaves it to the cluster itself.
  • Users are pretty much required to know how to use limits and how not to break a cluster
  • While metrics are possible to use with Heapster, other programs have to be connected properly and support certificates validation.
  • Deploying highly available applications on Kubernetes is relatively simple, but again requires knowledge.

All this is doable but requires everyone to do it in their own way. At KubeCon there were multiple stories about the ways internal teams have implemented some tools to help developers and hid most of the extensibility away.

I believe a single cf push for Kubernetes can solve most of this problems and this is what Kubernetes community is missing. This is where Cloud Foundry community should help.

For example, UAA brings LDAP and SAML support, but there is no login functionality. Someone could integrate UAA login with kubeconfig and make it easy to use.

Loggregator can improve logging flow and make it as easy as possible for users.

Integrating buildpacks, container registries and code artefacts can be something that Cloud Foundry can bring to the table. There is an experimental packs repo.

Default Kubernetes manifest generation can be one of the projects that someone will provide.

How is this different from project Eirini? I basically propose to bring cf push to Kubernetes. Isn’t it the same? I would like to share Cloud Foundry tools with CNCF community. No one can argue that Kubernetes community has many more people (and much more contributors). There were several times more people at KubeCon than at CF Summit two weeks earlier. If people start using at least some tools and see the benefits of everything working together, then who knows