The most interesting things we learned at KubeCon North America 2019

By James Pancoast, Distinguished Engineer

Kenzan + Sourced
Kenzan Cloud & Dev Blog


KubeCon North America 2019 took place in San Diego a few weeks ago (Nov 19–21). Two “lucky” Kenzanites were forced to attend against our will and now I’m being forced to write this blog post about it. It rained way too much this go around but at least there was a lot of great beer. Since Jeremy is much faster than I am he already has a blog post up about telepresence and our use of it.

KubeCon (AKA “KubeCon / CloudNativeCon”) is The Cloud Native Computing Foundation’s regular conference. There are actually several, one in North America, one in Europe, and one in China, every year. It’s the place to talk and learn about all things cloud native and all of the CNCF projects. When I attend conferences like KubeCon or Hashiconf I try to do two things. The first is to dig really deep into particular solutions by attending breakout talks being given by someone who’s already an expert (or close to it). The other is to try to recognize trends, opportunities, or callouts that I find interesting. Today we will focus on the latter.

James Pancoast discussing his Kubecon learnings

Here are a few of the trends I noticed:

  1. Make Kubernetes easier to use. This was called out several times in several panels and keynotes. Bryan Liles, in one of his keynote slots, said Kubernetes needs its “Ruby On Rails” moment. Matt Klein mentioned it in the TOC panel. While those of us that live and breath it may not realize it, Kubernetes is hard to use. It’s hard to install. It’s hard to update. It’s hard to manage. Some of the CNCF projects I feel are related to this space are Buildpacks, telepresence, and Flux.
  2. Security, Security, Security. I can’t remember who said it but operators need to remember that Kubernetes is NOT secure by default. There are multiple OS projects and vendors in this space. For OS projects there’s Falco, Open Policy Agent, In-Toto and Notary. In the vendor space there’s Sysdig, Neuvector and Aqua Security.
  3. Hybrid / Multicloud. This isn’t really a new “trend” so I’m not sure if it’s still even considered a trend. There are multiple vendors now offering enterprise management of multiple Kubernetes clusters in disparate locations, ie, whatever combination of cloud provider, on-prem, and VMs or Kubernetes you choose. Current service meshes can also tie all these resources together no matter their location or hosting method which can make cloud migration easier and provide redundancy.
  4. Multiple Clusters. This is a special case of the previous point. Some organizations are hitting the current node scaling limits in Kubernetes. To get around this they are running multiple clusters in a single environment. Most of the current service meshes can span multiple clusters and VM’s and make it relatively easy for a service on cluster A to talk to a service on cluster B.
  5. Chaos Engineering As A Service. I would classify this as an emerging trend. While I can only remember the name of one company offering this service (Gremlin) in the vendor area, I think there were others. Well, I’m pretty sure I saw ‘Chaos’ on other booths, at least. Maybe it’s just the normal state of our industry they were talking about. Services like these provide the ability to more easily run Chaos game days and the like. As more and more organizations embrace chaos engineering this space will only grow.

Callouts (ie, other interesting things I saw or heard about)

  1. Network Service Mesh. If you need something more than just GRPC and HTTP in your service mesh.
  2. Opentracing and Opencensus are combining into Opentelemetry
  3. Jenkins templating engine
  4. Vitess. I just really want to try this solution out.

The talks I found most interesting

  1. Runtime security with Falco
  2. Scaling resilient systems, slacks DB service
  3. 10 weird ways to blow up your k8s
  4. Open Policy Agent (OPA) Deep Dive
  5. Technical Oversight Committee Panel
  6. Hello from the other side, dispatches from a k8s attacker
  7. Debugging live apps in Kubernetes
  8. CAP_NET_RAW And ARP Spoofing in Your Cluster: It’s Going Downhill From Here
  9. Tutorial: attacking and defending Kubernetes clusters, a guided tour
  10. How k8s components communicate securely in your cluster
  11. From Brownfield to Greenfield: Freddie Mac’s Service Mesh Journey. This talk is where I got the “Mesh first, containerize later” mantra which everyone will soon become sick of I’m sure.
  12. Are you about to break prod? Acceptance testing with ephemeral environments

There are also the Day 0 Co-Located events listed here: There are too many to list here but most, if not all, of these conferences have links on each site to videos of the talks.

Some other peoples Kubecon recaps:
* KubeCon NA 2019: Top Ten Takeaways (Part 1) and (Part 2)
* KubeCon San Diego Roundup

Additional links:
* Most of the talks are on youtube in this CNCF playlist