KubeCon Europe 2022 Recap: Great to be back in person!
I‘ve been part of Kubernetes and Cloud Native community since 2017, where my first PR to kubectl was reviewed by Brendan Burns, the Kubernetes creator himself. Since then, I realized how warm welcoming the community is, especially for new comers and underrepresented group. Being underrepresented myself (have you worked together with Indonesian engineers in open source project? probably not), it was really hard to meet fellow Indonesians in community meeting, or in KubeCon and other international conferences. Indonesia was not part of the committers table, but today, we are ranked 23rd on top country committers through various projects, including Indonesian localized docs.
I’m writing this blog to share my recent experience attending KubeCon Europe in Valencia in-person (finally!!), and how I met amazing friends I usually collaborate with online. This is my 4th KubeCon (or 7th, if virtual KubeCon is counted), I usually didn’t see other fellow Indonesians in the conference. This time, I’m very proud that there are at least 4 engineers from Indonesia out of 7,000 attendees.
It was sad that my proposal got waitlisted, but fortunately as underrepresented group, I could attend through Dan Kohn diversity scholarship. I was lucky that I had a chance to talk with Dan Kohn back in 2019, a very humble person and amazing leader (R.I.P Dan). I hope many more Indonesians can apply the scholarship and more people get exposure to this amazing community.
Amazing Conversations I had — The Good People
My goal was obviously to support fellow Indonesians who were speaking at conference: Iqbal & Gio, and Mas Aris.
However, my primary goal of attending conference in-person was to make new friends and to make good conversations (there’s a term for this in KubeCon: Hallway Track), because otherwise, how is it different from watching recorded videos on Youtube?
Interesting Sessions and Booths I attended
Below are some of the attended sessions I like the most:
- 7 Years of Running Kubernetes in Mercedes-Benz
Glad to see more and more big name end-users are adopting cloud native ecosystem. Mercedes-Benz has around 1,000 clusters managed with Cluster API and they shared success stories of improving resiliency of their systems. Unfortunately, I could not catchup with them in-person to learn the details and get inputs for my Kubernetes work at office. - Datadog Production Incident related to DNS, GRPC config, and Linux kernel edgecase
This session was one of the most packed sessions at KubeCon, where Datadog engineers shared their learnings after debugging this production incident for few weeks. The symptom was DNS query spikes (triggered OOM), but they found edgecase in Linux kernel which triggered unexpected behavior of Cilium CNI (they raised PR to fix this in Cilium), and tuning GRPC client config. - The day we deleted production cluster — InfluxDB
This was another production incident talk I attended. A classic story where someone raised wrong PR in the infra code, which deleted their production cluster (I did this myself last year, where we had major downtime). However, I took them almost 5 hours to fully get back, because they could not simply revert the changes and the extra complexity of having stateful workloads on Kubernetes. They also had to tune their ArgoCD system to handle the case. - Bringing Chaos Engineering to Cloud Native Developers — creator of LitmusChaos & Okteto
This session gave new paradigm to “shift left” chaos testing towards developers. Instead of running chaos in CI through pre-deployment or post-deployment jobs, the idea was to bring chaos test right in developers’ laptop through ephemeral dev environment (think of running “make test”, but instead of unit tests, it runs chaos test in a remote dev environment). - Service Mesh at Scale: 22k Pods in Xbox Cloud Gaming with Linkerd
Microsoft engineer shared their journey to migrate to Linkerd based service-mesh within 1 year. The decision between Istio vs Linkerd for them them: it was no brainer to setup mTLS for service-to-service communication in Linkerd (given I have Istio production experience, need to try this out with Linkerd). What impressed me the most is how fast they could rollout Linkerd for 22k pods, unfortunately I forgot to ask the details around this. However, they face the same issue I encountered at work where we need to balance tradeoff between rich mesh metrics and high cardinality to avoid prometheus issues. - Simplifying Service Mesh Operations with Flux and Flagger — by Flux Creator
I learned from this session that starting this year, we can jump 2 minor versions when upgrading Istio (need to double check starting from which version). Istio community is “partnering” with Flux/Flagger to make Istio upgrading easy, through GitOps approach and canaries. At work, we have been religiously upgrading Istio using Istio Operator, so we don’t really need this. But, this new direction is interesting and worth to keep an eye on. - Guided Tour of Cilium Service Mesh — Liz Rice
Liz is one of the open source leaders I look up to. Some of you might know here from her amazing talk: container from scratch. I learned Cilium Service Mesh is proxyless mesh, still using Envoy (1 envoy per node), and they are able to optimize kernel space calls thanks to eBPF technology, which significantly improved the latency performance. It is still early days for Cilium Service Mesh, but worth to keep an eye on and see if there’s an easy way to migrate from Istio. Cilium claims Istio control plane can still be used to control their proxyless workloads.
Impressions & The Next Step
First, I’ve been part of KubeCon program committee for at least 7 times and I am super happy to see the talks I selected for KubeCon EU were really good talks.
Second, as part of diversity scholarship recipient, there was no networking session and opportunity to get to know each other which was really sad (I hope I didn’t miss this if there was actually one).
Finally, I really hope this blog and pics I shared are motivating you to attend KubeCon next time, or even submit proposals to speak at this amazing event.
Cheers,
Giri, CNCF Ambassador from Indonesia
