Apache Kafka and Immutable Infrastructure + Plus an introductory Kafka talk!

Brian F
Brian F
Nov 26, 2019 · 2 min read

The above video contains two talks presented at the Kafka meetup hosted at GumGum on 19th November, 2019.

Talk 1:

Speaker: Alex Woolford

In this session, Alex Woolford will walk us through some practical near real-time examples that touch on analytics, systems integration, and machine learning. We’ll show you:

- how to build streaming apps using a familiar SQL interface (KSQL)
- how to capture events from external systems
- how to deploy a machine learning model in a stream of events

Alex is a systems engineer at Confluent, the company founded by the original creators of Apache Kafka.

He’s been working with a diverse set of open source technologies for the past 7 years, specifically Hadoop et. al, StreamSets, and most recently Kafka. Alex maintains a YouTube channel of geeky practical examples which can be found at https://www.youtube.com/c/alexwoolford.

Outside of work he enjoys floating in a tank, playing guitar (very, very badly), and yelling from the sidelines at his kids sporting events.

Talk 2:

Speaker: Karim Lamouri

Apache Kafka and Immutable Infrastructure

Today is the day that you have to upgrade your Apache Kafka production cluster. You trained well on your staging cluster but there are a lot of steps to upgrade a cluster.
You are afraid that you will miss some steps on some brokers leading to production failures or configuration drift. You wish that you could deploy Kafka just like any of your stateless application. Immutable infrastructure can help you. It means that components of your infrastructure get replaced at every deployment, not just updated in-place. It ensures you a consistent, reliable, simple, predictable deployment process.

At GumGum we are running 4 Kafka clusters along with Kafka Connect and Schema Registry — more than 90 nodes in total. Using the techniques discussed in this talk we improved, simplified and accelerated operations on this infrastructure we manage. What you can take home from this talk:
- How to build a Kafka ecosystem as an immutable infrastructure
- Considerations to take into account to be able to have auto-healing
- How to speed up the configuration updates, upgrades and recovery of
your streaming platform with minimal headaches
- How to monitor your stack in this immutable infrastructure.

Karim is a French Software Engineer SRE at GumGum. He holds a master’s degree in Computer Science and he has been in charge of GumGum’s Kafka infrastructure for the past 2 years.


Thoughts from the GumGum tech team

Brian F

Written by

Brian F


Thoughts from the GumGum tech team

More From Medium

More from gumgum-tech

More on Data Engineering from gumgum-tech

More on Data Engineering from gumgum-tech

Apply Time Series Models to Detect Anomalies

Feb 13 · 4 min read


More on Kafka from gumgum-tech

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade