Thanks for sharing this with the community, Lei He!
I’d only like to clarify one thing:
> “This is when Confluent Kafka Streams comes in.”
The Kafka Streams API is part of the Apache Kafka open source project. In the latest Kafka version there is also a Scala API in addition to the Java API.
> Kafka Streams does not have a python based API at time of writing, so for data scientists that perform heavy analytics, it’s a very steep learning curve.
If you are a data scientists you might want to take a look at KSQL, the streaming SQL engine for Kafka. With KSQL, you write SQL statements to express your programming logic, no Java or Scala coding is needed.
Again, thanks for the write-up!