Thanks for the note. I was in the process of using the presto Kafka plugin with protobuf decoder feature, however my team had the following concerns.

  1. How does it connect to Kafka — starts reading from beginning always?
  2. Does it store result in some cache?
  3. How does it handle new record coming?

For Question 1 as the plugin reads from beginning always for each sql query we were concerned about the load it would put on our production Kafka. Any ideas on how we could overcome this limitation. Maybe we could cache the Kafka offset along with data for the plugin to seek from the cached offset instead of seeking from the beginning every time instead? Would love to hear your ideas on this.

)