Kafka Consumer Group Rebalance (2 of 2)

This is the second in a two part article on Consumer Group Rebalance. In the first part consumer groups, rebalances, triggers for a rebalance, and the configuration options that impact rebalances were covered.

In this second part of the article the rebalance strategies available, the ability to reduce the number of unnecessary rebalances with Static Group Membership, and the risks to consider with rebalancing are all explored.

Rebalance Strategies

Eager Rebalance

With eager rebalancing (the default), when a consumer group rebalances, all processing by the consumers stops while the topic partitions are reassigned. This means the number of rebalances and their impact on an application is critical to understand as it can have a significant impact on throughput.

The following diagram illustrates the impact on an existing consumer group containing a single consumer when a new consumer joins the group, and the time outs in play. However it is applicable to all three rebalance trigger scenarios, whether the Group Coordinator has received a JoinGroup or a LeaveGroup request from a consumer, or it believes a consumer may have failed.