How to add and remove nodes in Apache Pulsar
Adding and removing nodes is one of the most common operations when managing a distributed system. We’ve heard from users that have used both Apache Kafka and Apache Pulsar that adding and removing nodes can be operationally burdensome in Kafka because it requires operations teams to manually balance data between nodes in the cluster. This balancing act is known to be error prone and to consume a lot of bandwidth, making it all too easy to break production deployments.
Apache Pulsar, in contrast, is built on top of a layered, segment-oriented architecture that separates stateless serving from stateful persistence. This clean separation between concerns allows for independent scalability between the serving and persistence layers, which significantly simplifies operations. In Pulsar, adding and removing nodes is incredibly simple because there’s no need to rebalance data.
For some technical details behind node operations in Pulsar, check out our blog post How a segment-based architecture delivers better performance, scalability and resilience.
In this more practical tutorial video, I’ll show you how easy it is to add/remove nodes to/from an existing Apache Pulsar cluster.
Video source:https://youtu.be/_BW67HAkS-w
More on Apache Pulsar
If you want to learn more about Apache Pulsar, please visit the official website at https://pulsar.apache.org.
If you want to learn more about the differences between Apache Pulsar and Apache Kafka, please checkout our comparison series of blog posts:
- Part 1 — Comparing Pulsar and Kafka: unified queuing and streaming
- Part 2 — Comparing Pulsar and Kafka: How a segment-based architecture delivers better performance, scalability and resilience
You can also participate in the Pulsar community via:
- The Pulsar slack channel. You can self-register at https://apache-pulsar.herokuapp.com.
- The Pulsar mailing list.
For getting the latest updates about Pulsar, you can follow the projects on Twitter @apache_pulsar.
About the author
Sijie Guo is the founder of StreamNative, VP of Apache BookKeeper, and PMC Member of Apache Pulsar. Previously, he was the tech lead for the messaging group at Twitter, where he co-created Apache DistributedLog. Prior to Twitter, he worked on push notification infrastructure at Yahoo. You can follow him on twitter.
Like this post? Please recommend and/or share.
Want to learn more? See https://streamnative.io/blog. Follow us here on Medium and check out our GitHub.
