Wishlist : Beyond GlusterFS 7.0
Why call it “wishlist”, not Roadmap
Roadmaps are critical for the project, and they are the reason users keep wanting to use the project. No one wants a project which is not ‘active’.
Also, regardless of which company supports the project, it is not easy to ‘promise’ on delivery of features/tasks in an open source project. It can get done faster if there are more companies, developers interested in it, or it can take lot more time than anticipated if there is not much request, buzz around the features.
As a community lead, I can only propose requests for certain features, and prioritize reviews in those areas. But it will be role of entire community to rally behind something, and get it moving, and take it to completion. Hence, I am titling the blog as WishList, and not Roadmap.
Now, Question is how does Gluster’s WishList look like?
WishList 0 : Stability
Well, I should say, with an year of focused work in this area from the whole team, we are there. We can say glusterfs-7.0 contains most of fixes, and stability related work. While we are not planning to take this with any less priority, this wish list is under control.
WishList 1: Performance
Performance is always one of the major ‘challenge’ in distributed systems. Specially in storage. There are too many pieces involved, and a small issue in one machine’s component (can be bad drive, bad n/w card, or even degraded mode for CPU) can cause butterfly effect, and soon cause major bottleneck for the whole cluster.
While we can’t avoid such failures, the first and primary thing is to ‘identify’ the bottleneck. This is critical for users, to immediately see what they can do to work around, and also helps developers to design focused solutions to handle situation better in future releases. With that, for providing a Performance enhancement in distributed systems, I would say, having a good monitoring / diagnostic tools (or call it internal hooks as most of the times it may be inside the same process) are more critical. And, I should say, we understand it, and we have started working on this area.
Also, with 10+ years of experience in many performance problem debugging, our team also has identified certain key areas to improve performance, and we will soon come out with specific lists, and work on them.
If you are an user facing performance problems with Gluster, and that is the major pain point at this time, I am confident to tell, we will make you happy soon :-) But, please note that, there is no single solution for ‘all’ the performance problems, so continue reporting your issues, and we will analyse them case by case.
WishList 2: Monitoring
While I talked a bit about this w.r.to performance wishlist, monitoring is an important part for any large projects. We need to get better here, and converge on tools, and metrics we would track. Happy to collaborate here, and work with interested people.
WishList 3: Scale
When we talk about ‘distributed storage’, the immediate next question which comes to mind is how big can it scale. People ask about storage size, but technically, most of the times, the scale gets limited by number of nodes.
Due to few of the design decisions we made long back, current solutions available in GlusterFS have some limitations on how many nodes it can scale. There were proposals to solve it, but none could be completed till date. We intend to pick this up soon, and would like to have more discussions on this.
WishList 4: Better Container Storage story
In recent times, Hybrid cloud is a buzz, and GlusterFS, being choosen from very early versions of k8s to support storage, has had multiple solutions till date, and none of them are very convincing yet. Time is ripe to have some work done on this. We did work on GCS/GD2 from Gluster community, from the team @ RedHat, but that didn’t cross the finish line. It is not at a point to pick it up right now.
I believe, Aravinda’s proposal, about which we are talking at DevConf is a good solution. But surely it needs to be talked about and discussed in detail.
WishList 5: Newer/External network module
Gluster’s network layer is RPC based, which uses XDR for encode/decode. This is observed to be a performance bottleneck in current system. There are proposal for gRPC/protobuf based network layer for glusterfs. It needs to be worked on, and before getting to this, we need to make network code more modular than it is currently present. Anyways, again, time is good to start working on it. If you are interested, or have experience in network layer, and happy to contribute, let me know.
So, what next?
Currently, GlusterFS is ready to make release of glusterfs-7.0, and we are in the process of defining the long term roadmap for the project, and this blog is one of the early step in that direction.
If you are interested in any of the wishlist, either as ‘consumer’ or as ‘contributor’, do reach out to me, or community, we can figure out how we can collaborate.
I will write more about individual topic, and what are the particular issues we would like to pick first, and how we can address some of these problems soon.
