Applying Open Source contribution model at Xendit and building a better engineering culture
How the OpenSource model can work for InnerSource too!
The prevalence of OpenSource software is undeniable. If a software developer tells us that he/she has never do anything involving OpenSource Software (OSS), we’re probably looking at a time traveler from when the Internet didn’t exist!
Time and time again, the benefits of adopting and contributing to Open Source Software are highlighted. Nevertheless, there’s little to no literature to how an organisation can properly adopt and contribute to Open Source Software. A quick Google search would mainly show guides on how individuals can get started contributing to Open Source projects without anyone mentioning how we can do it efficiently at an organisational level and build a culture around Open Source.
This post will outline our attempt at Xendit so far trying to build a model, and in turn — a culture, around Open Source contributing.
Challenges at Xendit
We didn’t started building a model of contributing to InnerSource out of the blue. It actually came from a real need for us to deal with legacy code.
It all started with Xendit’s usage of micro-service architecture with the best of intentions: We want folks to be able to “move fast and break things”, to develop and deliver more features, more payment channels without stepping on one another’s toes.
Under this ownership model, there were a lot of common “utils” repositories born out of necessity, without much thoughts and considerations in place, as folks need a quick way to share code across different codebases.
Fast forward to the next growth stage of our engineering organisation, we have evolved to have proper teams with clear responsibilities and boundaries over specific codebases depending on the product
But this also mean that we end up with a bunch of “orphan” utilities repositories, which are ironically used by almost everyone in their codebases and capable of bringing about critical system issues if errors seep in. We actually got bitten by these problems more than a few times and decided to do something about it.
Tackling the “everybody’s business” problem
As the saying goes — “Everybody’s business is nobody’s business”, it’s natural, however detrimental and painful, for our folks to leave these critical pieces of our system unattended and only patch them when they break down our systems.
So we took a step back and a good bird-eye-view look of the whole system. These common modules play somewhat a similar role to external libraries like ExpressJS. This inspired us to figure out what we were lacking that made all the difference. We identified 2 missing pieces of the puzzles:
- There’s a lack of recognitions and rewards: Usually for Open Source projects, especially ones that are widely used, to which contribution gives developers “street cred” and respect from their peers. And with recent push in Open Source funding like Github Sponsors, there’re actual monetary incentives to writing free softwares.
- There’s also a lack of ownership: It’s common for Open Source libraries to be started by someone with a rather personal and specific need for their projects, then slowly evolve into a proper maintainer group and sometimes even an entire foundation. In any cases, the projects hit close to home.
So how can we emulate the same incentives for Xendit common codebases? How should we adapt the Open Source model into our organisation structure.
Open Source @ Xendit
To solve the above two problems, we came up with a system to reward engineers for their effort and a clear structure for taking on maintainer’s responsibilities.
Incentivising contributions
Developers are essentially humans and we all respond to incentives. Our current situation means that there’s only an incentives on the negative side — fix common libraries so our applications stay up, and that’s definitely not how quality softwares are built.
We decided that there must be positive incentives to motive our engineers. The first step to creating rewards is to measuring contributions so we came up with a formula to point the karma for each Github issue. This is the current version after multiple refinements:
According to the model outlined in previous section, Open Source rewards come in two forms — street creds and monetary. We have titles for top contributor of the month — Wizard of OSS, and for top contributor of the year — Lord of the OSS, both of whom pictures will be up on our internal OSS contribution leaderboard. On top of that, folks can redeem karma points for educational purchases in a progressive manner with the more points accumulated, the more each point worths.
Distributed ownership
The second piece of the puzzle for our program to be effective is ownership. We started by forming core maintainer groups based on historical contributions to each of the existing common repositories with their own slack handles and responsibilities (with great handles come great responsibilities):
- Answer all questions, concerns about the repositories that they are overseeing
- Code reviews for contributions
- Review feature requests
- Initiate new feature discussions towards project vision
- Maintain and publish new versions of the repository
- Nominate, assess and onboard new core maintainers
This mirrors the same structure that we often observe in Open Source projects out in the wild, with small groups of core maintainers keeping the projects in line with the directions and purposes they were created for. This also means that new common projects within our organisation will have proper ownership (which we initially had a few years back before all the problems) and succession plan (which was what we lack that is the main cause for ownership diminishing).
Small wins & future ambitions
Ever since its inception, our internal OSS program has:
- Grown our contributions to the organisation’s common codebases from happy-go-lucky to frequent grooming and maintenance by over 30 enthusiastic engineers (about 15% of the total engineering head-counts).
- Helped us cut down the turn-around for bug-fixes and feature releases of our common libraries to within a single week.
- Given us a proper platform to have discussion about organisational standards and move towards more widely accepted industry standards like OpenTelemetry instead of coming up with and maintaining our own.
With these foundations set up within our own internal operations, we’re looking to be involved more in Open Source work with the public communities, to be able to give back to whom we’ve benefited so much from.
We hope that our journey so far could be an inspiration for other organisations to develop Open Source culture with a less steep learning curve.
Last but not least, if you would like to join us in our Open Source adventure, don’t hesitate because Xendit is always looking for awesome people!
