An Open Source Odyssey at Catawiki
Open-source software often carries a halo of security in the public eye, primarily due to its transparent nature. Unlike closed-source software, open-source projects benefit from what is termed Linus’s Law — the idea that “given enough eyeballs, all bugs are shallow.” This principle, articulated by Eric S. Raymond in his book “The Cathedral and the Bazaar” (1999), and named in homage to Linus Torvalds, suggests that the open scrutiny of many contributors leads to more secure and robust software.
However, the reality of modern software development tells a more nuanced story. As projects grow in complexity, incorporating numerous external dependencies has become necessary to maintain pace with the demands for rapid release cycles. This trend has its pitfalls, exemplified by incidents like the Log4j vulnerability, where a widely used logging library became a vector for remote code execution, or more frequently compromised npm libraries designed to steal private keys from users’ cryptocurrency wallets.
Recognizing these challenges early on, we at Catawiki understood the critical need for vigilant monitoring of our external dependencies. We aimed to swiftly identify any signs of neglect or abandonment and formulate contingency plans for replacing unsupported dependencies. This recognition led us to develop robust internal capabilities for open source management and establish a dedicated committee to oversee these efforts.
To address these challenges head-on, we initiated a journey into open source management, beginning with establishing a dedicated working group.
1. Crafting the Open Source Playbook: Guidelines and Real-World Impact
Our journey into open source management began with the establishment of a working group dedicated to crafting a set of guidelines and processes. These were designed to maintain the organic interaction developers have with open source projects while enhancing transparency and oversight over new initiatives. We pinpointed three key scenarios to address:
- Integrating a new open-source dependency into our projects.
- Allocating work time for maintaining open-source software or contributing to an open-source community.
- Initiating the process to open source an internal project.
Our approach wasn’t built in isolation. We sought inspiration from the open-source policies of industry giants like Google and Gitlab, which are generously shared with the public. This benchmarking helped us formulate a workflow that our developers could navigate smoothly. To support this, we established a support portal as the central hub for internal stakeholders to lodge requests. These requests are then meticulously reviewed and addressed by our Open Source Committee during our regular synchronization meetings.
With our playbook in hand, the next step was to garner internal support and engagement to drive innovation and collaboration.
2. Uniting Developers to Empower Innovation
Embarking on an open-source project is a complex endeavor that hinges on internal support and participation. Recognizing this, we sought the insights of our Team Leads and Engineering Managers, engaging them in discussions about the vision and boundaries of our open source program. Key considerations we aimed to explore together included:
- Determining the criteria for deeming an external project as trustworthy or abandoned.
- Assessing our current engagement with open-source projects and our desire to expand these efforts.
- Understanding our collective knowledge around software licensing, including preferences for specific licenses.
- Identifying which stakeholders should be involved in each process.
To facilitate this, we conducted several surveys and organized workshops to take a measured, organizational approach to defining our new processes. These interactive sessions proved crucial in shaping our strategy. They clarified our guidelines and allowed us to rally the most passionate individuals within our ranks — selecting our most committed tech enthusiasts and our VP of Engineering as the founding members and guiding forces of our Open Source Committee.
Having solidified internal support, our focus shifted towards ensuring compliance and transparency in our technology stack.
3. Setting Up our Licensing Monitoring Stack
With our foundational processes and guidelines firmly established, our next challenge was to deepen our understanding of the licensing and maintenance status of our existing technology stack. Initiated in 2021, this program embarked on this journey well before the widespread industry buzz around the Software Bill of Materials (SBOM) began. At the time, incorporating such a comprehensive format wasn’t within our initial scope.
Instead, we were looking for a tool to seamlessly integrate into our existing Continuous Integration (CI) workflow, providing clear and aggregated reporting capabilities across our entire technical stack. Given Catawiki’s extensive history with Ruby on Rails, we opted for the LicenseFinder gem. This choice enabled us to implement a dual-function strategy: firstly, generating HTML reports directly within the CI pipeline for immediate developer review upon encountering a non-compliant dependency. This immediate feedback loop ensures that potential licensing issues are flagged at the earliest possible stage, fostering a proactive approach to compliance.
Secondly, recognizing the importance of broader visibility, we also configured LicenseFinder to operate in a reporting-only mode for more complex projects. This setup allows us to produce and export JSON reports to our analytics platform. Such a mechanism empowers us to aggregate and scrutinize compliance data across our entire business stack. It facilitates a structured approach to prioritizing and addressing non-compliant dependencies, a task we undertake with diligence as part of our Open Source Committee’s regular agenda.
By adapting our tooling to not just respond to immediate compliance issues but also to provide a comprehensive overview of our dependency landscape, we’ve significantly enhanced our ability to manage and mitigate risks associated with open-source software usage.
Armed with insights into our technology stack, we ventured into the realm of open-source contribution, marking a significant milestone in our journey.
4. A Milestone with our First Open-Source Release
Having laid the groundwork, we turned our attention to identifying an internal software project ripe for open-sourcing. Our choice fell on a newly developed product-agnostic gem, designed to enhance security by implementing per-user rate-limiting across our platform. This selection marked an exciting phase: it was the first project to navigate our revamped release process. Seizing this moment, we established a company account on rubygems.org, bringing together all our previous gem projects under one roof. We’re applying the same strategy for our JavaScript packages on npmjs.com, aiming for consistency and broader impact.
This venture into open sourcing our rate-limiting gem has not only augmented our security measures but also fostered a culture of innovation and sharing. This process taught us the importance of clear documentation and community engagement, lessons that will guide our future open-source endeavors.
Navigating Our Open Source Future Together
In charting the path of Catawiki’s engagement with open source, we’ve traversed from the initial recognition of its necessity through to the establishment of a robust management framework, and onto the pioneering release of our own open-source contributions. This journey has been as much about enhancing our security and development practices as it has been about embracing the ethos of open source — collaboration, transparency, and community.
The evolution of our processes, the challenges we’ve encountered, and the achievements we’ve celebrated, all contribute to a richer understanding of how open source can amplify innovation and security within our tech landscape. As we continue exploring this vast and vibrant territory, our commitment to contributing to the community and fostering an environment of shared growth remains steadfast.
We’re eager to hear from you — your insights, experiences, and perspectives on open source. If you’re passionate about open source and eager to make a difference in a rapidly growing company, we can’t wait to hear from you. Join us and help up on our mission to make special objects more accessible to passionate collectors worldwide.
