Taming triage: Partnering with Topcoder to harness the power of the crowd
New innovation challenge is looking for an algorithm to automate bug triaging in Bugzilla
We are excited to announce the launch of the Bugzilla Automatic Bug Triaging Challenge, a crowdsourcing competition sponsored by Mozilla and hosted by Topcoder, the world’s largest network of software designers, developers, testers, and data scientists. The goal of the competition is to automate triaging (categorization by products and software components) of new bugs submitted to Bugzilla, Mozilla’s web-based bug tracking system. By cooperating with Topcoder, Mozilla is expanding its open innovation capabilities to include specialized crowdsourcing communities and competition mechanisms.
Mozilla’s Open Innovation strategy is guided by the principle of being Open by Design derived from a comprehensive 2017 review of how Mozilla works with open communities. The strategy sets forth a direction of expanding the organisation’s external outreach beyond its traditional base of core contributors: open source software developers, lead users, and Mozilla volunteers. Our cooperation with Topcoder is an example of reaching out to a global community of data scientists.
Why Bugs?
Mozilla is using crowdsourcing to scale the effort we can bring to our product and technology development through collaborative crowds. Such a “capacity crowdsourcing” has already been successfully applied to the Common Voice project, Mozilla’s initiative to crowdsource a large dataset of human voices for use in speech technology.
However, we know that engaging crowds can have positive impact on other areas of Mozilla product and technology development. In particular, we focus our attention on processes that require large amounts of manual engineering work; automating these processes can result in significant lowering of development and operating cost.
Take, for example, bug triaging in Bugzilla, a manual process of categorization (by products and software components) of hundreds of bugs submitted each month to Mozilla’s web-based bug tracking system. Although the accuracy of the manual bug triaging is very high, it consumes valuable time of experienced engineers, which may otherwise be spent on other high-priority projects.
Why Topcoder?
Over years, the Firefox Test Engineering team responsible for bug triaging has accumulated a lot of data that could potentially be used to automate the process. Working together with the Open Innovation team, the engineers have engaged Topcoder, the world’s largest network of software designers, developers, testers, and data scientists, which many organizations, including NASA, use as a platform to solve complex algorithmic problems.
The result of this collaboration has been the Bugzilla Automatic Bug Triaging Challenge whose objective is to create an algorithm allowing automated bug triaging in Bugzilla with the accuracy comparable to that for the manual process. To select the winners of the competition, the Topcoder architect team has developed a scoring mechanism; to qualify for a prize (ranging between $1,000 and $8,000), any algorithm must reach a certain minimal score. Refer to the Challenge description for more detail.
The Challenge will be open for submissions until October 26. Although Mozilla employees are not allowed to participate, we do encourage all members of Mozilla’s communities to take part in the competition.
