Which Open Source libraries does Mozilla depend on the most?

Recently Mozilla launched it’s Open Source Support Program, allocating $1,000,000 dollars to support software that mozilla depends on, they’ve already started putting together a list of larger open source projects that they use on their wiki.

My recent work on Libraries.io got me thinking about all of the other open source projects that Mozilla uses, either directly or transatively, as dependencies of other projects they use.

Mozilla GitHub Projects

Since Libraries.io currently only supports parsing dependencies from GitHub repositories, let’s focus on projects that Mozilla has hosted on there.

There are 28 different Mozilla GitHub orgs listed on their wiki, across all those orgs they currently have 1,494 open source repositories ignoring forks.

Given a GitHub repository, Libraries.io can find the dependencies by inspecting the manifest files from many different package managers, files like package.json from npm, requirements.txt from pypi and 29 other file types from 16 different package managers.

Of those 1,494 repos, Libraries.io found 528 repositories with at least one dependency, with a total of 8,697 dependencies across all of them.

Most Popular Dependencies

Grouping all of those dependencies by name and platform, we can see how many times each dependency is used across all those repositories. Below is a list of the 25 most used libraries, complete list of all 2,591 libraries here: https://gist.github.com/andrew/2376599378b66fddd186

We can also break that down by package manager:

Unsurprisingly Mozilla depends on a lot of Javascript and Python Libraries, but also quite a few Ruby and Rust libraries.

Transitive Dependencies

So far we’ve only been looking at the direct dependencies of each GitHub repository, often each of those dependencies will be built on a number of other libraries, which could in turn have their own dependencies, all of which contribute towards the functionality of the top level project.

Libraries.io indexes and stores the dependencies for over 1,000,000 libraries across 29 different package managers, so we can use that data to get a more complete look at the whole Mozilla dependency graph.

Because each version of a library can have a different set of dependencies, for simplicity’s sake, let’s take the newest version of every library specified as a dependency, get a list of it’s dependencies and recurse all the way until we get to the end of the line, where there are only libraries with no dependencies, then group them all up to see which are used the most.

We’ll also ignore circular dependencies (A depends on B which depends on A) as those pave a path hell and are likely to bring down my postgres server.

The full list of transitive dependencies and their usage counts across all the Mozilla repos is here: https://gist.github.com/andrew/b976ea214880803daaa1

Interestingly a lot of the most used libraries are related to testing and code quality, Mocha, a javascript testing framework, coming out as the most depended on library.

As a side note, the lack of Python and Bower projects in that list is due to that information not being so easy to access from the package manager repositories, Pypi doesn’t keep track of the dependencies of each version of a package and so to pull that information out will require me downloading the tarball for every package and parsing requirements.txt and setup.py to get the dependencies for that list, I simply haven’t got round to doing that yet.

Bower doesn’t even have a real package manager repository and just falls back to GitHub repositories, so I’ll need to pull out the bower.json files from each tagged release on every repository at some point as well.

Out of interest we can also look at just the runtime dependencies of each library, here’s the top 25 transitive dependencies ignoring development dependencies:

This list shows up another issue with Libraries.io that I need to resolve, where dependencies of GitHub repositories don’t differentiate between development and runtime kinds, but ignoring that we can see a good number of low level javascript utility libraries that Mozilla heavily depends upon like Lodash, Request and Async.

All of these highly used dependencies would make great candidates for the Mozilla Open Source Support program, especially the ones without any form of corporate sponsorship already.

If you’d like to see which libraries your company depends upon the most in open source check out your organization's profile on Libraries.io at http://libraries.io/github/your_github_org_name, where you can see the top 10 most depended upon libraries across the open source repositories that it’s indexed so far, here’s Facebook for example:

If you have any questions, feedback or would like to have your organisation’s open source dependency usage analysed then get in touch on Twitter or email.

Discussion thread on Hacker News: https://news.ycombinator.com/item?id=10641800