Aaron Boodman
Sep 23, 2015 · 8 min read

How Chromium Works

Today, I’d like to tell you about how Chromium works. Not Chrome, the browser, but Chromium, the group of people who build the browser.

No branches

On many projects, it’s common to branch the source code to work on major new features. The idea is that temporary destabilization from the new code won’t affect other developers or users. Once the feature is complete, its branch is merged back into trunk, and there is usually a period of instability while integration issues are ironed out.

Runtime switches

We don’t create branches, but we still need some way to hide incomplete features from users. One natural way to do this would be with compile-time checks. The problem with those is they aren’t much different from just branching the code. You still effectively have two separate codebases that must eventually be merged. And since the new code isn’t compiled or tested by default, it’s very easy for developers to accidentally break it.

Gigantic amounts of automated testing

In order to release every day, we need to have confidence that our codebase is always in a good state. This requires automated tests. Lots of automated tests. As of this writing Chrome has about 12k class-level unit tests, 2k automated integration tests, and a huge assortment of perf tests, bloat tests, thread safety tests, memory safety tests, and probably more that I can’t think of. And that’s just for Chrome. WebKit, V8, and our other dependencies are tested on their own. WebKit alone has about 27k tests that ensure web pages layout and function correctly. Our general rule is that every change has to come with tests.

  1. It slows down development because everyone has to work around whatever is broken
  2. It encourages developers to make sloppy quick fixes to the get the tests passing
  3. It prevents us from releasing!

Relentless refactoring

Because we have pretty comprehensive test coverage, we can afford to be aggressive with refactoring. There are always a few major refactor efforts going on in Chrome. Right now the main ones are Carnitas and Aura.

DEPS

WebKit moves really fast too. And just like we cannot have feature branches that suddenly land, we can’t try and merge a month’s worth of WebKit changes all at once. It would destabilize the tree for days.

Challenges

Testing thoroughly is still an unsolved problem. In particular, flaky integration tests are a constant issue for us. Chrome is big, complex, asynchronous, multiprocess, and multithreaded. It’s easy for integration tests to have subtle timing issues that make them fail intermittently. On a project our size, a test that fails 1% of the time is guaranteed to fail multiple times per day.

Conclusion

So, how are the wheels still on the bus? In short: no branches, runtime switches, tons of automated testing, relentless refactoring, and staying very close to HEAD of our dependencies.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade