EXPEDIA GROUP TECHNOLOGY — SOFTWARE

Open Source at Expedia Group — Behind the scenes

Scaling open source with tools and simplified processes

The image shows a gate with an arc in Oia, Greece. In the background you will find blue sky and a view of the sea.
Photo by Florian Wehde on Unsplash.

At Expedia Group™ ️we have formed a group of open source enthusiasts who help employees with releasing projects to the public. Over the last few years we have released a number of internal projects to our github.com organisation. Processes that used to be tedious have been streamlined to allow us to move fast.

This blog post provides a summary of processes and resources which may be useful for doing open source at scale.

A bit of history

The logo of Expedia Group’s STYX project. The image shows a skull figure that rides a boat. The word STYX is written on the surface of the boat.
Styx, a reverse proxy for the JVM, was one of the first projects by Expedia Group.

Expedia Group ️has evolved through mergers and acquisitions (M&A). Different brands used their own tech stacks, maintained their own github.com organisations, and were using their own processes for open sourcing software.

As a matter of fact, one of the very first projects that was open sourced by the company was Styx, a reverse proxy for the JVM. At that time we did not have a common process for releasing projects to a public GitHub and it took almost a year to get the required approvals and release the project to the community.

As much fun as it sounds, it was demotivating for our engineers. When the bottlenecks are processes, or the lack of them, you know you have to improve.

Open source release process powered by GitHub

Screenshot of an internal submission for a project that would be open sourced (Insights Explorer). It shows GitHub’s Pull Request view where the author describes the submission and ensures the items in the checklist have been met. On the right there is a list of reviewers and the labels used for tracking progress of the submission (Review, Security, Legal, Approved).
GitHub-based process for open sourcing internal projects.

A member of our group had a brilliant idea: what if we use GitHub for any open source submissions?

The owners of a project to be released would raise a Pull Request (PR) in a repository that would be used for submissions and which would be monitored by our group. The PR would include a short business justification, and a link to its internal repository.

The PR would then be picked up by one of our open source champions who would shepherd it through the process.

Labels would be used for each of the stages (technical review, security review, legal review) and GitHub Projects would allow us to move projects across different stages. Additionally, instead of communicating with the stakeholders via email, GitHub Teams would enable us to tag them and add them as reviewers.

GitHub Project view of our internal open source submissions. Any submitted Pull Requests are linked to the GitHub Project where we track submissions. They are then moved to the current state (Submitted, Review, Legal, Security, Approved, Rejected).
GitHub Project tracking open source submissions.

We set up the above process 2 years ago and, following a number of iterations, we have reached a point where new projects get open sourced in days rather than months.

On-road experience for new projects

GitHub template repository for new open source projects by Expedia Group.
GitHub template for new open source projects.

The process so far would ensure a smooth experience up to the point where teams get the final approval to release their project. However, there is still work to be done after releasing a project.

To ensure any required files are in the github.com repository we provide a template which teams can use. The template includes a set of files that make the repository compliant, from a legal and security point of view, and open source friendly.

To make our engineers’ lives easier and to avoid maintenance overhead, our github.com organisation is configured in a way that allows us to centrally control settings such as integrations or the credentials which teams can use.

Visibility and Adoption

The landing page of Expedia Group’s Open Source website. On the top there is the company’s logo and a menu with links to the Projects, Tech Blog and to the company’s Careers website. In the middle, there is an image of a person riding a boat which is alongside the following text “Powering global travel through a world-class tech platform. Peek behind the scenes at our open source projects”. At the bottom one can find 6 projects open sourced by Expedia Group alongside their logos.
Expedia Group’s open source website.

Although we haven’t done much in that space, we do have a set of practices that help with increasing visibility and use of our projects.

First, we use Topics in our repositories which improve discoverability.

We also encourage project owners to write a blog post once they open source their project, or even when they publish a new major release.

To advertise our projects (but also as a fun way to learn certain front-end technologies), our team built a website that showcases our most popular open source projects.

Statistics on contributions to open source projects

The Open Source Contributor Index website. The image shows a list of companies ranked based on their employees contributions to open source projects. Visitors of the website can select the month and the year while the list includes the number of “active contributors” and the “total community”.
The Open Source Contributor Index ranks commercial organizations by the volume of their employees’ contributions.

It is common for Open Source Program Offices to look for ways to track contributions by their employees. There are many reasons for this:

  • It may be an auditing requirement.
  • It enables the team to provide metrics to stakeholders.
  • It creates a healthy competition between employees but also between companies.

The good news is that we can use the Open Source Contributor Index (OSCI) for this. The OSCI ranks commercial organisations by the volume of their employees’ contributions. To get meaningful results:

  1. Developers need to use their company’s email address for their commits. Since GitHub allows multiple addresses per profile, we recommend adding the work email address as a secondary address to the personal profile. This way employees can use that address for any commits done during working hours and from their employer’s laptop.
  2. The company’s email domain should be added to the list that is used to generate the statistics.

Final Thoughts

There is a significant amount of fine detail to consider when doing open source as a company, including licenses and Contributor License Agreements (CLAs), distribution of credentials, and promotion of projects.

Our ultimate goal as a group is to make the experience smooth and encourage our engineers to contribute to the community in a safe way.

Acknowledgements

A significant number of individuals have contributed to our group over the years. Special thanks to Daniel Albuquerque, Oltion Doda, Francesco Feltrinelli, Ronny Katzenberger, Dariusz Kuc, Trevor Livingston, Nathaniel McAuliffe, our legal and security counterparts, and to Adrian Woodhead for building and driving this community forward before his departure.

--

--

--

Stories from the Expedia Group Technology teams

Recommended from Medium

Manage Up: Take Charge of Your Own Growth

Hitting Delete Is Not Refactoring

How to Take an Algorithm From Paper to Production

AI Research to Production with Einstein Reply Recommendations

Local Cluster vs. Remote Cluster for Kubernetes-Based Development

Series: Secure way of managing Application Secrets on Kubernetes. Solution overview.

The 5 Superpowers (and Struggles) of Great Startup Software Engineers

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Nikos Katirtzis

Nikos Katirtzis

Software Engineer @ExpediaGroup

More from Medium

Chaos Engineering at Expedia Group

Many Lego blocks of different sizes, shapes, and colors

Migrating to a Multi-Cluster Managed Kafka with 0 Downtime

Introduction to TypeDB Cluster

Debugging as a Superpower

A flow chart showing the debugging process with nodes “run the test” and “can you see the problem” linked with some yes-no decisions