Improving Innersource: Leveraging more than transparency, symmetry, and inclusion (Transcript)

This is the transcript for our InnerSource presentation available at https://www.slideshare.net/mobile/AshleyWolf/improving-innersource-leveraging-more-than-transparency-symmetry-and-inclusion-112174774.

Slide 1: Improving Innersource; Leveraging transparency, symmetry, and inclusion

Thanks everyone for joining us for InnerSource Day!

Slide 2: Open Source at Oath

Ashley and I run Oath’s Open Source Program Office. What’s Oath? Think of it this way: it’s a collection of Internet brands that include Yahoo News, Sports, Finance, Mail, Aol, Huffington Post, Tumblr, TechCrunch and others. We’re a division of Verizon. So what are we doing today?

Slide 3: Today, we plan to…

We’re here to talk to you today about Innersourcing :

  1. The essential values of open source practices and how that applies to InnerSource
  2. The challenges of applying open source values to InnerSource
  3. And we’ll share some stories that should get you thinking about being more open.

Wait, if this is an InnerSource day, why are we talking about Open Source?

Well, we run the open source office. And InnerSource is thought of as applying Open Source principles inside a company. So today we’re going to look at InnerSource from that vantage point.

Slide 4: The code behind open source

At the core of successful open source projects we find these values:

  • Transparency — you can see my code and processes. We publish code and documentation in a place you can also see.
  • Symmetry of control — if I can do it, you can do it. No need to create libre versions, because we all have a stake in the project.
  • Non Exclusivity — Open source code is available to anyone.

Are you saying that open source projects are successful only if they have these three? Don’t the committers get to commit code? but other people can’t? Also, not every project is super-inclusive.

Well, technically, if you slap on an open source license you can say hey I’m open source. But what I’m saying is that to get a successful project, you need to make sure you have these three.

Let me put it this way: I’ve seen a lot of open source projects hit a rough patch where people get bent out of shape. It’s almost always because something is getting less transparent, someone seems to be wielding power over the project, or someone feels shut out. When that happens, people complain, quit, and fork Libre-versions of the code.

Slide 5: What makes open source successful?

Healthy open source projects are open to participation. No group wields power over the project — or if they do, people can freely fork the code and compete with a Libre- version (which happens every so often). Since anyone can join, you need to set up a cooperative model. You can’t order people around in an open source project. They don’t work for you, they work with you, and only if they want to. Which, by the way, sounds nothing like corporate life.

Slide 6: That sounds nothing like corporate life, right?

And that’s the challenge with InnerSource.

If Open Source is all about voluntary participation, without authoritative power telling you what you have to do, that’s not going to go over well with management. You can’t just fork a project and compete with your team. You can’t go rogue and expect to keep your job.

Slide 7: The code behind corporate code

When we look at corporate code — the core values behind it are about speed, risk, and control.

We all operate on deadlines. We want to fail fast, or succeed quickly. Fundamentally, someone is in control. It’s not opportunistic or voluntary. There’s somebody who is responsible to deliver. And they have to do it quickly, with low risk, and lots of control.

Slide 8: Quote

And that’s a problem for InnerSource. Open Source is the opposite of InnerSource. You might think that if you just “do open source inside a company” you’ll magically succeed at InnerSource. We’ve see it play out differently.

I’ve heard people tell me they are working on an InnerSource program, and they really mean they have an army of project managers ready to put up posters on the walls saying “we’re doing InnerSource”. Then they plan to move everyone to GitHub or GitLab and hope they’ll get code reuse, engineering efficiency, and all the value of open source, with none of the pesky legal issues. Sadly, that’s not how it works.

So I figured we’ll share three stories with you that will hopefully inspire new ways of thinking about InnerSource. For many of you, these might not be new. But they reflect our journey and hopefully you’ll find them interesting and useful too.

Slide 9: A Story of Love

Story #1 About 15 years ago we were looking to launch a new product, and we weren’t sure if was going to be a thing. We noticed that people were using the internet to find love (or things related). We wanted to see if people would use the internet for dating, but it seemed risky. Before investing too much into a site, we had engineers take code from a bunch of existing product repos, literally copied the code into a new repo and changed labels to stand up a site in three months. The site turned out to be a $100M business called Yahoo Personals.

We weren’t looking to do InnerSourcing, we simply wanted to rapidly test the market with low risk. InnerSource was a solution. Now the cool side result: The team working on that site also learned a lot about Yahoo Mail, Search, Sports, and Autos. We were looking for love, but we actually found ourselves engaging in InnerSource. InnerSource worked because the goal was high speed and low risk. No one lost control, everyone benefitted.

Slide 10: A Story of Transparency

Story #2. The next story is about transparency.

Unlike many companies, Yahoo did innersource by default. The policy was always that everyone can see each other’s code and theoretically contribute. And with Github, things got even better… until we found ourselves with the need to improve engineering quality and efficiency.

Our Developer Products and Services team (DPS) in this picture initiated a project to improve code review and metrics. Doing so, we found that many repos were private — which is a big problem for InnerSource. Without transparency of code, you can’t have InnerSource. So we ran a nightly job to force repos to be public.

This made some people upset, which is OK. As it turns out many people said they needed their code to be private. We set up a review process to determine if this was true. As it turns out, of the hundreds of repos that “quote” needed to be private “unquote” we found just two cases that were deemed legitimately private and we whitelisted them. We now have literally hundreds of thousands of repos that are visible to all internal engineers.

The truth is, just because things were open, developers were not contributing to random projects. Technically they could, but why would they? We gave out prizes and recognition, but we still didn’t get tons of interactions between teams that had nothing in common. We saw a lot of forking, building your own, but not great contributions. Was this a success?

Getting everything in the open made engineers do a better job of testing and writing code that passed scrutiny. This created a culture of transparency and trust that is critical to a proper engineering organization. The real value came during the tough times when many people left the company and we faced the need to deal with tech debt. Having the InnerSource culture lowered risk and improved our ability to fix products when the key people were working on other things, or elsewhere.

Slide 11: A Story of Reversal

Some people think you first InnerSource and then you Open Source — since it appears to be less risky. It’s the opposite. We had a team who build a pub/sub messaging technology that had superior scalability over Apache Kafka. They needed it for an ad system. They were very proud of their work, but the ad team did not want it. They wanted to use Kafka and were willing to rearcitect their product to make it work. It seemed silly to me — why redesign your product when you could use the solution that was built to handle the load. So I went to the Ads team to ask them.

Turns out, they did not want to use internal technology — they were burned once when the platform they used lost internal support and they were stuck depending on a technology that our central platforms team abandoned. So they only wanted to use open source.

I went back to the platforms team and ask them if they’d open source their code. They were thrilled. First, they had to make their code better. Bingo, that was a win. They did and we open sourced it. We got a couple of companies using it since it was indeed better than Kafka in many ways. The ads team loved it too since it was open source and they didn’t have to redesign their product to avoid obsolescence risk.

This is the story of Apache Pulsar. We open sourced it first, then we InnerSourced it.

Slide 12: Story recap

Like the Yahoo Personal’s story — use InnerSource to experiment and deliver quickly. Like the DPS story — use innersource to manage risk. Like the Pulsar story, remember that teams need some sense of control to manage their products, so you can use InnerSource with Open Source to give them control.

Slide 13: Take-aways

InnerSource is the application of Open Source in the corporate environment. We’ve found that you have to start with the Corporate goals: speed, low risk, and control. Then you add the open source values.

Transparency, symmetry, and non-exclusivity. Although open source is not natural to the corporate environment, if you focus on speed, risk, and control, you’ll be able to add open source values to the mix.

One way to know you’ll get there: see if your InnerSource program itself is transparent, symmetric, and inclusive. Ask yourself, can all employees in your company see how your program operates? Do they have a say in how it gets rolled out? Are you focused on helping them move quickly, reducing risk, and enabling them to maintain control over their outcomes? If so, we’re pretty sure you’re on the right path.

Thanks for letting us share our stories, and we’ll take questions. If you see us around, say hi and tell us about yourself and what you are doing.