Why I Open Sourced Five Years of Work
One click and then — that’s it. Irreversible. Five years of our work released into the web, forever public.
My team and I celebrated with a champagne toast once our codebase was public (Here it is.) We knew this was the right choice. Our biggest debate was whether we should have done it years ago. But then I went out with some non-coder friends and was greeted with shock when I told them what we’d done. “You gave all your work away? For free?” the bankers, lawyers, and doctors among my friends asked. “How will you make money? Anyone can just use it?” Even my parents grinned sort of nervously when I told them.
So why did I decide to do this? In short, I want Ufora to be bigger than me and my company. I want everyone doing data science in Python to use it. I want Ufora to live and breathe on its own.
It’s a pretty big change: open source is the polar opposite of the standard “proprietary software” model. With open source, you make all of your code and documentation public. Your software license lets anyone use it for free. As a company, you charge for support and development of custom features instead of software licenses. In return, anyone can also contribute to the project. The community can use our software without worrying that we’ll increase the price, or go out of business. So we give, but if we are compelling enough, we also get back.
Open sourcing a project is increasingly common in the tech world. In fact, for some kinds of projects it’s almost obligatory. The more complex your software is, the greater the need for an element of public evaluation. If you’ve had a breakthrough, if you’ve built software that looks like magic and you want others to build on top of it, it’s no longer enough to say “trust me, it works.”
At my company, Ufora, we have had just such a breakthrough. We figured out how to automatically parallelize calculations over humongous datasets that are written in Python. Adoption of our system can save companies in every industry months of development work because modelers will no longer have to figure out how to split problems up over lots of computers. We’ve automated it.
It sounds too good to be true, impossible even. That’s been the consistent response we’ve gotten from many people we’ve met with. This is hugely frustrating because, in fact, it is possible, and we did it. But it looks like magic, and software developers hate black boxes they can’t see inside. At some point in the last couple of months, Ufora’s head of engineering, head of strategy, and I looked at each other and we just knew: It was time to let people look under the hood.
The growing importance of open source code in computer science makes sense within a broader trend in our economy—the general public expectation of greater transparency from business and government. Wikileaks left a legacy that people should expect to see inside the machinery of government, and developers are demanding the same thing of the software they use.
Some of the largest technology companies today are open sourcing parts of what they do. Google’s mobile operating system, Android, is open source. Linux, which runs some huge fraction of the websites you visit, is entirely open. (The company that offers enterprise support for Linux, Red Hat, has a $15 billion market cap.) Twitter has over a hundred open source projects published on GitHub. More importantly, almost every tool I use in my work as a software engineer is open source. Twenty years ago, open source was a fringe movement. Today it’s the dominant ecosystem for software infrastructure.
If you run a record mile time, you have to do it where someone can see. If you paint on the caliber of Miró, you have to be viewed by others to be recognized. Today, as a developer, if you do something that is truly new and ground-breaking, it needs to be public.
So here I am, giving away the product of years of work. In the coming months, I’m looking forward to booting up our new business model. But for now, I’m just enjoying being starred, forked, and cloned. (Starring is praise of the project; forking and cloning are downloading it for use and modification.)
At Ufora, we fervently believe that open sourcing our work will lead to more organic adoption than our small team could have ever gotten through in-person meetings, selling our technology as a black box. It’s a harder road — we don’t have the lure of the seven-figure software contract in front of us. But the next time I demo Ufora, and someone in the audience asks me how Ufora works, I’ll be able to show them everything. That’s exciting.
