Pinterest Engineering

Apr 10, 2015

8 min read

Learn to stop using shiny new things and love MySQL

Marty Weiner | Pinterest engineer, BlackOps

A good portion of the startups I meet and advise want to use the newest, hottest technology to build something that’s cool, but not technologically groundbreaking. I have yet to meet a startup building a time machine, teleporter or quantum social network that would actually require some amazing new tech. They have awesome new ideas with down-to-earth technical requirements, so I kept wondering why they choose this shiny (and risky) new stuff when all they need is a good ol’ trustworthy database. I think it’s because many assume that building the latest and greatest needs the latest and greatest!

It turns out that’s only one of three bad reasons (traps) why people go for the shiny and new. Reason two is people mistakenly assume older stuff is slow, not feature rich or won’t scale. “MySQL is sluggish,” they say. “Java is slow,” I’ve heard. “Python won’t scale,” they claim. None of it’s true.

The third reason people go for shiny is because older tech isn’t advertised as aggressively as newer tech. The younger companies needs to differentiate from the old guard and be bolder, more passionate and promise to fulfill your wildest dreams. But most new tech sales pitches aren’t generally forthright about their many failure modes.

In our early days, we fell into this third trap. We had a lot of growing pains as we scaled the architecture. The most vocal and excited database companies kept coming to us saying they’d solve all of our scalability problems. But nobody told us of the virtues of MySQL, probably because MySQL just works, and people know about it.

Through the gauntlet, two of the most important lessons I learned building Pinterest were:

After about a year of fast, sleep-defying scaling at Pinterest, we had MySQL, Memcache, MongoDB, Redis, Cassandra, Membase and Elastic Search. Everything was on fire and breaking in their own special ways. We wanted to simplify and get rid of all the fancy stuff, but we also wanted something that would scale to the moon with us. It was time to start our grand re-architecture. To help guide us and our choices, we built a set of questions to apply to every different technology:

Maturity is natural

The harder and more passionately people push on a technology, the faster they will run across bugs or performance problems, fix them and hopefully contribute fixes back for the whole community to use.

Maturity can come a hell of lot faster if the technology is simple. Getting a five line Python program to print “Hello, World” to be bug free should take 30 person-seconds, whereas a program to handle world-wide distributed banking transactions will take many person-years to mature due to the complexity that must be hammered out.

So, let’s equation-ify this. Maturity increases with blood and sweat, but comes slower with more complexity.

Maturity = (Blood + Sweat) / Complexity

(I really wanted to throw in some e, i, and π into the equation, but just couldn’t justify it. Yet…)

As a technology hardens, collaboration occurs, understanding gets deeper and a wealth of knowledge is built out. As the technology itself gets more stable and beautiful, documentation and simplification occur and the frontiers and boundaries are tested and widened.

You don’t want to be on the wrong end of the maturity equation. There be dragons there:

Sometimes you have to be on the bad end of this maturity ratio. For instance, if you HAVE to have a Flux Capacitor for your time machine, recognize that you won’t be able to hire for it easily. There will be minimal community online to help you debug why you only went back in time 100 years and not 1,000 years. Support may not be able to help you understand why Marty McFly is now stuck in a supernova.

If you’re on the frontier, you’ll hit new bugs and issues that the rest of the world has never seen. They’ll be 10x harder to debug and will likely require a depth of knowledge that goes outside the comfort zone of your current engineers. You’ll have to dig in, push hard and learn fast. I send you my virtual hugs and admiration. I’ve been there. It will be tough. Blog what you find, collaborate and communicate.

If you’re starting or growing a company, and your scale is smaller than huge, consider maturity to be your most important factor aside from basic requirements. Ask yourself — does MySQL sufficiently meet my needs? If so, use it. If you’re wondering if MySQL will be fast enough, the answer is YES. Even better than fast, MySQL’s performance will be consistent.

Last Remarks

So I’ve wailed away on a bunch of technologies, but I seem to have a near-romantic thing for MySQL. I’d like to take a moment to mention that MySQL, while mature, does not solve all your problems. Sometimes you’ll have to venture away from the comforting warming glow of maturity.

Marty Weiner is an engineering manager at Pinterest. Follow his adventures on Twitter and Pinterest. Interested in joining his team? Check out our careers site.

For Pinterest engineering news and updates, follow our engineering Pinterest, Facebook and Twitter. Interested in joining the team? Check out our Careers site.

Inventive engineers building the first visual discovery engine, 200 billion ideas and counting.