Setting Up Tech Infrastructure For A Rapidly Growing Team: A Q+A With Hopper’s Head Of Engineering

Ken Pickering — Head of Engineering, Hopper

The Hopper Engineering Team has scaled by 3.5X since you started just over a year ago. Knowing your goal right off the bat was to massively scale the team, what steps did you put in place to set up the tech infrastructure to support the growing team?

Changing organizational structures is generally painful, but it’s almost impossible to scale by a large number and keep a lot of the established systems in place for perpetuity. 18 engineers working together do so a lot differently than 70 or 80. I think one of the primary drivers we need to account for as we scale is keeping an eye out for inefficiencies and change what needs to be changed.

Generally, if you believe in Conway’s Law, which says that “organizations…are constrained to produce designs which are copies of the communication structures of these organizations,” you need to take this into account as you grow and scale a business. While Conway was referring to specifically building software modules, I’ve found hacking organizational structure and the way teams operate can follow the same rules.

As we prepared to scale the team, we kept Conway’s law as a north star. It was important to me to set up a culture where we challenge internal decision making. I never wanted us to become a “that’s just how we do things here” company. With that in mind, we’re overhauling and rearchitecting our app, doing a pretty aggressively timed cloud migration, and have massively re-designed the technical footprint of entire products as Hopper has grown. This is to both produce the technical functionality our product requires and set us up to deal with more and more engineers contributing to the application. Essentially, the structure and processes you have with 18 engineers don’t really scale to 80 or more without rethinking a lot of what we do.

How do you structure the tech infrastructure for your more mature products?

We have two apps our consumers use: iOS and Android, and both apps need to provide a reliable experience no matter what. While we’re always looking at doing more within Flights, Hotels, and other product offerings, we keep customer obsession central to our decision-making, whether we’re working on one of our more mature products or testing a brand new feature.

We use an internal experimentation framework we’re actively iterating on so that we can always ramp our features up and down based on how much they resonate with our customers.

If we’re talking about a mature product, though, Hopper’s Flights product is where we start and is the thing Hopper is most known for. With air travel being such a complex ecosystem, the tech stack reflects that. We’ve been meticulous about dividing up the separation of concerns with microservices so our teams can focus easily on what services they need to work with, and keep it easy to introduce new ones.

And I should say, just because a product is more mature, that doesn’t mean we’re not actively investing in it or adding a ton of new features. It’s incredibly important for us to keep ourselves accountable and not fall into the easy trap of “that’s just the way we do things here.” Thus, we are constantly questioning our own internal decision-making processes. As our product rapidly scales and our customer base continues to skyrocket, we make sure that our flight product does not become stagnant, and that it continues to be able to scale with us.

What about structuring newer products, like Hotels and Insurance?

Building newer products offers more opportunity to analyze our processes and make sure that we are working efficiently. Newer features and products change more quickly than older ones, so we need to design both the infrastructure and the org charts to be flexible to change.

Our Hotels product, for example, has just gone through its first major redesign, and it uses completely different technologies than our flight product. We decided to leverage GraphQL because a hotel search involves much more complex criteria than a flight search. We needed to include the geospatial distance from a search location/geofencing and added different weights on hotel quality/amenities which can and will affect a result set.

When it’s determined that one of our newer products needs a new service, we also balance the weight of building one ourselves or using existing products designed by other companies. Of course, building it ourselves means we have control to create exactly what we want. On the other hand, using interesting prototypes in Firebase, for example, offers time-saving benefits and the ability to leverage more building blocks from companies like Amazon or Google to further offset product strategy. We take factors into consideration like which technology service the product would affect, what problem it would set out to solve and weigh the cost against the potential benefit to the team.

Honestly, though, we’re going full steam on our cloud migration strategy, which is one of our largest efforts to date… And probably most critical.

Why are we choosing to migrate to the cloud?

The same reason anyone goes to the cloud! Speed, efficiency, leverage of Platform as a Service, among a lot of other reasons.

Why now, though? Because we are quite literally running out of space and compute power on our physical, on-prem system. This is a perfect example of us having weighed the options between what we can create on our own and leveraging the building blocks from another company. In this case, we decided that migrating to the cloud is what will sustain us long-term as we continue to import billions of data points each day and scale our app product and user base immensely, with no slow down in sight.

Platform as a Service was really invented to help businesses scale in a uniform way. It just so happened to also be a pretty successful business model for Amazon, Google, etc. And, with Hopper’s rapid growth, it’s the perfect time for us to leverage it as well.

Stay tuned for a future Medium Article from Ken on our cloud migration strategy, including the business reasons for choosing a cloud platform and how Hopper’s engineers are doing the work internally to make this migration as smooth as possible.

— -

We’re on the lookout for smart problem solvers to join our team. Interested? Check out Hopper’s current openings