Ways to Balance Business Growth with Tech Infrastructure Stability and Scale

One of the more challenging situations tech teams face is building around, and on, brittle infrastructure. This situation can, and likely will, happen to most companies.

Why?

The business needs fuel

To exist as a company, and fulfill your mission, whatever that might be, you must create a means to fuel the business, usually investor cash or revenue streams. To acquire either you must build new features that can excite investors and bring in revenue (the latter needs to happen eventually). To build features, you must attract talent and ideally have a system to invest in your current talent to help them continue to grow.

The business needs to move fast

Building features isn’t always enough. Speed to market is critical, or your competitors may move faster and win. Idea iteration is essential to ensure you capture the best angles of your product and push them out to satisfy and engage your customers. To do this, software teams must move fast. Often this means they may have to build out MPVs (Minimal Viable Products). These satisfy the speed requirement, but there’s a catch.

The risks of MVPs

MVPs very often begin with accepted code debt (cut corners) and typically aren’t built for scale.

Yes, they get good stuff out in front of the client fast, provide a way to get data about an idea before it’s rolled out widely, and can delight customers and beat out competitors. The problem comes in when MVPs become the engineering norm. It’s very tempting to rinse and repeat the MVP cycle after it’s worked well a few times.

This is sometimes ok, particularly for an earlier stage company surrounded by the heavy risk of competitors and limited funding. But eventually, you’ll realize you built a decent house, and on top of that a suitable apartment, and now on top of that, you want a fancy castle. But castles are heavy, and that house is made of wood and sticks.

I think it’s clear where this metaphor is heading.

When the house collapses, it costs money, time, and much pain to the engineering team and the whole company. The good news is that well before this happens, whispers (not always whispers) will circulate in the engineering org and to management. “I’m spending so much time just keeping the lights on in our systems.” Or “I don’t have any time to clean up tech debt because I’m working so hard on project X or Y.” Or “I have no time to invest in technology.”

When this happens (ideally before it happens) it’s time to listen.

Ask your teams, where are the houses in your organization that need to be shored up to support that castle and allow us to continue to scale?

Dig into the highest risk areas first. Where do you need to slow down, staff up, and do some of the less fancy work that is ultimately the foundation of the business? Your technical infrastructure teams probably need some attention, and by restoring the balance between product delivery and technical investments, time, pain, and possibly attrition will be saved.

Here are a few thoughts on how to maintain this balance.

Know Where You’re Brittle

What areas of your business must be sound, and scalable for you to continue to grow as an organization?

Some examples:

• Do your engineers feel systems are stable enough such that you can double your order volume in 6 months? If not, why?
• Are you having to hire constantly to increase staff or seeing significant attrition in a particular department? Dig in. Are the systems they are using brittle and error-prone? Are there lots of workarounds folks must use to get things done? If these systems are in-house, are the teams supporting them adequately staffed to maintain them, and make them scalable? If the systems are off the shelf, should you invest in something more custom to your needs and expand your eng team to own those systems?
• Eng teams are bringing up staffing concerns, and sharing worries about code debt. Quantify the concerns, why do these teams feel overwhelmed? The answers will lead to cracks in the foundation and the ability to prioritize key infrastructure work before it’s too late.
• Teams are competing with one another for staffing and sometimes sharing that navigating the domain boundaries across tech groups is challenging. This may mean it’s time to zoom out and look at the org structure. Are there areas where systems can’t be thought of as much in silos, and teams must come together to build out a more holistic architecture to support some key area of the business?

Staff Appropriately

One way to get in trouble is by running too lean to allow for basic KTLO (keep the lights on) and general technical infrastructure work to ensure systems remain (or become) stable. Increasing staffing alone can help solve a lot of problems because it will give engineers time and breathing room to not only KTLO but come with recommendations on how to get ahead of scaling platforms for the business before the business scales beyond the capacity of the underlying platforms.

Although this can be expensive, staffing for breathing room is the preventative maintenance that may limit the need to dig into actual problems and smells after they already exist and are creating problems. One standard that can act as a forcing function toward platform stability is to staff your eng org such that X percent (say 30%–40%) of engineering time is focused purely on technical projects (they are likely already needing to spend about 30% of their time playing defense on tech debt while balancing product goals). The nice thing about this approach is it doesn’t necessarily mean more staffing (and higher costs to pay for more employees) but does require a willingness to “slow down to speed up”.

Communicate the Vision

Tech teams want the business to succeed as much as everyone else. Their concerns around system stability and staffing come from the same place as the drive of the larger org to fulfill its mission. Bringing tech teams along so they fully understand the mission, and beyond that, have a strong idea on the specific measurables the org is looking to achieve (scaling X% over Y months, ability to provide some new service by end of the year, going international, etc…) will help them bring the right recommendations to the table such that the underlying company infrastructure can support the future.

Help your tech teams turn that house into a strong foundation that can support any apartment, skyscraper, castle, or cathedral you’re looking to build.

Having been both an engineer and a manager during the course of my career, I’ve experienced the pain of moving too fast, the error of not listening closely enough, and also the reward when teams are given the time and support to keep infrastructure sound. I will continue to learn and iterate and hope this article is useful to others on their personal journeys as well.