Scaling up your engineering organization, part 1: The team owns it all

We’re all experiencing it: Technology in our world is changing rapidly.

We’ve gotten very good at developing scalable software services. Virtualization and cloud services have cropped up, providing us with tools and best practices. Everything is becoming elastic. And yet, the one thing you still cannot scale horizontally through cloud services is your engineering team.

Every time you grow the team new challenges come up. How do you onboard new employees? How do you safeguard the company culture? How do you maintain high standards in your coding and infrastructure? How do you keep your teams independent, engaged, quick to deliver and yet aligned with the company’s business goals?

In this blog series, I’ll lay out some of the lessons we learned and applied at Kenshoo over the past five years, while we grew our engineering team from ~40 to ~160 engineers.

Lesson 1

Make sure your engineering organization is flexible enough to align with the changing business requirements and can execute business initiatives without heavy coordination.

Where did it all begin?

As with any organization, we started with one team in one room developing the product. It all goes well, the communication flows and each team member becomes an expert on a subject.

Then, as the product and the company grows, you start defining your components and architecture and start forming teams.

What did a Kenshoo team of ~40 developers look like?

Teams are organized around components. Feature development is split among teams as required.

It’s quite a simple structure, but it gets complicated. Each team consisted of 5–6 developers. Architects were doing DevOps and helping keep teams synced. Product managers were pushing projects forward and coordinating shared resources as needed.

What problems came up as this structure grew to ~60?

1. Dependencies. Projects are split into user stories that go in the team backlogs. For example, adding support for a new Google ad type means changing the Data Model (Campaign Management team), the API integration (Publisher APIs team), tracking and reporting for the new ad type (you guessed it- the Tracking & Reporting team), and so on. The product manager working with the team tries to juggle the priorities but things never get aligned. Some teams, especially those in charge of commonly used infrastructure like our Tracking team, are involved in pretty much every feature, blocking the critical path so entire features are blocked. Once you find yourself you’re discussing an individual developer’s timeline with your VP Product, you’ve got a problem!
2. No end-to-end responsibility. Teams don’t own the implementation end to end.
3. Inflexible engineering org, with an inability to move agendas or developers between teams. We’ve tried moving developers between teams short term, but it rarely worked. It required convincing the developer’s manager, getting the developer’s blessing to move, and then moving that developer back to their original team.

What’s the remedy?

Switch to feature teams. You might be thinking, “hey, we’re already structured as feature teams, working in agile SCRUM and using all the best practices”, but in reality a feature is not owned by a team. Our different product components had distinct owner teams (well, most of them did) , but features work across components.

A real feature team is one that can ideally deliver any feature required by the business. (Read more about it here).

So, we defined a new entity: Initiatives. An initiative is a set of features that share a common goal and can be executed by a team in an average time of 6 weeks. Initiatives are managed in a KANBAN manner and any team who’s free can take the next initiative.

But wait, how will the Application team develop a feature that needs to change or add code in the API and Tracking teams’ components? The simple answer is — they just do it. The design and the code will be reviewed by the component owner, but the team that owns the initiative will do everything needed to deliver it.

What’s the takeaway?

Always look into your organization and ask : Are we flexible enough to deliver business value? And remember, independent teams are more efficient. You just need to make sure the objectives are clear.

Interested in reading more on the subject? Take a look on Part 2