Product Development — Taking cues from the Bees Algorithm
The Explore / Exploit Trade-off
In business, you are always juggling the classic conundrum of explore versus exploit.
Take marketing, for instance. Find a channel that’s working and the temptation is to milk it for all you can. But here’s the rub. Channels aren’t eternal springs. At some point, every spring runs dry. You can’t afford to go all in on a single strategy that may dry up or hit a threshold at any point.
Now take the opposite example. if you’re spreading yourself and your team too thin, you probably have opportunities in front of you that aren’t being exploited to their full potential.
Most people intuitively understand some percentage needs to be explore and some part exploit. But how much?
Looking to nature for inspiration
Interestingly, this problem also presents in the world of honeybees. There is even an algorithm named after them — the Bees algorithm.
Some small percentage of the colony will act as scouts. Flying in random patterns, they search out new food sources. When a scout finds a profitable food source, they return to the hive and perform what’s known as a Waggle Dance.
The waggle dance itself signals a food source and the duration of the waggle dance signals how plentiful the scout believes the food source to be.
The Bees Algorithm
A pseudocode algorithm is captured on the Wikipedia page for the Bees Algorithm
It’s not super important to understand the ins and outs of this algorithm for our purposes. There are a couple of interesting pieces that it surfaces up:
- There is a two tier search algorithm. A global search which is where the scouts go find pastures new. And then a local search which is where they recruit foragers to do a local search of a flower patch they have found some food in.
- No matter how many food sources are currently being exploited, scouts are always searching for new global sources of food.
Unlike in this algorithm, in the real world, the ratio of honeybees scouting to foraging is not static.
When food is plentiful, they reduce the number of scouts and when it’s less plentiful they increase them.
Takeaways for Product Development
Exploration is critical
This is doubly true for technology companies.
The list of items on your roadmap gets larger and larger. It starts with customer features and bugfixes when you’re smaller. Then it’s performance. Documentation. Security. Localisation. Compliance. Accessibility. Analytics. And on and on.
There must be enough slack in the roadmap for two critical items: technical debt and exploration.
Monitor and adjust
However you end up allocating resources, it’s important to adapt. There is natural seasonality in some businesses leading to critical times of the year for roadmap and operationally.
Early stage
Pre-product market fit.
Here, you want to be massively overweight future innovation and features. Every customer is new. Sometimes it’s obvious what roadmap needs to be built and it’s just giving enough time to execute on it. Sometimes it’s very blue-sky like R&D.
A simple pareto principle (80/20 rule) can work well here, where the vast majority of your efforts should be working on what gets you to product market fit. That might be a blend of R&D and feature development. Maybe it’s 90/10 or 70/30 depending on where you are. Adjust to suit.
Growth Stage
Product market fit achieved and growing > 50% YoY
Rule of Thirds (adjusted) aka Rule of Fourths
The typical rule of thirds goes like this in the growth stage:
1/3 New Features, 1/3 Maintenance and Iterations, 1/3 Technical Debt and Infrastructure
This doesn’t leave room for our other box.
So, if you’re being aggressive on investing in innovation you can move to equal parts 25% or a less aggressive view might be 30%, 30%, 30% for the original buckets and 10% for other.
Mature
70–20–10 Principle
Google talked a lot about this in the early 2010’s. Eric Schmidt is famously quoted as saying Sergey Brin proved the “other” bucket (innovation bucket) needs to be 10% with mathematics.
The 70–20–10 model has historically been used in fields like finance and investing and even as a model for household budgeting.
Simply, it overweights the core work you do (the 70%), allows room for adjacent work (20%) and then allocates a small, but meaningful percentage for unrelated work (10%).
Brin supposedly proved that for this model to work, the unrelated work bucket must be at least 10%.
Support vs Growth
Another helpful rubric to think about time allocation is to align with the top line growth of the business.
If you’re at a mature SaaS company growing 20% YoY, then you align your growth bucket towards that number and 80% of your time is allocated to support.
I’ve seen these figures like 60/40, 70/30 and 80/20 at larger companies.
Why does the 10% Matter?
I suspect Eric Schmidt’s comment that Sergey Brin “proved” that the other bucket needed to be a 10% is a little overstated. You definitely need to make some assumptions to get to that figure. But it may be that they had enough data on the bets they had already made.
Arriving to this figure isn’t quite as difficult as it’s positioned. At it’s core is a concept called Portfolio Theory. This is another model once again borrowed from finance and investing.
The idea is that not all bets will pay off. Secondly, bets that do pay off will pay off to different degrees. Lastly, particularly in software markets, some small percentage of bets will have a large asymmetric return. Meaning that they could return 10, or 100 or 1000 times the original investment.
Wrapping up
Resource or capital allocation is a frequently talked about topic in finance but less so in development and engineering. It’s great to be able to borrow some thoughts from other areas without trying to reinvent the wheel.
One thing to look at is your own allocation. Where are you putting resources? How much is allocated against each bucket? Are those buckets the right size?
Smaller companies frequently look at larger companies and think about the endless resources they have. Ironically, if you talk to most line managers in these larger companies, they will most likely complain about being understaffed.
It’s important to understand for any company in growth mode, there will always be vastly more work than people. Trying to complete all work is a fools errand.
Ultimately our job as leaders is to prioritise correctly as much as it is to execute.
