The Culture Clash of Innovation

‘Innovation’ vs. ‘optimization’ require different organizational cultures that neutralize each other when mixed. But many companies need to do both. How can they succeed?

The most common scenario is large, mature companies who perceive a need to shift from a culture of optimization to one of innovation to reverse a slowing growth trend or a contracting customer base (e.g., AOL in 2011, Yahoo! in 2014). But startups also have a version of this problem, going in the opposite direction, when they try to move from an innovative culture intent on finding product/market fit to a culture designed for optimizing and scaling their recipe for success. I think this problem is also at the heart of the so-called “founder’s syndrome”. Either way, mixing cultures of optimization and innovation wrecks both yet it is inevitable in the lifecycle of every company. This article will show you why this happens and what you can do about it.

I will focus on consumer Internet companies, because this is what I know. (I have been involved in product labs teams in late-growth enterprises for over 12 years.) The cycle time is accelerated in the tech industry but the observations presented below should be applicable to companies in most industries, many of which have had to solve versions of this problem for a lot longer than tech companies and may be better at it (comments welcome).

What do we mean by optimization and innovation?

Optimization and innovation, as used here, have a cyclical relationship in the lifecycle of a company. The boundary between optimization and innovation is product-market fit.

I define product/market fit as that point when your venture is consistently EBITDA positive (profitable), with customer growth greater than the average for your market (i.e., your venture is doing better than coasting), that is sustained by keeping on doing what you’re doing. To clarify, this is in contrast to experiencing declining, stagnant, or volatile growth that forces you to change your strategy every month or every quarter.

The irony that creates the cycle is that optimization will necessarily create the conditions under which you lose product/market fit and it becomes necessary to innovate again. But when that situation arises, your company will have lost the ability to create innovation. The cycle is death to most companies in the late growth phase (even if they continue on as zombies), but the greatest companies learn to master it. This is particularly true of Internet technology companies, many of whom are hitting this point in the cycle for the first time in their industry history.

An optimization culture means that everything you do from how specialized your workforce is (engineers, product managers, designers, QA, etc.), to how we measure and test (e.g., A/B tests on DAU, engagement, revenue, and retention metrics) is geared for incremental, consistent, risk-free, most bottom-line-enhancing improvements coordinated across the company’s platforms to meet executive targets.

All mature, well-run product organizations in competitive markets operate this way. It explains their pace — they can move only as fast as they can A/B test small isolated changes. It explains why they love imitation — copying the the competition is a risk-free way to compete — and why any field of competitors in a mature industry all offer the same thing, even though this worsens their long-term differentiation and creates the need again to innovate to improve growth and margins.

The problem is that when that need to innovate arises again, the type of product-development culture that made the company scale up to this point excludes innovation. This is by design. Optimization-focused cultures need to make it very difficult for risky, untried, and untested ideas to be prioritized and developed. If it doesn’t make it hard, all kinds of unforeseen delays, risks, and process incompatibilities arise.

When a strong optimization culture tries to innovate within itself, it tends to measure the wrong things and draw the wrong conclusions. Project, team, and company objectives become incoherent, causing the organization to vascillate and lose focus. Mission, vision, and quarterly objectives become disconnected. Resources are not allocated efficiently. The core business is neglected and suffers. Morale declines. Management credibility deteriorates. The best people leave.

The result is that any organization that tries to operationalize innovation within the same values, processes, structure, and resources it uses to drive optimization will 1) fail to get the innovation work prioritized at all (product managers in optimization cultures typically struggle to devote more than 10% of their resources to innovation), and 2) if they succeed (usually by executive order), they will both fail at innovating and gradually lose the ability to optimize. AOL and Yahoo! are well-known examples of this phenomenon but it’s a universal problem driven by rational self-interest that catches up with every company eventually. (I’d be interested in hearing about counterexamples.)

Differences between cultures

What follows is an exhaustive set of over 30 themes that describe the different goals, values, processes, and behavior of optimization and innovation cultures. Optimization is on the left side of the comparison; innovation on the right.

1. Quality: Everything up to brand quality standards vs. cheap, low-end, except for the innovation itself (but it doesn’t need to scale)
2. Org Structure: Horizontally integrated org (design, PM, eng…) vs. vertically integrated org (one team can do everything)
3. Resource Management: At least four different teams to get anything done (design, PM, eng, QA); specialists vs. small scale, a couple of people, generalists
4. Justification: Trends and statistics at 90% confidence intervals can tell you what is working vs. Small data sets, intuition, empathy, anecdote, experience
5. Precision: Small improvements are detectable vs. only big improvements are detectable
6. Volatility: Sensitive to metrics volatility (spend upwards of 20% of time investigating unexplained deviations) vs. relatively insensitive to volatility (because the numbers are usually small, or trend up-and-to-the-right, or too many changes going out at once)
7. Learning: Permute small combinations to choose the one with the best metrics and tradeoffs vs. iterate hypotheses based on insights from previous hypotheses to get traction
8. Failure: Failure does not tell you much about how to succeed vs. failures are more informative than successes because they invalidate hypotheses that give you clues about how to succeed better next time
9. Leverage: 80% of the effort generates the last 20% of value vs. 20% of the effort generates the first 80% of the value
10. Pace of Change: Design changes little per iteration vs. design can drastically change per iteration
11. End Results: Incremental DAU and revenue are the end results vs. product/market fit (growing faster than the market and EBITDA positive)
12. Commitment: Personal investment in the idea is not very important for success vs. personal investment is critical for making an idea succeed
13. Telic Style: Planned, discrete projects vs. serendipitous, continuous tinkering
14. Launch Style: Hard launches vs. partial releases and soft launches
15. Cognitive Mode: Rational, controlled, legislated vs. creative, artistic, visionary, emotional
16. Specialism: Specialist, narrow, disciplined vs. generalist, multidisciplinary, amateur
17. Side Effects: Changes can have large side-effects for other products and teams vs. changes have no side-effects for other teams
18. Imitation: Copying the competition is sound way to enhance your product with low risk vs. trying what no one else in your product category has done
19. Markets: Competing against an existing market vs. creating a new market
20. Fitness Function: Local maxima (gradient descent) vs. global maxima (Lévy flight)
21. Estimation: Can’t estimate without a similar benchmark to compare it to vs. uses a wide variety of qualitative and quantitative methods to estimate value without the luxury of benchmarks
22. System Complexity: Maximize one variable while holding others constant vs. optimize a set of interdependent variables
23. Maxim: “Take care of the pennies and the dollars will take care of themselves.” vs. “Fortune favors the prepared mind.”
24. Dependencies: Tends towards modularity everywhere (e.g., platform vs clients) vs. tends towards interdependence especially where the innovation is most competitive or disruptive
25. Cost/Reward: Bell shaped (two-thirds of the cost gets you an average amount of reward) vs. Convex shaped (optionality, as costs go up linearly, reward can go up exponentially)
26. Significance: Small changes affect the behavior of large quantities (millions of users) vs. large changes affect the behavior of small quantities (hundreds of users)
27. Prioritization: The largest increases in absolute users & revenue wins vs. the largest amount of learning about what to try next wins
28. Technology: Widely-adopted, well-supported, common technologies vs. experiments with the latest tech for minimizing development time and maximizing capabilities
29. Cost structure: Trades variable cost for predictable volume at discount vs. trades discounted volume for variable costs at outset
30. Outsourcing: Prefers in-house core vs. outsource everything that can be efficiently outsourced even if it won’t scale
31. Scalability: Scales vs. doesn’t scale
32. Management: Top down vs. bottom up
33. Type of creativity: Permutations of related or similar components or products vs. “Exaptation” (transrational borrowing from unrelated products, areas, disciplines, industries, epochs)
34. Feedback: Positive feedback loops dominate (keep doing what you are doing) vs. Negative feedback loops dominate (change direction)
35. Investment: Based on known return of the product vs. the ability of the team to consistently reduce uncertainty by trying many small, creative, cheap experiments with large potential upsides and small downsides, until value is proven.
36. Patience: Patient for growth, impatient for profit vs. patient for profit, impatient for growth (Clayton Christensen).
37. Rewarding staff: proportional to effort (bonuses and raises) vs. disproportional to effort (equity).
38. Competition: likes competion vs. dislikes competition (would prefer to change the game to something competitors can’t play).
39. Procurement: Purchase orders vs. credit cards.

Buying your way out of the cycle

All companies try to optimize their products but many large companies don’t try to innovate at all, preferring instead to acquire the innovations they lack at the point in their lifecycle when those innovations can be integrated into their mature optimization culture.

But opportunities are rare, hard to discover, the timing is often bad, their prices are usually high (if the market is competitive), most acquisitions cause collateral damage to both company’s cultures, most of the best talent cannot be retained, and in the end the investment usually fails to deliver the long-term shareholder value that was promised at the time of the acquisition. The larger and more innovative the acquisition relative to the parent company, the worse the outcome is likely to be.

For these reasons, it would be good for long-term shareholder value if companies mastered the cycle of optimization and innovation and used innovation acquisitions as last resort.

One way companies try to cope with the cycle is the Internal Startup Model, which we shall evaluate next.