Coders Should Think Like Scientists, Not Like Engineers
Lessons from product management to improve your outcomes.
tl;dr
- Writing code has clear outputs and expectations.
- For impactful work, think of everything you do as an experiment.
- Use the Experiment Template to clarify your thinking and measure progress.
Writing code for the first ten years of my professional career was rewarding since I could clearly and quickly see progress. I could write and deploy to production the same day and immediately see my work’s results, even when there were bugs. Instant dopamine hit. “If I do (x), then (y) will happen.” If I build this feature, then I’ll move on to the next exciting feature. If I refactor this code, then it’ll run ten times faster.
After changing careers to product management, I quickly learned that coding was only one step to achieving an outcome. I no longer had the immediacy of progress that comes with writing and shipping code. I had to shift my mindset away from building to different ways of gathering feedback and experimenting.
Measuring progress provides us with guidance by visualizing how close we are to our goals. But when there’s no way to measure progress, or it takes months to measure, it feels like you’re lost in the woods and always second-guessing yourself. Just ask any entrepreneur who works hard for weeks at a time, but doesn’t see progress. “If I build this feature, then users will use it” became “we spent weeks building this feature, and no one is using it.”
It’s easy to measure progress by writing code.
Coders enjoy the confidence that comes from knowing that “if I do (x), then (y) will happen.”
Just like clothing, code has a tactile feel to it. It can feel rough or smooth, uncomfortable, or soft. You can tell the difference between an elegant one-liner versus 20 lines of nonsense, even though they accomplish the same thing.
This feeling works to a coder’s advantage because of how quickly it comes. You know that feeling you get when it works for the first time (“it’s ugly but it works!”) and the progress when you push a button to ship it to production. Seeing progress goes deeper once it’s in production; you can immediately see how fast it performs and how many people are using it. Measuring progress only takes minutes and hours, rather than days or weeks.
Another advantage of this type of progress is that it’s very black and white. The code either works, or it breaks. It has bugs, or it doesn’t. It’s fast, or it’s slow.
But you’ll soon find that this mindset, the one of an engineer, only leads to frustration and disappointment. Taking on an experiment mindset clarifies your thinking, makes sense of ambiguous results and separates your ego from the results.
A/B testing to the rescue?
People love the precise results from A/B and MVT testing. With a clean test setup and a single, actionable KPI, you can definitively say if your work made an impact or not. A/B testing is the most precise and definitive way to measure progress. It either worked, or it didn’t. It’s a tool that says, “I did (x) and (y) did or did not happen.” Of course, the results assume the test isn’t flawed, doesn’t conflict with other tests, and that you only have to run it once.
There’s a surge of confidence when you present the results to your peers or boss, even if it failed. This confidence is what makes A/B testing so seductive.
But measuring the impact of ideas isn’t black and white, and it isn’t always straightforward. The most impactful ideas can’t be wrapped up in an A/B test. And this is the problem. How do you measure progress on your work when the results are ambiguous? How do you avoid the feeling of being lost when you’re not sure of the outcome of your work?
Take on the mindset of a scientist and not of an engineer.
The most successful innovators, entrepreneurs, and product managers take on the mindset of a scientist; they write out what they expect to happen, go all-in with confidence, and when the results come in, choose to double down or throw out the idea. Every idea is an experiment.
“If I do (x), then (y) will happen” has a different meaning if you think like a scientist. It’s full of hidden assumptions that need to be uncovered rather than promises and guarantees of success. And it’s your job to tease out these assumptions and find a way to measure results while embracing ambiguity.
Thinking about “if I do (x), then (y) will happen” as an experiment does several things for us:
- It separates your ego from the idea, which permits you to take risks.
- You can tell others with confidence about the experiment, rather than making guarantees or promises.
- When the results are unsatisfactory, you’re more likely to investigate and learn rather than becoming frustrated and bitter.
Use an experiment template to clarify and articulate your thinking.
One challenge I find myself in is when I have a vague idea that is difficult to express. Or the opposite, when it’s a very narrow solution without an apparent problem.
To unblock me, I’ll often write the idea or concept out as an experiment. It moves it one step closer to a scientist’s mindset rather than an engineer’s “if I do (x), then (y) will happen.”
We believe that _________
To verify that, we will _________
And measure _________
We are right if _________
Source: Testing Business Ideas: A Field Guide for Rapid Experimentation
We believe that ______
Say it with conviction and confidence.
To verify that, we will ______
This is action and work you do.
And measure ______
Define what you want to get out of this.
We are right if ___
Draw a line in the sand for what is good, great, or just meh.
We believe that a chatbot will answer customer’s common questions
To verify that, we will install an off the shelf chatbot on our website
And measure the number of support cases opened
We are right if support cases drop by 30% within 30 days
I’ve found this template to be extremely useful in many situations:
- It turns a vague or narrow idea into something concrete and actionable
- It sets expectations upfront for what success is. Should 100% of our customers use the new feature? Or is 10% realistic?
- It articulates the tool used for the experiment. Should we run an A/B test, a survey of 100 customers, or build a prototype?
What’s next?
I hope you’ll come away understanding that writing code is often the easy part. The difficulty comes when you have to decide what to do before you write code and know if it made an impact. With a scientist’s mindset, rather than an engineer’s, the experiment template can help you define progress, measure it, and learn from your work.
Further Reading
Side note: progress is not linear
We often expect progress to be linear. At the very least, we hope it will come quickly. In reality, the results of our efforts are often delayed. It is not until months or years later that we realize the true value of the previous work we have done. This can result in a “valley of disappointment” where people feel discouraged after putting in weeks or months of hard work without experiencing any results. However, this work was not wasted. It was simply being stored. It is not until much later that the full value of previous efforts is revealed.
Clear, James. Atomic Habits (p. 22). Penguin Publishing Group.
