Simulate annealing as you build products and companies
Simulated annealing sounds awkwardly technical, but it is a genius technique that took physical observations and brought them to the world of computer science.
The physical version came in areas such as metallurgy:
“Annealing, in metallurgy and materials science, is a heat treatment that alters the physical and sometimes chemical properties of a material to increase its ductility and reduce its hardness, making it more workable. It involves heating a material to above its recrystallization temperature, maintaining a suitable temperature, and then cooling.
In annealing, atoms migrate in the crystal lattice and the number of dislocations decreases, leading to the change in ductility and hardness.”
The simulated type uses a similar high level idea. At high temperatures you have more randomness, and as the temperature drops the randomness drops. Using this technique in algorithms can allow us to do well by not stopping at local maxima.
Ok, so how do we bring this to the real world of engineering our products and experiences?
Startups and Innovation
When you are working on a brand new project that requires innovation, you will want to start with a lot of randomness. This means that you need the freedom to come up with crazy ideas and get prototyping on the ones that are best.
As you measure the experiments (via A/B testing and friends) you need to make sure that you are applying the appropriate level of randomness. This is hard to do, as we aren’t programmed for it. Instead, we tend to start measuring and walk up the hill in the direction that the data sends us. Thus, we often end up with local maxima. This is one of the problems that I have with how teams run A/B tests…. they tend to send one or two darts out and then iterate to the local win.
Standards
We tend to learn time and time again that the best standards come along after a lot of experimentation. Once that happens it is time to cool down and codify the best practices so we can get the benefits of consistency.
The best APIs come from learnings (e.g. jQuery => querySelector()) and the worst come from standards groups inventing (e.g. Enterprise JavaBeans).
Prototyping
If you aren’t prototyping, and getting many of those prototypes in front of customers, you aren’t going to deliver enough randomness.
The Web is a great way to deliver these prototypes, as the only code that is required to be on the users device is the generic web platform itself.
This doesn’t mean that you can’t do any of this work on native platforms, and indeed you should have a plan to do just this. It may even involve the app being a platform that can run various experiments.
UI Consistency
UI consistency is important for user success. If a user comes to your product for the first time and is able to do a lot without thinking, because you follow similar patterns, that is a great thing.
So, if you are building a new niche e-commerce company, you may not want to go totally hog wild on the checkout flow, and instead work within the boundaries. The downside is that you may not create enough activation energy to get out of the local maxima of that world.
J Curves
Whenever you put out a large change in the design of your product you often see a J curve effect [https://en.wikipedia.org/wiki/J_curve] where metrics dip as users work out the differences and re-program their habits.
As you watch the metrics head downward it is hard to know if you are just in a J curve that will bend up and beyond, or if things are just broken. Thus, you need to be able to test long enough, or have strong enough faith, to pull through to the other side.
Companies
The annealing effects aren’t just for products, they can apply to companies too. When a company is in an early phase there should be a lot of randomness as you try to find product market fit. Then as you grow and find a business model you will naturally slow down.
If you are fortunate enough to be successful the innovators dilemma will kick in and you will be fighting to add randomness which will often seem at odds with the cash cow.
If you stop adding randomness you are destined to stay at the local maxima and depending on how it compares to the rest of the ecosystem that may be OK for some time, but eventually the landscape may change enough that you are washed out by a competitor that found another, higher mountain top that enticed customers.
To combat this you need to constant add back randomness, and keep moving in the wrong direction to look for a larger peak. This will feel painful though. It may feel like the company isn’t full aligned. Some will take the path down and never make it up, and thus fail. Organizational hierarchies will keep changing and not everyone will like it. The only thing constant is change and this can feel uneasy, especially as it feels like change is accelerating these days.
Understanding simulated annealing has helped me handle this change and understand that even if sometimes you go backwards, you need to agitate and keep trying, else you will be stuck.
