Martin Müntzing

…into meaningful cohorts and try to deeply understand the impact of the change you made. In general, experiments should be run to make good decisions about how to improve the product, rather than to aggressively optimize for a metric. Optimizing is not impossible, but often leads to opportunistic decisions for short-term gains. By …

It’s easy to decide not to test a change — you’re busy, you just know it will be better, you don’t want to risk the confusion that’s always possible anything you’re running a split test. …