Experimentation

This is a TL;DR of a much longer article. You can read the entire article by going to the Invesp blog. Experimentation is a vital tool for growth in modern businesses. But, it’s important to measure the impact of these experiments to know if they are worth the time and…

This week I learnt that I have been confusing people for years about what a hypothesis really is. …

Deep dive into MLRATE - machine learning regression-adjusted treatment effect estimator and comparing it to other methods — This is the second post in the series of articles where we are discussing variance reduction in experiments. In the first post we discussed why reducing the variance of our outcome metric is necessary in experiments and showed how simple regression adjustment can result in substantial benefits as well as…

Let’s say you want to display a coupon with a modal dialog that has a button to redeem. In a feature addition like this, you may want to put it behind a feature flag. A feature flag is a way of wrapping a feature so that it can be remotely…

I’ve enjoyed the last 8 days of writing. I haven’t done anything like this before, so it’s been a fun, new experience. But I’ve also noticed that I am taking way longer to write than I would like. Perhaps some of this is expected in recovering from perfectionism. Maybe people…

Marianne Stjernvall, Founder of Queen of CRO, discusses the potential drawbacks of personalization, the significance of using standard terminology, and how to effectively manage personalization as part of a CRO program. — In Personalization Talk #13, we interview Marianne Stjernvall, the Founder of Queen of CRO, a consultancy specializing in CRO, personalization, and experimentation. During the conversation, Marianne discusses the potential drawbacks of personalization, the significance of using standard terminology, and how to effectively manage personalization as part of a CRO program.

Tips and insights to make you a better decision-maker. — Cassie Kozyrkov in her interview in the Author Spotlight series “Making Good Decisions: An Art, a Science, or a Bit of Both?” defined good decision-making as the act of turning information into better action. She pointed out the relationship between decision-making and data science: one depends on the other to…

In most of the companies I work with, the decision to develop a new product is made based on an assessment of the likely revenue and margin the product will create. This assumes a clear specification of the intended functionality of the product as a basis for the cost estimation…

The intuition behind variance reduction and why it is important in randomized experiments. — This is the first part in a series of two articles where we are going to dive deep into variance reduction in experiments. In this article we are going to discuss why variance reduction is necessary and build an intuition behind its mechanism. In the second part we are going…