Member-only story
Propensity Score Matching (PSM) for A/B Testing: Reducing Bias in Observational Studies
A comprehensive guide to implementing PSM with your experimental data, including Python code
A/B testing is a widely used experimental design in which two or more interventions are compared on an outcome of interest. The goal of A/B testing is to estimate the causal effect of the interventions on the outcome, while controlling for potential confounding variables. Randomisation is often used to achieve balance between the treatment and control groups, but it may not always be feasible or sufficient to achieve balance on all relevant covariates. As a result, the estimated treatment effect may be biased due to differences in the characteristics of the treatment and control groups.
Propensity score matching (PSM) is a statistical method that aims to reduce the bias in the estimated treatment effect by creating comparable treatment and control groups based on their propensity scores. The propensity score is the conditional probability of receiving the treatment given a set of observed covariates, and it summarises the information about the covariates that is relevant for estimating the treatment effect. PSM matches individuals with similar propensity scores in the treatment and control groups, which can balance the…
