Designing and evaluating metrics
Now more than ever, we need clear thinking about what measurements we should use to understand our world, our products, and ourselves. A metric is simultaneously 1) a designed artifact, 2) a lens through which we observe phenomena, and 3) way we set and monitor goals. The goal of this post is to articulate how I think about metrics (after spending several years on dozens of data science projects). I propose five key properties of metrics that represent key tradeoffs in the design process, as well as a depiction of the lifecycle of a metric.