Double the pain, double the gain

The stories that take twice as long to write tend to perform nearly twice as well

It sounds obvious: when you spend more time working on a story, that story should perform better. But now we have the stats to prove it:

Doubling the time spent drafting corresponds to an 89% increase in Total Time Read.

To calculate this, I began by plotting the time authors spend writing stories (the “draft times”) by the Total Time Read (TTR) for the first 30 days after those stories were published (“30-day TTR”). When you write on Medium, we automatically save every few seconds as you make changes, so we can add up the time between those saves to calculate the total draft times. I also used 30-day TTR rather than all-time totals because it helps us compare new stories to older ones. (I only included stories that were at least 30-days old.)

In this chart, every dot is a story. I heavily cropped both axes to more clearly display the data, but even so the dots are bunched up along the bottom left. Logarithmic axes fixed the problem:

The data is much clearer now (again, cropping the axes), but the variance still produces a fuzzy, blob-like shape that is hard to interpret. Similar to the method I used for the optimal post length analysis, I bucketed the stories in equal increments and calculated the medians. The median serves as a better middle-value than the average because it isn’t skewed upward by the high-end edge cases.

Not bad! If you squint, you’ll notice the shape of a line:

Manually-drawn blue line to show the linear trend.

To understand the slope, I ran a linear regression. Unsurprisingly, it fit really well with significant results. Here’s the resulting equation:

We can interpret this by saying, “an 88.9% increase in x (draft time minutes) corresponds to a 100% increase in y (median 30-day TTR).” And there we have it — stories that take longer to write tend to see more TTR.

But—correlation, not causation!

While a strong correlation exists, we can’t say for sure that the extra TTR is a direct result of the extra draft time. We actually have a few hypotheses:

1. Well, maybe it is. We hope that when authors spend more time writing, they end up with stories that are more engaging, resulting in more TTR.
2. When authors spend more time writing, they might also feel more invested in the finished product. That might lead them to more actively share and distribute the story, resulting in more TTR.
3. Longer stories take longer to write, and they also tend to see more TTR. So we might just be seeing the positive correlation between total TTR and story length (but, of course, only up to 7 minutes!).

There are probably more possible explanations too. Please let me know if you think of any! As with most analyses, I ended up with more questions than when I began—we’ll just have to keep digging in.