When Teams Deliver Value to Users More Quickly — We See Impact on Many Levels

Co-authored with Archie Cowan

Our teams working on JSTOR have worked really hard the past few years to deliver value to our millions of users at a rapid pace while also embracing ownership of their process and quality with astounding results! Agile practices, quality engineering, and excellent technical execution — combined with good alignment across our organization — is something we have absolutely achieved.

One example to highlight is that we’ve seen consistent acceleration in the number of deployments to production. As you’ll see in the chart below, our deployments continue to increase year over year, with a projection to top 8,000 in 2017.

When we share this data with others, we are often asked if we see user issues within our JSTOR products increase due to the volume of changes we are continuously releasing. Up through 2015, the answer was yes. In part this was due to a re-platforming initiative and better tracking of defects found in our live environment. The chart below shows there was a positive correlation between the number of deployments and the number of production issues between 2013 and the end of 2015. At the end of 2015, we see the correlation change and for the better! Since that time we’ve observed a negative correlation between the number of deployments and problems our users encounter.

So, how is this even possible?

Here are some of the things we have seen take shape across teams over time through continuous retrospection and iteration:

• Empowered teams responsible and accountable for more
• Teams never stop retrospecting and iterating
• Every change is planned with the expectation it will be individually deployed to production, and smaller changes mean less risk and rapid value add for users
• More, smaller, higher quality user stories
• Better backlogs
• Smarter/more automated tests: every story increases our unit and functional test suites providing for immediate feedback loops to engineers
• We’ve made it easier for our teams to release new products and services with Sagoku—our deployment platform—and our state of the art continuous integration pipeline
• Teams don’t just have a “tester” to assert quality, they truly leverage quality engineering practicing allowing for less time on re-work and more time innovating
• Feature flags are used to release enhancements to production faster with reduced risk
• Monitoring tools are in place to help identify, troubleshoot and quickly mitigate problems

For us, smaller, more frequent changes to the platform do equal fewer issues for our users, even as we deploy new code to the live environment hundreds of times per week. Our experience is not the result of blind application of best practices, but rather one of continuous learning and improvement to achieve high levels of success across many teams. Teams always approach their work striving to provide the best possible research experience for our users. In other words, it is by focusing on improving the inputs that we are delivering greater outputs.

As we look to the future, we would like to be able to show a correlation between good product team practices and our organizational performance as measured by our KPIs. More information to come as we have new information to share!

Interested in talking with us and others about continuous improvement, quality, and automation? Join our Build Smarter Meet Up.