How the Sprint Retrospective became a pivotal event in Scrum
Scrum then and now, part 12
The ‘Scrum then and now’ series discusses the evolution of Scrum per specific topic. All its articles have this theme and can be read on their own.
Scrum has been around for years. Jeff Sutherland and Ken Schwaber presented it to the world at OOPSLA in 1995. They based it on “The New New Product Development Game“ (1986) by Hirotaka Takeuchi and Ikujiro Nonaka.
Since 1995, many elements of Scrum have not changed. By contrast, some other parts of Scrum have continued evolving. With this series I aim to show you how radically Scrum has changed over the years. Through this I wish to achieve transparency on why certain ideas about Scrum materialised and help raise understanding on the current definition of Scrum.
With this article, it is my goal to give you an overview of the evolution of the Sprint Retrospective throughout the years. The evolution of this event that is typical for many Agile approaches shows the cross-pollination between the approaches very well.
The New New Product Development Game
The paper that started the ball rolling — ‘The New New Product Development Game’ — doesn’t discuss an event where teams inspect themselves to improve the way of working. However, it discusses “self-transcendence”:
“Starting with the guidelines set forth by top-management, they [project team, WJA] begin to establish their own goals and keep on elevating them throughout the development process.” — Hirotaka Takeuchi and Ikujiro Nonaka 1986
Here Takeuchi and Nonaka bring forward that a team should aim to improve itself, which is very much in line with what Scrum says nowadays.
The more I learn about the nuances of this 1986 paper, the more impressed I am with how much of what would become Scrum was already in place.
The first Scrum paper — 1995
With the OOPSLA paper in 1995, Sutherland and Schwaber introduced their own adaptation of Scrum. This paper is fully focused on how Scrum is about delivering products in complex environments. This paper doesn’t discuss any kind of retrospective.
First Scrum Book 2001
The book “Agile Software Development with Scrum” introduces many things, among others the Scrum Master and the Scrum Values. The retrospective however does not exist yet.
Birth of Retrospective in the Agile world
According to the Agile Alliance — Agile Glossary it is Alistair Cockburn who first described the act of reflection after working on increments:
1997: in “Surviving Object-Oriented Projects“, Alistair Cockburn describes several projects (dating as far back as 1993) informally using the practice, but does not give it a label; he summarizes it as “Work in increments, focus after each” — Agile Alliance — Agile Glossary
It also became one of the principles of the Manifesto for Agile Software Development in 2001:
“At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly” — Manifesto for Agile Software Development
This must have triggered the creators of Scrum to add this reflection moment to the Scrum framework. Hence it got introduced soon after Agile came to existence.
Paper ‘What is Scrum’ 2003
The paper “What is Scrum?” (2003) mentions the retrospective in Scrum context for the first time:
“After the Sprint Review and prior to the next Sprint Planning meeting, the ScrumMaster holds a Sprint Retrospective meeting with the Team.” — Ken Schwaber 2003
It is a three-hour time-boxed meeting. The Scrum Master (called ScrumMaster) encourages the team:
“…to revise, within the Scrum process framework and practices, its development process to make it more effective and enjoyable for the next Sprint.” — Ken Schwaber 2003
The paper emphasises both effectiveness and enjoyable. The world enjoyable still is there in the 2017 Scrum Guide. In fact: this whole sentence still is there.
First edition of Scrum Guide — 2010
Ken Schwaber and Jeff Sutherland published the first version of the Scrum Guide in 2010. Since that moment the latest version of the Scrum Guide is Scrum’s single source of truth.
The section about the Sprint Retrospective starts exactly the same as described in Schwaber’s 2003 paper, except for the fact that it isn’t the Scrum Master holding the Sprint Retrospective, but the Scrum Team. This little nuance shows how the role of the Scrum Master has changed towards a coaching/supporting role.
The first Scrum Guide then explains WHY the Sprint Retrospective exists:
“The purpose of the Retrospective is to inspect how the last Sprint went in regards to people, relationships, process and tools. — Sutherland and Schwaber 2010”
It then proceeds with explaining HOW a Sprint Retrospective should be conducted (what went well, what could make things better) and proceeds to explain what could be a topic for the retro (team composition, meeting arrangements, tools, Definition of “Done”, methods of communication, processes used).
At the end the team should have defined actionable items for improvement to be implemented in the next Sprint.
Second edition of Scrum Guide — July 2011
The second Scrum Guide — from July 2011 — starts with the retro section with:
“The Sprint Retrospective is an opportunity for the Scrum Team to inspect itself and create a plan for improvements to be enacted during the next Sprint.” — Sutherland and Schwaber July 2011
Note the subtle difference with the 2010 version of the Scrum Guide: it doesn’t only discuss inspection, but also creating a plan for improvement.
The next notable information is the following:
“During each Sprint Retrospective, the Scrum Team plans ways to increase product quality by adapting the Definition of “Done” as appropriate.” — Schwaber and Sutherland
Adapting the Definition of “Done” is now specifically discussed as a separate item during the retro. This version of the Scrum Guide highlights that this requires extra attention.
This edition also emphasises that:
“Although improvements may be implemented at any time, the Sprint Retrospective provides a dedicated event focused on inspection and adaptation.” — Schwaber and Sutherland
Third edition of Scrum Guide — October 2011
There’s only one change — a subtle one — in the October 2011 edition of the Scrum Guide:
“Although improvements may be implemented at any time, the Sprint Retrospective provides a formal opportunity focused on inspection and adaptation.” — Schwaber and Sutherland
The subtle change is that the retro changed from a ‘dedicated event’ to a ‘formal opportunity’. It is made clear that the Sprint Retrospective shouldn’t be skipped, even when improvements are implemented any time.
Fourth edition of Scrum Guide — 2013
As with other events, the fourth edition clarifies and nuances the length of the Sprint Retrospective. Where previous Scrum Guides say:
“This is a three-hour time-boxed meeting for one-month Sprints. Proportionately less time is allocated for shorter Sprints.” — Schwaber and Sutherland 2011
As from 2013 it is:
“This is a three-hour time-boxed meeting for one-month Sprints. For shorter Sprints, the event is usually shorter.” — Sutherland and Schwaber 2013
It also clarifies the role of the Scrum Master for this event:
“The Scrum Master ensures that the event takes place and that attendants understand its purpose. The Scrum Master teaches all to keep it within the time-box. The Scrum Master participates as a peer team member in the meeting from the accountability over the Scrum process.” — Schwaber and Sutherland.
This actually is interesting stuff! Because:
- Apparently it’s not (necessarily) up to the Scrum Master to facilitate the event!
- The Scrum Master participates as a peer team member;
- The Scrum Master participates from the accountability over the Scrum process.
Sixth edition of Scrum Guide — 2017
The latest version of the Scrum Guide has this interesting line added as responsibility for the Scrum Master:
“The Scrum Master ensures that the meeting is positive and productive.” — Sutherland and Schwaber 2017
Now the Scrum Master isn’t a participant only (anymore).
Also on the topic of the Definition of “Done” a nuance is added:
“…the Scrum Team plans ways to increase product quality by improving work processes or adapting the definition of “Done”, if appropriate and not in conflict with product or organizational standards.” — Schwaber and Sutherland
It makes sense to emphasise that a Definition of “Done” can’t conflict with set standards.
Last but not least, there’s an important item added to the Sprint Backlog section:
“To ensure continuous improvement, it includes at least one high priority process improvement identified in the previous Retrospective meeting.” — Schwaber and Sutherland.
To ensure that improvement items from the retro aren’t forgotten, it is highlighted as an activity during the Sprint Planning.
Conclusion
Hirotaka Takeuchi and Ikujiro Nonaka were the ones that brought forward the initial ideas of Scrum back in 1986. They already touched upon the fact that teams would be able to improve themselves. How this would happen wasn’t discussed though.
When Scrum came to existence early 90’s it was without a retrospective. Cross-pollination with other Agile approaches — and probably especially the Agile Manifesto — led to adding the Sprint Retrospective to Scrum.
Since then — around 2003 — there were some changes:
- The Definition of “Done” got an important role;
- The role of the Scrum Master was clarified;
- The minimum length of the event changed in a subtle way.
The Sprint Retrospective initially wasn’t a Scrum event, but it’s an integral part of Scrum now.
