It is well established that the application of Scrum to develop software makes teams faster, better, and happier. You can see that for yourself by just searching for keywords like agile and scrum here on Medium. The Scrum values of Courage, Focus, Commitment, Respect, and Openness apply to any team-based approach to problem-solving. Scrum is now being used outside of this traditional space in diverse areas ranging from construction projects, integrating mergers and acquisitions, and many more. It’s even been applied in the classroom to teach students to achieve better learning outcomes via a framework called eduScrum. In this and subsequent articles, I hope to document my journey in applying Scrum to develop Massively Open Online Courses (MOOCs).
MOOCs are online courses designed to scale, have unlimited participation and open to everyone on the web . The course content consists of traditional instruction aids like readings, lectures, problem sets, and quizzes. They are also becoming more interactive and can include forums and other forms of in-person peer to peer interactions. If you have ever taken a course on Skillshare, Coursera or edX, you have interacted with a MOOC.
The Covid-19 pandemic drastically exploded the interest in online learning and MOOCs in 2020. For example, Coursera, edX, and FutureLearn saw as many new registrants in April 2020 alone as the whole of 2019. Coursera saw a 100% jump in registered users in 2020 . With these exponential growth numbers, the MOOC industry will rake in big bucks in the coming years. The learners’ demand has also seen a drastic shift towards highly engaging, just in time, on-demand content that offers maximum interactability. But one question that remains is how do content creators develop courses that meet this need most cost-effectively and as efficiently as possible.
Content development for courses is a costly affair, with an average of 500+ hours to develop 1 hour of thoroughly engaging content . In addition to this, content developers also have to worry about making sure what they create is relevant and needed by users. Exacerbating these problems are the constraints of time, budget, and resources. Thus, there is a real need to do things better, faster and see if we can be happier during this process.
Is Scrum the Answer?
These are precisely the questions we were struggling with when we started our journey to create our engineering simulation-based courses platform. When we started, we followed a version of the ADDIE methodology  (Analysis, Design, Develop, Implement, Evaluate). The process begins with Analysis, where the course objectives, learning objectives, and target audience are defined and codified. This is followed by a Design phase, where the course content is planned out in detail to achieve the objectives set out during Analysis. The Development phase deals with the actual work needed to create the content, be it handouts, exercises, and videos. The Implementation phase deals with publishing the courses to show to stakeholders and maybe beta test with select users. The final step is the Evaluate phase, where a formal assessment is done to see if the course we created successfully solved a customer need.
ADDIE course development cycle very much resembles a waterfall development methodology. The teams started developing the courses with some starting content from the Subject Matter Experts and completed the courses in a few months. By the time the SMEs or the customers saw the final product, it has already become irrelevant. This was leading to at least a lot of rework and technical debt that kept ballooning. A change was needed. Based on my experience as a Scrum Master and Product Owner on software development projects, I realized that we needed a serious attempt at implementing Scrum. All said and done, we implemented Scrum in all our teams, leading to at least a 2x increase in team velocity, a 40% increase in happiness, and we got better by focusing on learning as a team. Let’s see how we did it.
Riding the Wave Between Order and Chaos: Getting Started
If you are wondering how to get started, a useful resource is the Scrum Guide, and the book aptly titled A Scrum book: the spirit of the game. Nine basic patterns detailed below got us started, and it can get you started as well.
Teams and stakeholders tend to see engineers as a pool of fungible resources and not as a team that can share and live a vision and creates products. We made sure that this does not creep into our thinking. This led us to think of the teams as a unit of people working together to achieve our vision. The teams develop and realize that they start sharing the same mental models, and we shape the product more effectively and cohesively. The other apparent consequence we saw was measuring if we were getting better every sprint in terms of our processes and team metrics like velocity and happiness. The team’s definition includes the development team, the scrum master, and the product owner.
Each course starts with the product owners defining the key learning objectives in 3 to 5 bullet points during the grooming session. We then break apart the course learning objectives into lessons and sections. We then tease out the tasks required to make these lessons and sections in user stories, thus providing us the initial product backlog. If the unknowns and the risks involved are known, the team then estimates these user stories using planning poker. This gives us an idea of the overall work and the points to complete the course. We add a drift estimate to these total points to account for any changes in scope or any new information. We aimed to bring down this drift with each course we created and keep it below 30% based on experience. If we know the team’s average velocity for comparable courses, we can develop a release plan. Next, this release plan is presented to stakeholders (the budget holders) and see if we can proceed. If there are any budget or time constraints, the scope is adjusted. We use the above process to create the product backlog. We then use the backlog grooming sessions every week to define which items are ready to be pulled into an upcoming sprint. Our ready gate consists of three criteria: is it actionable immediately, is it independent, has the highest value to the customer.
The team’s velocity during the previous sprint is an excellent guide to know where to start. We plan our sprint to pull in the same estimated points as was completed in the last sprint and manage stakeholder expectations to reduce the number of sprints where we fail to meet the sprint goals.
When we started our Scrum journey, the teams preferred to work as individuals on their work-pieces, leading to items in WIP states that matched the number of team members. This led to the teams being less efficient and the perception of not being productive. Swarming is a radically new way of thinking that went against many engineers’ working styles and personal preferences. We initially experimented with pair programming, but the two-person teams could only be useful to a degree. We then shifted into working on one high-value backlog item at a time. The team members focused on different aspects of the item or user story, including content development, peer review, and any testing. We saw a marked increase in team velocity, but more importantly, the team dynamics improved. Overall happiness and engagement metrics increased significantly.
We cannot avoid Interruptions; they are a part of life. They come in many forms and from many sources — stakeholders, sales, marketing, defects. Not planning for or assuming that there will not be any leads to disaster and many heartaches. We allocate points as a percentage of our sprint velocity for unplanned work and interruptions and pull user stories for the remaining points. The product owner triages all requests outside of the sprint goal. IF the points added from the unplanned work exceeds the preset buffer, we abandon the sprint and go into sprint planning. The product owner will inform the stakeholders about the slippage. We do not need the buffer most times, which allowed us to increase the sprint velocity.
Product quality is everyone’s responsibility on the team. It is not just on the engineer who is leading or working on that item. Defects and tech debt builds up if it’s not fixed the moment it becomes apparent. It gets more challenging to improve as time passes. We decided to apply this pattern to ensure that the content we create is clean and ready to be worked on the next day by any team member. With our teams being geographically dispersed and working remote, this becomes a critical requirement. At the end of the day, team members will clean up and tie up any loose ends and update them on the work completed. Other team members who don’t share the same timezone can then “take the ball further down the field” and achieve the sprint goal faster and better.
Scrumming the Scrum
During the sprint retrospectives, the teams list all the impediments to becoming a better, faster, and happier team. We vote and pick the most important impediment that will give us the most bang for the buck. This impediment is then captured as a user story in the next sprint, with acceptance criteria and points allocated to them. We work to get this story implemented and measure if the effort made a difference in the next retrospective. This lets us create content faster by at least 40% in all teams and happier by 30%.
We place a premium on the happiness of the team — happiness is a leading indicator of success. We found out that this pattern, more than others led to team autonomy and engagement, especially when transitioning from a waterfall style approach to drive all the teams’ activity. We measure team and individual happiness on a subjective five-point scale and how we can get this score up. We use Management 3.0 practices like celebration grids, kudos, and Niko-Niko calendars to become happier.
Teams that Finish First Accelerate Faster.
We use yesterday’s weather to guide us to plan our sprints. A key observation is not to be too ambitious and pull in too much work. When some of our teams did this, counter-intuitively, those teams became slower, and they decelerated. The teams that could pull in just enough work could take a step back, think about what will make them faster, better and implement process improvements. They can pull in more user stories and increase their yesterday’s weather, thus accelerating in the long run.
Quick Start Guide
If you are in a pinch and want to start as soon as possible, just start with the first two: a stable team and a ready backlog.
Scrum: Repeatable and Measurable Outcomes
So how did we do? In the last year, we have created around 50+ courses on our free course platform. We have had great feedback and reception from our customers, with over 30,000 unique users in the six months since launch. We achieved this in less than half of the industry average of 500 hrs per one hour of content, and we are getting better every course we create. I hope to explore more about the practices that worked for us in upcoming posts.
These are my personal experiences and views and in no way represent the opinions or stances of my employer, Ansys.
- Scrum Guide
- A Scrum Book: The Spirit of the Game