Scrum: Learned lessons

17 min readMar 21, 2018

Many technology companies presume to be at the forefront in software development methodologies. I’ve heard many people say they implement Scrum because every day they meet 10 minutes to present their progress. Other people say that it only works when you separate the UI design, testing and deployment from the construction. Others say that Scrum doesn’t work because it means losing time with so many meetings. But what I’ve mostly heard is that almost everyone have adapted Scrum to their “needs”.

Having said that, I’m gonna share what I’ve learned from Scrum after a certification and during a couple of years working with it.

It is not a methodology.

The first thing to understand is that Scrum is not a methodology but a framework. The difference is that a methodology tells you exactly what and how you should do it, whereas a framework is a set of “tools” available to do it.

Although I’m not defining what Scrum is or what those tools are, I’m just gonna say that no matter how particular the business, the project or the team are, every single of these tools will always be the same and the key is how we can use them in every particular case.

It’s not acceptable to say that Scrum was adapted to the needs of the project, since the problem didn’t come from the tools, but probably they never have used them correctly. In this case, they are not implementing Scrum.

Not all activities are tasks of the Backlog.

Some people add activities such as meetings, courses or any activity that consumes hours from the capacity of the Development Team, regardless of whether or not it is directly related to the development of a user story, or even worse, with the Sprint Goal.

The workshops like software design or user flows analysis with the team that are necessary for the product development are the only kind of meetings allowed to register in the Product Backlog. Other meetings like user interviews, meetings with stakeholder or any definition meetings shouldn’t be part of the Backlog items, but deducted from the capacity of the members of the Development Team in case they are involved.

In other words, the meetings needed to answer “what should be done?” must be done outside the time of the Sprint and the capacity of the team, and those that are needed to answer “how should it be done?” must be registered within the Product Backlog.

QA is not an area, but a process.

There are two perspectives to QA concept: In one hand the QA inside a software factory and on the other hand QA as a process in a product development team. Some people say that QA is an area or discipline in which testers are responsible for testing applications and ensuring that the product has quality. In other cases, the testing team consider that they have sufficient criteria to evaluate and report suggestions about what they consider a “lack of quality”.

In a Product Development context, QA must be conceived as a constant and iterative process from the definition stage to the release and deployment of any User Story through good practices that help to ensure quality in many ways, and not only bug-free software.

These practices range from the definition and drafting of user stories, validation of hypotheses, implementation of different types of tests (both manual and automated) and the software development techniques such as TDD, Pull Requests, etc. All of these activities will reduce the number of bugs, misinterpretations and rework and allow the generation of high quality deliverables.

You must have a true Product Owner.

As well as any role within Scrum, the Product Owner (PO) is extremely important for the success of the implementation of Scrum, since it must be the only role that defines, prioritizes and validates user stories. However, there is a poor conception of this role.

For example, many managers of a business unit or head of product as POs only request what they think is good for the project or the product, but never spend time to validate, measure, plan or give a complete and clear definition of their request. And the worst of all, they are never available to clarify doubts or make day-to-day decisions.

Another example is the PO who only follows the instructions of the Stakeholders, and doesn’t have any authority or autonomy in making decisions that allow adding real value to the project.

In both cases, they hurt both the Scrum Team and the product, generating rework or simply provoking work that doesn’t add actual value to the business or the user, and consequently, causes demotivation of the Scrum Team.

Communication is the key to the success.

In any project, company or methodology, communication is essential to be successful.

The cause of misunderstandings, rework, lack of definition and frustration inside the team or the company is the result of poor communication. That is why Scrum Meetings and the physical location where the members of the Scrum Team are sat are vital.

Another important aspect is the communication channel. The PO shouldn’t be giving any indication that affects the Definition of Done (DoD) directly to any member of the Development Team without at least the Scrum Master (SM) being aware. However, it doesn’t mean that the SM always has to be present when the PO and a member of the Development Team need to interact; I mean, the communication of a change of priority, a change of DoD or a change in the scope of the Sprint, the PO must warn both the team and the SM.

The documentation is important.

Although it seems contrary to what the Agile Manifesto says: “Working software over comprehensive documentation”, the documentation is extremely important for the success of Scrum.

Although some people understand this sentence like “there should be no documentation”, the documentation allows to have clear at any time the definition of what and how needs to be done, and it reduces the gap between what was requested and what was actually delivered.

The difference between this documentation and the traditional software development documentation is that it is brief and less bureaucratic. The necessary documentation is the guide to the context acquired during the verbal communication between the team members, PO and SM.

That is why it’s extremely important to have a description of the user’s story or PBI (Product Backlog Item), but it’s much more important to have the DoD complete and well written, even though is extremely difficult to reach a point of actual completeness, since the PO almost never has clear exactly what he needs or wants. This is why the team must be in constant communication with the PO since the team must have the complete context and implications of the user story.

The Development Team is “a well-oiled machine”.

It’s essential to have an actual team and not just a group of people with different skills. It is necessary to know each other and have a good interaction and communication.

In fact, the speed of the team depends directly on how well they work as such. If the team has a new member, the important thing is that he quickly coupled with the team’s way of working to increase the capacity to perform the most tasks necessary to meet the Sprint’s objective.

This is contrary to the idea that the members of the teams are “interchangeable” or that a person can be member of several teams, since a high concentration is needed to perform tasks.

Moreover, adding more members to the team doesn’t mean that the speed increases, since the greater the number of people, the greater number of problems of communication, specialization and organization we get.

Having said that, there is also the possibility of eventually exchanging a member of the team for a period of time. This becomes a good practice that allows pollination of knowledge and techniques developed across teams.

Effective hours.

There’s a difference between working hours and effective hours. While the working hours refers to the official shift at work, the effective hours are related to the level of concentration needed to complete a task. Therefore, we can’t think that if we have a task that is estimated at 8 hours, it can end in 1 day of work, since it would be leading to overtime and demotivation; and worse, not meet the Sprint Objective.

Additionally, the calculation of the hours of the tasks is very important, since this depends on the commitment made by the team with the PO. Speaking exclusively of software development, it is necessary to perform within the effective hours the implementation, unit tests, software design, documentation and any adjustment on the go. If this activities are not part of the good practices of the team, it results in technical debt, lower quality of the deliverable and serious problems of maintenance and scalability (really serious), since this sacrifices agility of which I will speak in the following points.

Story points are not measured in hours

The Sprint Planning is divided into 2 stages focused on answering 2 questions: “what” and “how”. In the first stage what has to be done is established, or what the user stories are that the team commits to perform. In the second stage, the development team establishes the strategy for accomplishing the plan by defining the necessary tasks.

The story points should estimate the effort, the risk and the uncertainty involved in accomplishing a User Story, and it helps the team to forecast how many user stories could commit in the Sprint Planning.

The hours must be applied to the tasks based on what the team calculate as necessary to accomplish the User Story. This helps the team to self-organize based on the capacity of each member.

Listen up: At the end of the Sprint, the only thing that matters is how many Story Points are done, since it doesn’t matter who did it and how it was done; but that the commitment made in the planning has been fulfilled.

For this reason, the record of how many hours a member worked is not a productivity metric.

Definition of roles and profiles.

It’s extremely important to count with the right person for the right position. For example, many SM as former Project Managers have difficulties to understand (or accept) the differences between both roles.

The biggest challenge of the Software Developers is the change of mindset working with different roles inside the Development Team, since they are not used to working side by side with designers, testers and business people. In many cases they tend to think like “It was not in the requirement”. However, this mindset is just not compatible with the philosophy of teamwork within Scrum.

That’s the reason why I insist: communication is extremely important to improve accountability.

Figure 1. 3 Main players on Scrum, and what they do .

The more dependencies, the more impediments.

Scrum only works with autonomous and self-sufficient work cells . And this is very important to emphasize, since many consider that Scrum is only a Backlog, a Sprint and Daily Scrums.

The Scrum teams must be interdisciplinary and self-sufficient to accomplish each and every single task necessary to perform a User Story. From the database to the UI. Any dependence on another team, business area or provider increases the risk of not finishing the Sprint committed to the PO.

It’s naive to think that other teams are going to modify their priorities and planning in order to be able to match our team’s so that Sprint compliance would be at risk, since it is very likely to become an impediment.

Now, if this happens in each Sprint, it is better to include the other team or someone who is able to solve these tasks within our Development Team in the same Sprint.

Therefore, the recommendation is never to include tasks from other teams or areas to our Sprint Backlog.

The Scrum Master is not the Product Owner’s assistant.

The SM is a fundamental role within Scrum, since its main participation is being a facilitator of the Scrum framework, to resolve dependencies and tackle impediments, facilitating and improving communication, meetings, good practices, and even ensure the good writing and definition of user stories.

The big mistake that SMs make is to become the PO assistant. In many cases the PO only “explains” to the SM what is it that what he wants to be done, and the SM becomes responsible for the analysis, communication, prioritization, etc.

But the SM’s responsibility is to work together with the PO to facilitate this analysis, maintain the Backlog and even facilitate the correct prioritization thereof, encouraging communication and also sensitizing the PO of what happens both within the Development Team and in other areas.

Agility is not speed.

Another big mistake is confusing agility with speed. But agility is the ability to respond to change, while speed is the amount of tasks that can be built.

Both are directly related to several factors:

Individual skills within the team.
Software Architecture
Good practices
Framework
Discipline
Dependencies

Any of the above directly affects agility.

Team velocity is not a productivity metric.

Velocity is a metric that allows the Development Team to forecast in the Sprint Planning based on the history. It is based on the number of Story Points done in the previous Sprints. It is a metric that is only useful to the Development Team.

Neither is it a metric to compare between Development Teams, since the speed depends on the level of maturity of the product and the team.

Technical debt in Scrum.

“Higher technical excellence means lower technical debt. Lower technical debt means faster throughput”.

Technical debt is the “cancer” of software which consists on unfinished tasks, patches, deprecated code or malfunction of the code and infrastructure that were not attended or detected on time and tend to accumulate over time, becoming a negative impact on the health of the software. We could say it is the result of poor execution or design decisions but every software eventually has technical debt, the difference is how much importance we give to it.

Although it is a fact that it is the responsibility of the Development Team, it is important to consider several aspects:

The team should be technically good enough to avoid the technical debt as much as possible through good practices such as design patterns, refactoring, code reviews, etc.
The software must evolve and technical debt cannot be avoided and that sooner or later must be “paid up”.
The earlier the technical debt is resolved, the shorter the resolution time and the possible future consequences, increasing the value of the product.
Although the Product Backlog must contain only User Stories that increase of value of the product, it’s very healthy to identify, describe and keep track of the technical debt to be as transparent as possible, being the Product Backlog the right place to keep this track.
The Development Team must be mature enough to identify and accept the technical debt and be able to convey to the PO the importance of its mitigation as soon as possible to avoid future rework.

Fragmentation and incremental development.

It is the responsibility of the PO, SM and the Development Team to understand and work with the strategy of an iterative and incremental development, which means that it’s necessary to design, develop and deliver User Stories always thinking in the minimum change but enough to add tangible value to the user (and the product).

A good User Story complies with the “INVEST” model:

Independent. Reduced dependence = easier to plan.
Negotiable. Details added through collaboration.
Valuable. Provide value for the customer.
Estimable. Too big or too lazy = not estimable.
Small. It can be done in less than a week by the team.
Testable. Good acceptance criteria.

For practical purposes, each User Story or PBI must have the following characteristics:

It must add perceptible value for the user/product. Heads up: we are talking about User Stories or Bugs but not tasks related to technical debt or infrastructure changes that must be treated in a different way.
“Acceptance criteria needs to define some measurable and testable definition of improvement”. If it can’t be shown at the Review meeting, it’s not a correctly defined User Story.
It shouldn’t be able to fragment into other User Stories. In other words, if it can be fragmented into other functionalities, use cases or flows, it is not a sufficiently fragmented User Story.
The interdependency between User Stories must be avoid as possible, so the software design skills must be good enough to be able to connect all the pieces without dependency of another User Story within the same Sprint.

Scrum Development Team

The textbook of Scrum says one member of the Development Team should be capable to accomplish a hole User Story, or at least de Development Team should be self-sufficient and cross-functional, having the necessary skill to develop the product . That allows the team to be self-sufficient to perform any User Story at any time and without depending on anyone else.

Today there are several roles with particular skill sets that are needed to design and develop a successful product in comparison when Scrum came out in 1993.

I remember asking Mike Beedle if Scrum should only be limited to software developers (programmers), and he looked at me with fire in his eyes he asked me “who told you such a thing?!”.

Many companies “implement Scrum” as a workflow reserved only for software developers. And, in the best case scenario, they only get the UI design and a requirement document as input, then they only implement and deliver to the next area. The testing and deployment processes are out of the scope of the Sprint. That’s what we call “mini waterfall”, because a dependency between teams or functional silos still exists and the User Story ends up being just a traditional requirement document and the Sprint just a time box.

The ideal implementation of Scrum is to have each and every single activity related to the development of the product and not only the software development such as tasks of UX, UI, testing, infrastructure and deployment. This is only possible with the integration of different disciplines within the Development Team.

Just like the activities, not all the tasks should be included in the Backlog, since there are tasks of any discipline that don’t necessarily satisfy a particular User Story, but cross several Sprints or simply are endless but useful to establish strategies, validations and standards. Below are some examples of tasks that should and should not be included in the Product Backlog:

Outside: Interviews, wireframes, user journey.
Inside: User Story flow design, navigation, communication to the user.

Outside: Design guides, concepts designs.
Inside High level or final interface design.

Software engineering

Outside: PoCs, architecture designs.
Inside: User Story implementation and bug resolution.

Tester

Out: Test planning, any kind of test unrelated to any User Story.
Inside: Tests analysis and design related to user Story. Results reporting.

DevOps

Outside: Scheduled maintenance, resolution of incidents with suppliers.
Inside: Infrastructure provisioning needed for a particular User Story.

Physical board

Although tools such as Jira and Visual Studio Team Services allow having a dashboard-like view of the Backlog, it is essential to have a physical dashboard, since it is needed at the Daily Scrum meeting. Also, it is recommended to have the board in sight to encourage and facilitate interaction, not only of the teammates, but also any stakeholder allowing having at hand the status of the Sprint just by looking at it.

Clearly the maintenance is double, but the advantages of the physical dashboard allow all at the same time to be watching and discussing about it.

Scrum ceremonies

All the tools (meetings and artifacts) that Scrum offers are essential for the success of the framework, and mainly of the project. To achieve this, the meetings MUST be performed during the Sprints on time and the way they are designed, that’s the reason why they are called “ceremonies”.

Next, the Scrum meetings and their recommendations:

Sprint Planning: It’s not the forum to define and describe User Stories. All of User Stories must have at least a title, description and DoD. The meeting should be divided into 2 stages: Stage 1: With all the team (PO, Development Team and SM), the PO prioritizes and clarifies doubts, while the team calculates the effort based on Scrum Poker technique and establishes its commitment to the Sprint. Stage 2: The PO could be absent so the Development Team can define tasks, self-organize and finish the planning. It is recommended to build the physical board in that moment.
Daily Scrum: Everyone standing at the physical dashboard every day at the same time. This meeting is not a progress report to the PO or SM , the main goal is to inform the rest of the team what everybody is doing and warn about any impediment outside the team to de SM, as well as reorganize if necessary to achieve the Sprint Goal.
Sprint Review: Definitely the purpose of this meeting is to present what was developed at the end of the Sprint. However, it’s desirable to be in touch with the PO in order to validate the progress as soon as possible and reduce the risk that there has been a misinterpretation of the User Story. The ideal is to reach the Sprint Review with the largest number of User Stories previously validated by the PO.
Sprint Retrospective: MUST be brief and concise. It is important to take note of what is said to keep a record and try not to fall into the same mistakes and improve the team’s work.
Backlog Refinement: It must be based on the result of the Sprint trying to define the course of the project in a better way. However, this Refinement shouldn’t be just a meeting but a constant process in every single Sprint to keep the strategy and the Product Backlog up to date.

Ideal requirements

There are requirements that I consider the ideal in order to exploit Scrum as much as possible, adding as much value as possible to the business.

User centered design: 100% focus on the user and not only on what the PO or the Stakeholders think. Otherwise, it generates rework and demotivation on the whole team when building features with no value to the user.
DevOps: The Development Team must be autonomous and able to provide, operate and deploy any infrastructure related to the product, since any eventuality related to the project the team must be able to attend it and solve it, improving the accountability of the team and every single member.
Automation: Everything must be automated as much as possible, from tests, UI design sharing, builds, releases, monitoring, maintenance, etc. This improves everything related to software development.
Testing: The participation of the testers contribute with great value to quality assurance. However, they must be included in the Development Team and have development skills, since Scrum’s tendency is to automate processes and continuous release. In addition, they must have all the context of the User Story to perform their job.
Sprint Goal: The Development Team must be clear about the Sprint Goal and what it’s committing to. This way, the accountability of each member is encouraged.
DoD: If the Definition of Done (or acceptance criteria) is not clear or incomplete, there is a risk of working more or less and not finishing the Sprint, incurring large “adjustments” or re-definitions of the User Stories.
Cross-functional teams: As in the previous points, the team must be able to implement, test and deploy any User Story. It’s obvious that a UI designer can not configure a server, or a tester can not make modifications to a design. However, it’s sought that at least in the technical point of view, it’s necessary to have the minimum knowledge to identify and diagnose a problem or be able to support another member of the team.
Unit testing: The unit tests is the minimum requirement that allows to have great control of the changes made to the code and to identify immediately any negative impact to the other pieces of code, speeding up the maintenance and maximizing the product quality. This kind of tasks must be considered in the calculation of the Story Point and in the time capacity of the Development Team.
CI / CD: Integration and continuous delivery are the key to the software development that allows the agility to Scrum, allowing to implement, integrate and cause the least number of conflicts between software developers.

Conclusion

Scrum is so flexible that allows us to put our signature on it as long as we respect the basics concepts and use correctly the tools that it offers. The key is to have self-organized teams with freedom and autonomy to perform what it’s needed to accomplish what is good to the product.