Agile 101

Yay! Today was my first day as an Associate Development Operations Engineer at Sky. I have a month of training to do, starting off with today’s lessons on Agile.

I’ve written up my raw notes (they’re on GitHub) — if I’ve made any mistakes, let me know!

There are a number of different methodologies that use the Agile philosophy to develop software in a rapid, iterative and adaptable way (amongst them Scrum and XP).

In 2001, a group of software developers published the Agile Manifesto, a series of values:

Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan

That isn’t to say that the developers didn’t see value in the ideas on the right, just that the ideas on the left led to ‘better ways of developing software’.

When compared to traditional waterfall projects, where work is done following a strict spec and a number of stages following one after the other, you start to see the benefits of the Agile methodologies.

While the waterfalll system may provide 100% of the functionality desired, it may be woefully out of date by the time it’s finished (and it’s usually late anyway). Costs may be higher than expected, and quality may suffer because QA is usually the first task to be trimmed or dropped.

Agile methodologies, meanwhile, may not deliver 100% of functionality. However, it’s almost certain the the main functionality will be delivered on time and on budget.

Scrum is probably the most-subscribed to methodology under the Agile umbrella.

It is based on three pillars: transparency, inspection and adaptation.

It is a framework for developing and sustaining complex products (so it doesn’t have to be used while programming — you could, for example, write a collaborative piece of journalism using the Scrum framework).

Its main draws are its simplicity but effectiveness. The underlying philosophy is empiricism — knowledge comes only from experience, and we should base our decisions on what we know (I wrote about this (and Hume) in uni!).

We start certain roles that some in the team must fill.

First off is a Product Owner (PO). They are the link to the business side of the organisation, filtering needed features and boundaries for the product, and who are held ultimately responsible for the success or failure of the product.

The Scrum Master (SM), meanwhile, helps the team self-manage. They aren’t the boss of the team (everyone is, nominally, equal), but they ensure that the Scrum process is followed. They also remove impediments that the team has to make sure the team is working to the best of their ability (for example, they stop the team from being interrupted from external factors).

The developers (note that the term is used not to refer to coders or programmers, but anyone working on the team) manage and organise themselves. They turn the PO’s requirements into a product.

To do this, the product is broken down into ‘sprints’ by the PO, using the principle that the most important features are completed first. This forms the ‘product backlog’ — everything to complete.

A sprint, according to the Scrum Guide, can last for up to one month but usually lasts for a shorter period of time (say, two weeks).

For a sprint of one month, a sprint planning meeting of eight hours is held (though a shorter sprint may only need a shorter sprint planning meeting) divided into ‘What’ and ‘How’ sections.

The ‘What’ section asks what needs to be done during the sprint (it sets the sprint goals); meanwhile, the ‘How’ section asks how the developers will do it.

The developers also break the sprint down into ‘stories’ and further ‘tasks’ which are also ranked according to importance.

Every day during the sprint a ‘daily Scrum’ is held, during which developers present what they did the previous day, what they will do today, and any impediments they expect to face. Because of this idea of a daily Scrum, Scrum aims to constantly iterate and improve, smoothing out any wrinkles in the team and improving quality.

After the sprint is finished, the team present to business representatives. Anything that isn’t complete at the end (this should be the tasks and stories that are least important, because everything that is most important is tackled first) is handed back to the PO, who may move it to another sprint or de-scope (remove) it entirely.

The team then takes time to assess what went wrong with the sprint, and what can be improved. This is similar to traditional waterfall development — the difference being that waterfall retrospectives take place after the product is complete, which doesn’t really help improve the product itself. Scrum retrospectives, however, take place in the middle of the product. They can help the product itself (and the team) improve.

To assist the PO and the team in deciding what features can be de-scoped or otherwise concentrated on, they can employ MOSCOW prioritisation. The name comes from the levels of prioritisation:

MUST, SHOULD, COULD, WON’T

‘MUST’ should take up most of the team’s time (the Scrum Guide recommends 60%). It is made up of the things that the product would not survive without (because the product wouldn’t work, or would be illegal, or not up to some business standard).

‘SHOULD’ is made up of things that the product would suffer without. The user will either need a workaround or the product will somehow be negatively affected without it. The ‘SHOULD’ category should take up 20% of the developers’ time.

The product will survive without items in the ‘COULD’ category. The team doesn’t need to focus on them, and if developers have underestimated the time stories and tasks will take, ‘COULD’ items are the first to be re-relegated to the product backlog.

To estimate the amount of time each story will take, developers will use story points. Modelled, perhaps, on the utilitarian theory of hedons (a unit of measure of happiness), story points try to place a more objective measure of a subjective idea. A story worth 20 story points will take twice the amount as a story worth 10 story points, but only half the time as a story worth 40 story points and so on and so forth.

While this seems fairly woolly (how can you expect to always know how long something will take to craft? This blog post has taken me far longer than I expected…), over time our estimates become better and better (our ‘cone of uncertainty’, as Boehm put it, decreases until we get a precise value) and Scrum allows adaptivity and change even in the estimates we make.

For example, if a team is unsure how long a particular task or story will take in terms of story points and effort, it may choose to ‘spike’ the task — that is, research and develop the idea. In the short term this may not deliver any tangible value or functionality, but in later sprints could prove useful when the PO has to decide a course of action.

Further, we can estimate our pace over sprints. A team’s ‘velocity’ is its story points delivered, per day in a previous sprint, per developer.

story points / previous sprint days / developers in team

Its ‘capacity’ — how many story points it can expect to deliver — is its velocity, multiplied by its developers, multiplied by the days in the sprint.

velocity * #developers * #days in sprint

To track how close the developers are to finishing its sprint goals, it must be transparent in using artefacts. These include the sprint backlog — the list of stories a sprint contains — and the burndown (or burnup) chart, which notes how many story points the developer team have done over the sprint. This way, anyone passing through (and the team itself) can easily see how well a team is doing throughout the sprint. Both the backlog and the burndown chart are housed on the ‘task radiator’, a whiteboard which acts as a kind of hub for the Scrum team.

Scrum methodology demands that each increment of the product be done before it is included in the sprint review. What done actually means is different for each product, but the team can use user stories and an acceptance criteria to create a ‘definition of done’ list (which includes any standards or guidelines that the developers must follow). User stories usually take a form like this:

As a <ROLE>, I want <GOAL> so that <REASON>.

For example: as a user, I want a login/password system so that my account is secure.

When the user story is written, we can make acceptance criteria for that user story — for example, “the acceptance of this criteria means that the user is logged in when the correct password is submitted”.

Though fine on its own, the Scrum methodology can be supplemented by other Agile patterns and processes. One of the most used of these (and, in its own right, one of the most used Agile methodologies) is eXtreme Programming (or XP).

While Scrum focuses on having a potentially shippable product at the end of each sprint, XP demands that the product is shippable at any time. The build is continuously integrated and any breaks in the build are fixed pre-emptively through a system of test-driven development (TDD) — unit tests are written first, checked to fail, then functionality to pass the test is added until the tests pass. This functionality is then optimised and tested again to ensure the build doesn’t break.

XP also favours pair programming (a feature absorbed by some Scrum practitioners). Two developers work at one station — one ‘drives’ and the other checks for bugs, defines unit tests and so on.

More to come, maybe!