Return on investment in ux planning
I have recently completed an attachment in the BBCs Radio and Music team as a product owner.
The BBC invests heavily in training, and I have greatly enjoyed learning and practicing kanban, scrum (agile), and release planning techniques.
One of these techniques was release planning using return on investment (ROI).
Assigning business value
In the ROI technique, the set of possible tasks are put on tickets — one ticket for each task. Each ticket is first ordered by business value. The most valuable at the top, the least valuable at the bottom. The value of each ticket of work is recorded on the ticket.
For example, here are some (made up) tickets, for a site that already exists, but is being rebuilt:
- re-Branding of the whole site to give it a consistent feel.
- New horizontal navigation bar to give users access to significant parts of the site.
- Revamped search with autocomplete and search results page.
- New homepage redesign.
- Content management system redesign
Each task ticket is then given a number for its business value, the highest number at the top of the list of tickets, the lowest number is at the bottom.
In conversation with our stakeholders, we might decide the following business values. These values are made up during that conversation — it is not the value itself but the relative value when compared to other work, that is important.
- New homepage redesign. (value=250)
- Revamped search with autocomplete and search results page. (value=150)
- New horizontal navigation bar to give users access to significant parts of the site. (value=100)
- re-Branding of the whole site to give it a consistent feel. (value=25)
- Content management system redesign (value=10)
ROI Release planning lets you answer:
Which bits of work will have high value to the business and be easy to complete?
We can see that the stakeholders really want a new homepage, and also really value search. They clearly see some value in a navigation bar across all pages of the site, and a limited value to rebranding the whole site. The content management system is considered a necessary cost, but if they could get away without having one, that would be fine by them.
Technical complexity using poker points
Now the tickets are re-eveluated by an experienced technical member of staff (usually a tech lead).
Each ticket is considered for its complexity — we can use planning poker points or any other approach. So, the complexity of a ticket now has a numerical value.
The numerical complexity is now recorded on each ticket.
In the case of our made up list, the list is now:
- New homepage redesign. (value=250, complexity = 21)
- Revamped search with autocomplete and search results page. (value=150, complexity = 21)
- New horizontal navigation bar to give users access to significant parts of the site. (value=100, complexity=5)
- re-Branding of the whole site to give it a consistent feel. (value=25, complexity=13)
- Content management system redesign (value=10, complexity=13)
Points to note: Don’t worry about the numbers too much. The value of this process is that it results in an ordered list of tickets which can be discussed, not in the exact number assigned to each ticket.
Return on investment is relative
Return on investment (ROI) is now: Business value/Complexity.
The ROI can now be recorded on each ticket.
The backlog of resulting tickets can now be ordered by ROI.
In our case this gives us:
- New horizontal navigation bar to give users access to significant parts of the site. (value=100, complexity=5, ROI=20)
- New homepage redesign. (value=250, complexity = 21, ROI = 11.9 )
- Revamped search with autocomplete and search results page. (value=150, complexity = 21, ROI=7.1)
- re-Branding of the whole site to give it a consistent feel. (value=25, complexity=13, ROI=1.9)
- Content management system redesign (value=10, complexity=13, ROI=0.76)
For large groups of tickets, its worth subdividing the group of tickets into smaller ‘feature’ groups. This will help answer “what should we do?”
The tasks in this roi list, with the highest roi number represent your best guess at what is valuable to the business and easy to do?”
Reusing the ROI technique in UX planning
The really exciting thing about this approach for me, is that it can be reused in a variety of situations. Particularly, it can be used in planning UX work. For example, we can replace ‘business value’ numbers with ‘user value’ (how much will the user value this feature?). Likewise our estimate of complexity can be based on just the time required to complete the ux work required, or ux+tech+test staff estimates.
So, as a creative director or head of ux, you can now look at a slate of work, and this ROI technique lets you look at the slate in a new way — which bits of work will have high value to users, and be easy to complete?
Having got a list of tickets ordered by roi, you can now plan a release, by considering a group of tickets you are going to complete for that release.
Using your roi list order, you can see which tickets represent good value. Some tickets of work will be high value, high cost (therefore limited ROI), and still be worth doing. In our made up slate, the item:
- Content management system (CMS) redesign (value=10, complexity=13, ROI=0.76)
has very limited business value, since the stakeholders view it as having no benefit to the business (unlikely, but lets suppose). However, if the new homepage cannot be built without a new CMS, the CMS would have to built as well, even though it doesn’t represent a good ROI.
Planning a ux slate using roi
This ROI prioritisation technique can be reused with different axis (user value replaces business value) and it can be applied without having perfect estimates. If we replace business value with user value, we can use qualitative (eg a card sort with sample users) or quantitative (eg traffic data) to find user value.
The list of work ordered by ROI is showing you which tasks look like good valuable work, and which don’t.
For me, it feels like a good technique with which to have very constructive stakeholder or management conversation.
Originally published at www.chris-thorne.co.uk on July 22, 2013.