Measuring Lostness

Everyone wants a useful metric for measuring how effective your product or service is (or at least I do). The lostness metric will do this. It can tell you how lost users are when they use your product.

The metric itself is pretty simple as it ranges from zero to one. A high score of lostness means users are having trouble finding what they want. A low score means the opposite, that they are able to find what they want with relative ease. Anything under 0.4 indicates that there are not lost and anything over indicates that they’re lost.

The only way you can measure it is when you know what your users are wanting to do. It’s unlikely analytical data or logs would help you measure it because you don’t know what the users are trying to do.

So the best way to determine it is through usability testing, where you get users to complete a task. This way you know exactly what they’re trying to do and what is the best (quickest) path and how lost they are. It’s all based on the formula below.

The Formula

L = Lostness
N = The number of unique screens visited during the task.
S = The total number of screens visited during the task.
R = The minimum number of screens that must be visited to complete a task.

The best way to understand the formula is through an example. Let’s pretend that we’re observing a user completing a task in a user test.

Warning: I’m gonna do a shameless plug here; the app I’m gonna use for the test is one of my own. It’s called Craft Beers, you can use it to discover new craft beers and you can download it from the Google Play Store.

The Example
We ask the user to pick a beer that interests them and view the website of the company that brews the beer.

The task requires the user to visit 3 screens to view the brewer’s website:
1. Search Beers Screen
2. Beer Screen
3. Brewers Website Screen

This means that R is equal to 3 in our lostness formula.

However, our user visited 7 screens to view the brewer’s website:
1. Search Beers screen.
2. View Beer Screen.
4. Back to the View Beer screen.
5. Back to the Search Beers screen.
6. Back again to the View Beer screen.
7. Brewers Website screen.

In our lostness formula, this would result in S being equal to 7. Remember S is the total number of screens visited in the task.

Also, the user visited the following unique screens in the task:
1. Search Beers screen.
2. View Beer screen.
4. Brewers Website screen.

So in our lostness formula, this would result in N being equal to 4. Since N is the number of different screens visited during the task.

This allows us to calculate the lostness of this user during the task so prepare yourself for some math.

By completing the formula we get a lostness value of 0.50. This indicates that the user struggled to access the brewer’s website.

Now that we know that users are struggling we could either take some time to see how we could improve the flow/designs so that we can reduce lostness for our users or do nothing at all. 🤔 I highly recommend you don’t do the latter.

Through this metric, we’re now able to measure how difficult it is to use a product or service. This info should help us to improve the product in the future.

I wouldn’t have been able to write this post if not for the book “Validating Product Ideas: Through Lean User Research” by Tomer Sharon, it’s a fantastic book with tons of methods on how you can validate product ideas and I would highly recommend it.

Did you like this article? If so, tap the ❤️ below and I might write some more.