Do our designs improve our customers lives?

We design to create the best customer experience and help people achieve their goals. But… do we know if we made an impact on our users? Did we help achieve those goals and improve our customers lives? How would you know? In this article I’ll tell you how we did it.

We designers love to make beautiful things that add value to the lives of people. We know how to make things easy, usable, and we always aim to create a good experience. We usually start with determining the goals of our users, and after that we hope to make an impact by helping them to achieve their goals. Most designers do user interviews and usability studies. But how do you check if your solution reaches the users and if it helps them achieving their goals as well as the goals of your business? How can you be sure you made an impact on your users? I often notice that when the business is happy, the design looks slick and the usability test doesn’t show any mayor problems, the design is considered good.

However, the real impact on the user is important and should be checked. If our designs help users achieve their goals in a better way this will show in their behavioural traits. Examples of measurable behavioural traits are a higher task completion, less time needed per task. Examples of traits focused on business goals are, for instance increased sales, or less products churned. It can be harder to measure these traits for features with a more long-term goal, such as increasing financial health or engagement. To show you how we did it, let me share a story about ING’s forecasting feature: Look Ahead.

forecasting feature

Look Ahead

The ‘Look Ahead’ feature is a section in the app that you can reach from the transaction overview of your Current account. In this section we predict your income and spending and tell you what your balance will look like for the next 35 days. With ‘Look Ahead’ we aim to help customers get a grip on their finances and climb the financial ladder (growing their assets like savings or investments). As a user you can easily check if you will have money left at the end of the month or if you should cut back on spending.

For some users ‘Look Ahead’ might be the first step to creating a buffer for unexpected costs. Others use it to manage their spending, so they have more left at the end of the month. First and foremost, we hope it helps all users to improve their financial situation. If you’re wondering what’s in it for ING, we hope this feature leads to more engagement and appreciation from our customer. With a bit of luck, it could lead to more sales in the long run.

‘Look Ahead’ has been around since 2016 and has 1.2 million monthly active users, compared to roughly 5.3 million monthly active app users. In other words: we think there is room to increase the number of users. This number of course, but we never really evaluated if the feature helps achieving the user and business goals we mentioned before. Our customer feedback module doesn’t tell us the full story.

What we do know from the feedback we get, is that our customers love Look Ahead. However, they would like to add some extra predictions to make it more precise.

Then something shocking happened; we stumbled on research that showed how making people aware of budgets or visualising future scenarios could lead to overspending and decreasing financial health…

One of the findings of this research is that the more precisely a future scenario was presented, the more negative the impact on a user’s financial prospects. Oof, some discovery, after just having spent 5 years to make our predictions more precise…

Of course, our biggest fear now was that we didn’t help our customers achieving their goals, but were messing up their financial health instead. Our Look Ahead team quickly dug up some data rock stars to compare the financial behaviour of ‘Look Ahead’ users and non-users. Our nightmare came true… The comparison showed that Look Ahead users saved less money and had less assets on average compared to customers who didn’t use ‘Look Ahead’. We thought that 1.2 million people every month were off worse every month, because of our feature; at least, that was our assumption at that moment. We only did a quick comparison and didn’t understand this outcome. How did we end up with these results? It could be a lot of things. Needless to say our next mission was to find out why we got these miserable results.

At this point we could no longer focus on increasing the number of users for this feature. First, we had to solve the mystery. To do so, we defined the following questions:

• Why do people use the ‘Look Ahead’ feature and what benefits do they see?
• Does Look Ahead really help customers achieve their goal of a better financial health?
• Does using Look Ahead lead to more engagement with the app?
• What could we improve to achieve a better financial health for our customers?

First analyses

We started with a quick data analysis on how Look Ahead was used and by who.

Most people use ‘Look Ahead’ in the app. The majority uses it once or twice a month and the usage peaks at the end of the month. Not only is the feature opened more often, the average session times (recorded at the end of the month) turn out to be longer. The usage peak manifests itself at the end of each month and continues in the first days of the following month. Furthermore, we see that people without or hardly any savings are using ‘Look Ahead’ more often.

To answer why customers use or don’t use Look Ahead, we scheduled 17 customer interviews. Here are the main findings at a glance:

• Some people fully relied on forecasting to manage their finances. However, most people indicated that the feature does not add extensive new insights in addition to what they already know. They use forecasting to verify their recurring payments or check when the recurring costs change. It gives all users a sense of control.
• The feature is often used to check the balance to see if it’s sufficient for upcoming payments. The predicted balance and salary were well appreciated for that reason.
• People that receive their salary weekly or every 4 weeks still manage their money on a monthly basis. That is because most fixed charges are debited monthly. Forecasting helps them to plan the month ahead because the payout date of their salary is constantly changing. So for these users, the feature might even be more helpful than for people with a fixed monthly payout date.
• The main benefits of ‘Look Ahead’ are confirmation of expectations, an overview of subscriptions (to see if they should cancel one or more) and it helps them plan the coming month.
• The main improvement (according to the participants) was making the balance predictions more precise, by including more transactions.

What is interesting is that we know ‘Look Ahead’ users feel more in control about their finances. On the other hand, we know that they are saving less on a monthly base than non-users on average. The difference in saving might be caused by an overrepresentation of users with a lower income and less assets in the user group of Look Ahead.

Correlation versus causation

We detected a correlation between using ‘Look Ahead’ and a less healthy financial live, but we don’t know what causes it. You would have a similar problem if you found out that ice cream sells really well on days that more people tend to drown. It doesn’t necessarily mean that ice cream sales has any effect on the chances of drowning (or the other way around). There might be another reason, like the fact that hot days that are more suitable for swimming (and drowning) and enjoying ice cream. So our next mission is to find out how ‘Look Ahead’ affects financial health.

In an ideal world you would grab some pour soul that is using ‘Look Ahead’ ánd his identical twin brother who doesn’t use Look Ahead. Then it’s just a matter of observation. However, we chose an alternative route and assumed the role of matchmaker. With the help of data scientists, we matched 35,000 individuals that used ‘Look Ahead’ with another 35,000 that didn’t. We made sure those 35,000 couples were as similar as possible on all relevant variables we could think of: age, income, for how long they have been a customer, saving balance, number of kids, partner y/n, ING products, app or web use, number of subscriptions and more. We came as close as possible to finding clones of people, with only one difference: the use of ‘Look Ahead’. We observed the differences between the 35,000 couples between Q1 and Q2 and distilled the average of their differences. We measured the difference in:

• Overdraft amount (negative account balance)
• Number of days in overdraft
• Checking account balance
• Savings balance
• Total assets amount
• Failed transactions

Results

With all the variables added to the equation, our matchmaking business couldn’t have brought us closer to proving a causal effect of Look Ahead on financial health.

And then a miracle happened…: the method defused our earlier finding that Look Ahead might be bad for your financial health (that was the correlation we found). Our comparative research showed that the Look Ahead users are healthier financially! Less overdraft, almost €30 more in savings each month and less missed payments than their non-using counterparts.

The earlier correlation might be a result of the user group of forecasting that has more customers with a low budget. This group probably saves less than customers with a higher budget (of which less use Look Ahead).

With this new matching model, we could also check what Look Ahead means for our business goals. Well, the outcome was very pleasing: Look Ahead stimulates people to check the app more often, spend more time in the app and use other features. And last but not least: Look Ahead users are less likely to stop using ING services.

We are very happy to have this data and a have model that was and is so easy to re-use for money management features such as: payment requests, saving targets, tags, and mobile payments.

Knowing this, we can safely promote Look Ahead to other users. On top of that, we can use this research to increase the number of customers, and make future improvements. No doubt about it, I’ll be back with a new article about this stuff.

Conclusion

We made a big mistake when we started a project with the aim to improve customers financial health without ever making the goal explicit enough to measure the impact we made. We lived 5 years with the believe we helped customers, without knowing if we actually did. Luckily we know it now.

Given the fact that we designers are the advocates of our users, we shouldn’t stop when a product looks nice and does the job. Let’s aim for customer wellbeing and prove it with data. We’ll have to focus on changing and improving user behaviour, and make sure the change is measurable. For some features it will be easier then for others, but it’s worth it. We’ll make better products and serve our clients better as a result, and it will help us in the whole prioritization discussion about, for example, customer experience. And who doesn’t like to see and show the ROI of our hard work : )

I’ll end with my secret tip: always team up with the data rock stars in your company…

Thanks for the long read, my name is Freek. I am a designer at ING. ING is the biggest retail bank in the Netherlands. I work at the Manage Money team to design smart tools and insights to help our customers manage their money.