When playing back users’ data, context is key

Sometimes doing the same thing in different contexts produces surprisingly different results:

Paul Izod
Paul Izod
Apr 30 · 6 min read
A picture of Compare The Market’s homepage, showing the products they offer comparison on, displayed on a laptop screen
Compare the Market’s homepage

Recently at comparethemarket.com, we have been exploring ways to make the form filling process quicker and easier for our customers. One of the methods we are exploring is to source the user’s data from third party sources on their behalf and pre-populate answers for them. The idea being that this would result in less effort for the user and in more accurate answers - a better experience for the customer and, hopefully, better conversion for us.

So, we tested applying this to two of our journeys - home insurance and energy switching - and were surprised to get quite different results.

So what gives?

For home insurance, everything went more or less to plan. We asked the user for their address, sourced their data while advising that we were doing so, then played back the resulting information and asked the remaining journey questions.

Pictures of the Compare the Market Home Insurance prototype, where a user is asked for their address, then shown a loading screen, which then leads to a page where information about their property is displayed
Pictures of the Compare the Market Home Insurance prototype, where a user is asked for their address, then shown a loading screen, which then leads to a page where information about their property is displayed
The Home Journey Prototype — Address, Source, Present

We ran four test phases and the overall response was positive, with no specific concerns around privacy and security, as expected.

The response for the energy testing was quite different though. We applied the data sourcing in the same way, by asking for the user’s address, sourcing the data behind an interstitial that advised the customer we were doing so, then played back the usage data that we sourced. However, the user response was much less positive.

Pictures of the Compare the Market first Energy prototype, which asks the suer for an address and then displays the energy usage data for that address
Pictures of the Compare the Market first Energy prototype, which asks the suer for an address and then displays the energy usage data for that address
The first Energy prototype — Address, Source, Present

We ran two test phases, resulting in some users responding negatively to being shown their energy usage data (annual kWh usage figures for both gas and electricity as well as the provider company). In addition, a small number expressed concern around how the data would be used - quite the difference from the results on the home insurance journey.

The results of the first Energy prototype testing, where 29% of users responded negatively to seeing their data, 8% were concerned with how the data would be used and 66% did not understand what a kWh represents
The results of the first Energy prototype testing, where 29% of users responded negatively to seeing their data, 8% were concerned with how the data would be used and 66% did not understand what a kWh represents
The first prototype’s testing results

So, I did some further exploration by adjusting the designs. The feedback indicated that a proportion of the negative reactions were around users feeling that we had sourced their data without their permission, so I created a prototype of a journey where we ask the user’s permission up front, in the form of a pre-checked checkbox labelled ‘use my address to look up my current energy usage for me’, as well as some reactive messaging if the checkbox was unchecked.

A picture of the second Energy prototype, where a checkbox giving permission for Compare the Market to source the users data has been added, along with some supporting infirmation
A picture of the second Energy prototype, where a checkbox giving permission for Compare the Market to source the users data has been added, along with some supporting infirmation
The second energy prototype — Pre-selected permission checkbox

I found some interesting results in this testing phase, where nearly all users proceeded with the permissions box checked, but a proportion of those users reacted negatively to being presented with their usage. Alongside this, nearly all users had incorrect assumptions around what data we would be able to retrieve: with expectations ranging from information about their house, their meter type to just ‘no idea’, rather than their provider, and gas/electricity usage.

the results of the second Energy prototype testing, where 95% of users proceeded with the permission checkbox checked, but 7 of those users reacted negatively to seeing their data making a 17.5% negative experience
the results of the second Energy prototype testing, where 95% of users proceeded with the permission checkbox checked, but 7 of those users reacted negatively to seeing their data making a 17.5% negative experience
The second prototype’s testing results

From this, I theorised that the negative reaction was possibly due to a combination of users not noticing the checkbox fully and a discrepancy between what data they expected us to source and what we actually played back to them.

To address this, I created a third prototype, this time with more explicit reference to what data we would source, additional surfaced support messaging and by replacing the checkbox with a non-defaulted yes/no radio question. My assumption was that by forcing the user to answer the question it would, along with the clearer messaging and support, mean that users were making a more conscious decision on whether to allow us to source their data.

A picture of the third Energy prototype, where the permission checkbox has been replaced with a mandatory yes/no binary question, with a label explicitly detailing the data that will be sourced
A picture of the third Energy prototype, where the permission checkbox has been replaced with a mandatory yes/no binary question, with a label explicitly detailing the data that will be sourced
The third Energy prototype — Explicit permission question and signposting

This time the results were more positive. While less users gave their permission this time, all of those that did reacted positively to being shown their usage data. Reasons given for those that refused permission were around privacy concerns and not understanding what they were being asked to give permission for.

So, with all the testing complete, we could see a significant baseline response from the different products’ user groups to having their data played back, but why? I did some further review of the general nature of each sector, as well as going back and reviewing the users’ verbatim comments during testing and identified what I believe to be the two main reasons for the differences - industry precedents and data perception.

The results of the third Energy prototype testing, where 83% of users gave permission for data sourcing and none of them reacted negatively to seeing their data. Of the 5who refused permission, 3 expressed privacy concerns and 2 were unsure what they were giving permission for
The results of the third Energy prototype testing, where 83% of users gave permission for data sourcing and none of them reacted negatively to seeing their data. Of the 5who refused permission, 3 expressed privacy concerns and 2 were unsure what they were giving permission for
The third prototype’s testing results

The maturity of the data-sourcing model in home insurance and energy are at quite different stages. In home insurance, the sourcing of users’ data from third parties is well established, with a great deal of mainstream companies utilising the functionality. This means that the precedents for data sourcing have already been established in the users’ minds and they come into the quote process expecting some of their data to be sourced for them, allowing them to see it as a convenience rather than an intrusion. In the energy sector, however, data sourcing is in its relative infancy. While some companies are using this functionality, it is mostly smaller entities, meaning the precedent has been set for a much smaller proportion of the customer base, potentially meaning the data being presented back comes as a surprise.

The perception of the nature of the data being sourced also appears to be different between the two products. On home insurance, users largely perceived the data as being basic factual information about their property, rather than something directly linked to them. However, on energy users indicated that they perceived the usage data as information about them and their behaviour. The information felt more immediate and personal, making them more protective about its sourcing and use.

From looking at the application of data sourcing in our home insurance and energy products, I was able to identify these four key takeaways:

1. Context is key: Users’ perception of data varies dependent on how they perceive its context — is it material or personal?

2. Pre-existing precedents: Established precedents, or lack thereof, within a sector/industry can dramatically effect user response to sourced data.

3. Seeking permission: Asking the user’s explicit permission up-front gives them greater ownership of the decision and can lead to a more positive response.

4. Dual-Funnel approach: giving those users who refuse permission an alternate ‘manual’ route makes it possible to retain them also.

We’re currently expanding the team at comparethemarket.com and are looking to hire UX researchers and designers. Get involved.

Compare the Market

The people behind comparethemarket.com and the Meerkat App