Putting myself in the shoes of the User: The Journey of a Data Scientist
A user journey is a visualization of how an individual interacts with a specific system. It’s represented by a timeline of touchpoints between the user and the product. Throughout this journey, you’ll not only be able to understand user interactions with your product, but also their main goals and motivations. By having the complete picture of what tasks they wish to complete and why, you’ll be able to learn their main needs and where opportunities for improvement lie. This process will help prioritize current issues and anticipate possible problems. Since you’ll be putting yourself in the user’s point of view, this picture will naturally piece together. You’ll be surprised, trust me! :p
It just so happened, creating the Data Science (DS) user journey was part of my first task at Feedzai! By creating a DS user journey, I had the chance to really understand the users I would be designing for. This made my onboarding and understanding of the Machine Learning Engine so much easier. If you are a designer who recently joined a new company, this exercise might be a good way to quickly grasp your users and more importantly, understand their needs in a very efficient way. It is a type of exercise that will give you some perspective over the product, but also give your company a valuable process to understand what and where to improve in the product.
Continue reading to see how creating a user journey helped our company.
Why are user journeys at Feedzai so important
Feedzai’s Machine Learning Engine aims to support data scientists in delivering faster and better machine learning models to fight fraud and financial crime. However, we also believed there was more we could do to provide a better user experience. In order to do so, we needed to learn more about the data scientist and hence created a user journey which, ultimately, helped us create a better product.
A user journey would enable us to answer some compelling questions and collect valuable information such as:
- What are data scientists’ main goals when using the Machine Learning Engine?
- How might we empower data scientists in order for them to complete their main tasks more efficiently?
- What activities do they perform on a daily basis inside the product, and what actions are limited by the platform?
- How might we better support daily frequent activities?
- How might we better support activities done through external products?
- Where do they face the most difficulties and how can we improve them?
- How might we improve the product in a way that major difficulties stop occurring?
- Do all data scientists use the product in the same way? If not, what are the common standard procedures, and how do they differ from each other?
- How might we standardize processes to support our users main needs?
- How much time does it take to have a machine learning model ready for deployment? How is that time distributed between phases?
- How might we reduce the time it takes to deploy a machine learning model?
- Are there other relevant personas contributing to the data scientist user journey?
- How might we support the needs of other product personas?
Lacing up: Our approach to designing the Data Scientist User Journey
With my team of UX researchers, product managers and engineers, we got to work:
First, we reviewed our existing documentation to understand the process at a high level. Among the documents, we located the results of a survey inquiring data scientists about their pains with the product.
Next, we interviewed our subject matter experts (SMEs) and internal data scientists to get the details we needed to get access to their direct point of view. At Feedzai we like to be well prepared when having sessions with clients, this means we undertake research first with internal SMEs in order to use the time we get with our clients in the most efficient and productive way.
After we gained some insights on how data scientists train machine learning models, called the Data Science Loop due to its iterative process. At this point we were able to map out the major and most important steps in their workflow. We also were able to identify at which steps data scientists experienced the most difficulties.
From this high-level first analysis, we identified a higher number of opportunities for improvement in the feature engineering and model training pipelines. A following analysis would help us understand more in depth which steps needed urgent attention in the flow and where we should be focusing first.
Curious about how we validated the collected information? Read on to the next section…
Validating our assumptions
We finished the user journey, but can we really say we were 100% sure it was accurate and covered all use cases? At this stage we wanted to validate our assumptions with our clients’ data scientists. In order to do this, we established a user research plan and collected the data.
The goal of this exercise was to understand if the information we had put together matched our users’ reality. We created an online survey and sent it to 45 data scientists across different clients and internal users, obtaining 31 responses in total. Our survey aimed to understand if users experienced the pain points we had listed and to what extent.
The first metric we came up with to be able to measure and prioritize their issues across steps of the Data Science Loop was to request our participants to rate the frequency and frustration of each pain point.
We discovered some interesting insights we were not expecting. Specifically, the prioritization across all the major steps to build, train, and deploy the models did not match the results we had experienced via internal usage. This helped us to understand that the journey might differ between Feedzai and client data scientists. One possible reason is that teams of data scientists often receive different types of training in how to use the tools.
Besides gathering the frustration/frequency metric, we felt the need to dig deeper into that analysis. We understood it was not enough to make informed decisions on strategy so we decided to create an additional metric: time-wasted. To specify this metric further, we created two different categories: small scope and large scope projects. This gave us a pretty good overview of the impact pain points were having in the team productivity.
It turns out, this validation was an essential stage in our UX process since the initial priorities changed after analysing the results. Without it, we could have ended up focusing our efforts in the wrong direction.
Making our journey useful for the company
Our final challenge was to package our findings into a readable and digestible format. We wanted to make it both entertaining and visual. While enabling data scientists to relate to the user journey we designed was important, we also wanted to share this knowledge company-wide.
To accomplish this task, we grouped the following information into three categories: main goal, actions, and use of external products. Next we prioritised pain points and mapped it into the different stages of the journey. This view enabled us to visually have a sense of which stages were more/less complex and where the areas of improvement lied throughout the user flow.
You must be wondering:why did we invest so much time into mapping out this journey? Please ease your mind; we had a purpose: to take our product to the next level! How do we succeed? By incorporating the identified areas of improvement into our roadmaps, along with informing our strategy with the information we collected. In the end, who would be better at helping us improve our product than the users who work with it every day?
Lessons learned
Looking back now, it would’ve made sense to collect customer feedback earlier on so that the validation with end-users wouldn’t be that different from what we were expecting. Of course, as we all know time constraints and availability aren’t always playing in our favor, but still we learn and grow from our mistakes: our internal perceptions don’t always match reality.
