Part II: Redesigning our Rules Experience
After concluding a major milestone of unifying all of Feedzais’ products under one roof, the RiskOps Studio, and setting the baseline to start improving key user journeys, our team put on their armour and decided it was time to tackle the beast: redesigning how users defined their risk strategy by making use of rules.
What are rules?
You may be asking yourself what we mean by “rules” and why they matter when fighting financial crime. Rules are a core component when it comes to defining a risk strategy. They are a mechanism that enables teams to define specific “and/or” conditions based on certain behaviors or attributes of the entities captured by our system (such as transactions with exaggerated amounts or customers from high-risk countries, etc.). A common rule would reflect something similar to this:
Depending on the risk level that the rules are linked to, the rules may lead to different decisions. For instance, rules that decline transactions straight away are usually connected to a higher risk level than rules that send transactions to be reviewed by an analyst. On the other hand, the ones linked to an accepted decision are usually of a much lower risk level, since the goal is to scan out transactions with lower likelihood of being fraudulent.
An example of this type of rule could be:
If the customer ID is present in an accept list AND the amount spent by the customer in the last six months is consistent with the usual spendings of that customer AND the amount is equal or lower than 1.000, THEN accept the transaction.
As you can see the two conditions above are identifying either healthy or harmful patterns, but they make use of the same structure.
Now that we have clarified what rules are and how they can be used, let’s dig deeper into how our users were experiencing them before we introduced something better, more powerful, and easier to understand.
Our UX team previously conducted discoveries to understand the journey users experienced when using rules. We learned the customer experience was often cumbersome, time consuming and, most importantly, very hard to learn and execute in an autonomous way.
There were two types of rules present in two different products, the Self-Service Rules (SSR) and the Pulse Rules. SSR were a feature present in a product aimed for analysts to review transactions and contribute to the risk strategy based on their identified fraudulent patterns. Pulse, on the other hand, was a product used by more experienced users like data scientists who write more advanced rules, create customer behavior profiles, and train machine learning models for instance.
- SSR were designed for analysts with a user-friendly, click-based UI but with less functionalities which led to low usage;
- Pulse Rules were designed for expert users with a code-based UI and allowed for more complex rule scenarios design, but these rules were hard to master;
Users were dealing with a fragmented experience, meaning we were not making use of the best of what the two solutions had to offer. Luckily (or more accurately, strategically), with the RiskOps Studio project, which unified all our products into a single experience, we were about to change it all!
Discover and workshop what great looks like
The first step was to collect all the previous information and feedback available not only from the UX team, but also from other product and customer success teams. Great! Now we were able to analyze the information and compile a summary that allowed us to understand where users struggled the most throughout their journeys.
Secondly, it was of higher priority to understand what we were trying to achieve. We asked ourselves, what would great look like? In this sense, we organized a workshop that had the ambition to:
- Compare the two different rule journeys (SSR and Pulse), identify their strengths and pain points
- Identify the top 3 How Might We’s
- Define the ideal Rules User Journey
The next step was to quickly create a prototype that allowed us to visualize what this journey could be and showcase it to the product leadership team for further buy-in. Expectations were managed that what was being shown would go through multiple iterations, since the heavy work had not yet begun. We got the green-light and we were ready to embrace the challenge!
Break the journey into smaller chunks
This project was H-U-G-E! It needed to be broken down into smaller pieces so the UX team could iterate faster but also avoid dependencies during the implementation phase, enabling us to deliver solutions to clients efficiently.
The rules journey is a sequence of steps that need to be followed in a linear fashion. In the current product, the first step was to write a rule composed by one or more conditions which will state the behavior the rule is analyzing and link to a decision that should label the transaction once the rule triggers. After having the rule created the user would define its priority among the other rules implemented, (i.e., which rules should be analyzed by the system first and in which order, since the first rule triggering will be the one labeling the transaction).
The next step was to test and tune any necessary parameters in the rules so they would match the KPIs defined that can be different for each individual customer. For instance, a customer with a smaller analyst team might give more attention to KPIs like “alert rate”, meaning how many alerts are being triggered by the system, since they need to closely monitor the team’s capacity.
If metrics are looking healthy, the rules can be made effective in the system (Go live) and the monitoring phase will begin. In this step, users will undergo an established recurrence check on the rules’ performance and understand if they are contributing to the strategy in an efficient way or if anything needs to be updated.
Rule Writing
Rule writing is one of the most complex tasks users currently struggled with, so we decided it would be the step we would address first.
The team dedicated a lot of time to defining requirements, one of the most important steps of the UX process, and many times underestimated in its relevance. Jointly with the PM and subject matter experts we wrote and rewrote user needs and framed them as user stories. The user story format would help us later on when creating the designs, since needs would be framed in a way that would make us look at the problem we were solving for the user and not think of solutions straight away. An example of how we framed it was: “As an analyst I need to have a way to choose between multiple decisions (review, decline, accept) so that I can link rules to multiple risk level types and build my strategy in an effective way.”
Then, we used a Figjam board to build specific user flows, search for inspiration, design multiple versions of wireframes and collect concrete and complex rule examples which allowed us to really understand the level of complexity our users experienced. Along the whole way we had the pleasure of working with a very talented data scientist who supported us in tasks such as providing examples, clarifying questions, understanding data, and completing usage patterns. His participation was key, since we could make decisions based on data-driven insights instead of assumptions and colleagues’ opinions.
Writing rules is a complex task which requires us to act as if we were completing the journey ourselves. This meant a lot of excel sheets mapping which fields should be available for users to use, identifying dependencies between them (e.g., if a user chooses a certain field in the first input they should only be able to choose between X and Y fields), and document error messages. These documents worked as deliverables for the engineering team as well, due to the project’s inherent complexity they would support them in writing their Technical Design Documents and guide them through implementation details.
The last step culminated into designing the UI itself with high-fidelity mockups and creating a working prototype to test ideas early.
Although the first solution performed well in usability tests with a task success rate of 95.41% and a users’ satisfaction of 5.38 (based on a 7-point scale), we wanted more. Specifically, we wanted both higher usability and higher user satisfaction.
Our team is very resilient so we decided to keep going and improving our solution even further. We designed a second and improved version that included new features including:
- Quick filtering
- More intuitive and fluid rule reading
- Automated defaults and field inference
- Keyboard shortcuts
- Fewer steps and clicks
As expected, the second version performed much better during usability testing sessions, both user testing (users collected from a tool we use to gather participants that match a certain criteria) and client feedback was very positive (10 users total). This new version was very dependent on text writing so we combined forces with our front-end developers and created a working browser prototype, instead of using a design prototype which came with a lot of limitations and could put results in question.
Success!!! Results increased drastically when we compared the new version to the current one. Metrics like functionality didn’t decrease and ease of usage and learnability showed great progress.
This was only possible due to a very close alignment between teams, the UX team, engineering, and other departments such as research were meeting on a regular basis to guarantee nothing was overlooked.
Rule Strategy
We were on the right track and very motivated to continue improving the rest of the steps within the rules journey. This was only the beginning of a new and complete experience.
Besides being able to write rules, users needed to have a way to define how rules should interact between each other, in other words, define the priority between rules. In the current solution this was identified as one of the most important but also one of the most cumbersome tasks. We had to dedicate special attention to solving this problem.
In the current system, priority would be defined at a rule-level. It’s essential to highlight that some of our clients have hundreds of rules enabled, which makes this task hard to perform with a drag and drop design. Additionally, it was difficult to understand and memorize what each rule would do and think about the level of importance at that level of granularity. Jointly with the data scientist in our team we brainstormed multiple options and came to the conclusion that the simplest solution was to stop prioritizing rules and start prioritizing decisions. This would reduce the prioritization process from more than 100 rules to 4–5 decisions. Another important decision was to remove the functionality of branching out priorities and simply imposing a linear rule evaluation.
One of the most important conclusions from the team was that besides linking priority to decisions instead of rules, the prioritization step should become implicit in the experience, instead of making users have to go through that step every time they wrote a rule. The solution was for users to have their decisions prioritized in a linear sequence and be able to write/add rules directly linked to a decision, meaning directly linked to a priority. This would cut down steps and simplify the overall journey. This also meant our journey would change its current order, instead of starting with the rule writing step, users would begin by implicitly defining the rule priority while adding rules to their strategy.
In a nutshell, one of the methods that contributed to the success of the solution was not to look at the current rule implementations as hard requirements, but to analyze why users would do tasks in a certain way. Then, look into whether that method could be changed into a simpler solution that would continue to support users in their end goal but make the path ahead more straightforward. It was really satisfying to see how all the complexity was turning into something much simpler, but yet very powerful.
The team will continue to work on the project and complete the remaining steps of the journey in order to have a complete and improved experience. We will be testing and releasing the experience to clients as we improve the solution in an iterative way, making sure we learn and adapt in short cycles.
Having the chance to execute a project like this gave me the opportunity to learn many new concepts and interact with multiple teams. It was a great opportunity to apply the whole UX process in order to simplify an experience that is inherently complex and hard to maintain. I’d like to give a special kudos to Laura Michalik, the supporting designer in this project. I had the pleasure of mentoring her throughout many adversities we faced and I’m sure that after this huge milestone achieved she’ll be ready to face any upcoming challenges.
