Designing A Machine Learning Compliance Product / Part 1
Understanding the challenges & product landscape
Introduction
This series is a running journal I kept over a 10 month period while designing a financial technology (FinTech) application for a major US financial institution.
The product’s goal is to enable financial compliance analysts to discover and mitigate employee malfeasance. Analysts often have legal backgrounds or more often are former traders themselves. This domain knowledge of what terms traders use and topics they discuss is the best tool banks have currently.
This series will cover the following aspects of the product design:
- Part 1: Understanding the challenges and existing product landscape
- Part 2: Picking a design language framework
- Part 3: Agile sprints, User stories, & Atomic design
- Part 4: User Research
- Part 5: Composition Evolution [Will return after updates, sorry]
Problem Set
The financial industry is suffering from an out of control wildfire of fraudulent activity that has cost financial institutions billions in fines from regulatory agencies. Wall street needs software that will assist them in being able to discover and mitigate employee malfeasance on an enterprise-level scale asap.
Product Roadmap Strategy
As with most other products a kickoff meeting was conducted over a 3 day period. This was the stakeholder’s chance to set the agenda and establish the overall direction for the product. Features considered critical path included Information Leakage Detection, Collusion Detection, Trader Profiles, and Email Communications Analysis.
The meetings were on track until day 2 when one of the project managers began to draw on his iPad a wireframe of how he foresaw the product looking. I quickly chimed in to say,
Until we complete some initial research, discussing the layout of the app is a complete waste of time
The reaction by most in the room was split. Half the room agreed and understood many steps were being skipped. While the other half were shocked and dismayed that anyone would stand up and say no to a project manager.
I continued by sharing the vision that this would not just be a product that picked up where competitors had left off. But that it would be a completely new person-centric, not alert based, machine learning driven compliance product.
As a user experience designer I feel it’s important to establish at the beginning of a project who is responsible for what tasks. If anyone, most often an over zealous project manager, tries to do your job for you it's on YOU to speak up and explain why it isn't. Having the guts to defend design (or sometimes over defend) is, in my opinion, one of the more lacking characteristics of less experienced UX designers.
There is also a fine line with this. Push back too hard you’ll offend the stakeholder. Do too little and you risk being run over and have every design decision questioned. It’s up to you to evaluate the balance needed here based on your relationship with the stakeholder to successfully work together without surrendering your ability to be creative and try new things.
Existing Product Landscape
With my background in US federal consulting it was oddly comforting to see that both government and financial industries suffer from similar issues. It’s changing slowly, with groups like the US Digital Service, but user experience is still an afterthought.
When I was on a call not long after the kickoff meeting one of the stakeholders was talking about another product and simply stated,
Just get the infrastructure up and we’ll slap a UI on it.
It was clear from this point that I had my work cut out for me when it came to educating the stakeholders about all the effort that goes into designing a truly amazing user experience.
This lack of empathy towards users in an enterprise setting is understandable to a degree. There is no real incentive for companies to dedicate time and money when the employees are required to use the software to perform their job.
But there are certainly qualitative factors that employee’s demand for happiness and satisfaction. These qualities have been shown to be a factor in overall employee productivity.
Browser Spreadsheets
Spreadsheets have been bastardized for years in situations where a database would be the better solution. What became a simple ledger evolved into pivot charts. I pledged to myself at the start of this project that I would work as hard as I could to avoid spreadsheets.
Below is an example of CA Technologies Dataminder. It uses regexes made of simple “dumb” rulesets to identify possibly bad behavior. But as traders evolve their slang so does the work of the analyst to modify these rules to continue to discover malfeasance.
Since the rules are term based the recall is incredibly low and analyst spend much more time managing the rules and eliminating false positives than they do reviewing for potentially bad behavior.
User Interview Impediments
It was a struggle to get the stakeholders to understand the value of contextual user interviews and the data it would yield. Obtaining this data would go a long way in guaranteeing the user experience was designed to meet both the business and user needs. Until now the stakeholders, not the users, had total control over all user experience design decisions. It was clear this would be the first major battle and one I was determined to win.
Stakeholders, not the users, had total control over all user experience design
The Blind Designer
My initial requests to speak directly with users was met with bewilderment and disbelief. I was told by one project manager,
The user's time is very valuable to this firm. If you’d like to know anything I can tell you. What would you like to know?
It was very clear he was gatekeeping. It was also clear that he had no concept of user research as he told me, “How does amazon know to make changes to their site? They don’t call my grandmother and ask her what she thinks, they just build it.” This is of course a ludicrous statement as Amazon has amazing user research teams in Washington state and London, UK.
I left that meeting with no resolution and no access to users. So with no research to help me define the problem I would be trying to solve, I was stuck in a awkward position.
In a situation like this I often use the following metaphor:
Imagine you hired an interior decorator for your home. Throughout the entire process, where you select and install furniture, wallpaper, paint, etc. you never have a single meeting with the designer. There is no opportunity for them to obtain an understanding of your taste and style. Without this interaction the likelihood that you’ll be happy with the results probably won’t be very high.
It was clear I was just going to have to gather what information I could from stakeholders knowing that it would carry bias and not be entirely accurate. Their area of focus was to concentrate on speed and reducing the amount of mouse clicks. When I finally got to interview the users this turned out not to be anywhere on the main list of user needs.
