Evaluation and user experience of a financial trading platform
While it is easy to measure the financial performance of a service, the quality of its user experience is intangible and is what drives success.
We worked as New York based collaborator with Livework studio to support J.P. Morgan in measuring the service outcomes of their trading platform, to improve the onboarding experience and work processes of traders (users), the bank (business), and the bank’s financial advisor that run the service (operations).
A service outcome is the task or result that the service accomplishes for stakeholders, it can be broken down to outcomes for each stage of the user experience.
In an hypothetical example,a possible outcome for a pre-transaction phase of trading platform service would be ‘the trader views in real time what financial products are launched on the market’.
Research
We interviewed digital and business analysts to create a blueprint of the current state. By including three distinct service swim lanes in the blueprint for the three outcome types — user, business and operations. This structure of the Blueprint illustrates the work model behind the trading platform, including how outcomes and signals can be observed at any stage.
Building the metrics through a Temporary Environment
We adopted a metric design framework by Livework, that maps Outcomes, Signals, and Metrics
Signal: indicators of a successful service interaction. (Eg. ‘traders use phone calls to enquire on new financial product launched on the market’)
Metrics: numbers of percentages that can measure the accomplishment of the outcome, using the signal. (Eg. ‘Number of inquires per day’)
Signal and Metrics are annotated on each outcome swim lane on the Service Blueprint. To create and start using the metrics, we operationalized this framework working directly with JPM service and digital team, following this process:
For a period of time we tested 12 drafted metrics (4 for each service outcome category) by presenting them to traders and data analysts and asking if those ways of measuring outcomes were valuable to inform their decisions and better evaluate their work and the quality of the service.
Scale through the Path to Data
The last step to activate the use of the metrics was to understand how to source the data, and what new touchpoints should be designed for the data to flow from the traders interactions to the service team. For this step we created the Path to Data tool that is a chart that includes information about each data stream like where to find the data, how to extract them, who manages them and what they can indicate in the context of the service.
Based on the insights from validation, the blueprint was modified to include these touchpoints that would allow to track data and qualitative information while using the platform.
We visualized them into a future state high level UX story to hand over for the implementation phase and the automation of the metric tracking.
Outcomes
— Future state user experience geared on relevant user, operational, business needs.
— Applied metric framework (included research and validation techniques) ready to be applied to other services within the bank.
Reflections
— Service metrics are not ‘objectives’, nor solely based on efficiency, they are affected by underlying values that the service providers want to embed in the service interactions. (eg. Speed of delivery should be a metric if the provider values the time of its users).
— Measuring a service is the result of a human centered research process, to uncover information and facts that motivates service providers to change behaviors or work processes. (eg. If resources are allocated to measure speed of delivery, under acceptable delivery times the providers should be prepared to change back office processes to speed it up).
