My Journey into Research Ops: Integrating UX Research and Leading Change
Banking & Credit Card Company (Fortune 100)
Role: sole UX Researcher supporting 3 products
Within: Center of Excellence on Digital Acquisition, Global Website for both Personal, Business, and Referrals
Collaborating with: Product Owners, CXO, Directors, Marketing, Analytics, Personalization, and Design Teams
Managing: Scope of Work for Ongoing UX Research, Vendor Relationships, Qualitative Findings
Research Ops manages many relationships, priorities, timelines, and processes. It is a challenge. Teamwork, templates and clear methods help keep the most urgent questions and long term goals progressing toward more understanding.
Here are 5 steps and one example of how I approached the challenge of integrating UX research into more and less agile product teams on digital acquisition.
Step 1: Assessing Existing Practices & Knowlege.
I reached out to other UX Researchers within the company to understand best practices, vetted vendors, tools, and share strategies and applicable research assets.
There was little community within the UX Researchers dispersed within Amex and a meager central hub for sharing assets. We set up a monthly gathering, a slack channel, and started working toward a central location for sharable assets. It was a start to unify knowledge across UXR community.
Step 2: Discovering Stakeholder’s Questions, Goals, and Assumptions.
I met with different roles within Product, CXO, Analytics, Design and Marketing to understand their persistent questions and their pressing goals for the quarter and beyond.
Questions: Even though they had numbers as to what people were doing on the shop website, they did not know WHY the people were behaving that way or WHO the prospects were.
- Why did certain design components “win” on A/B or MVT?
- Why did people drop off the funnel at certain spots in the journey?
- Who were the prospects that came to the website?
- What motivated the journey to get a credit card?
- Which eligible prospects dropped off, and what were they looking for?
Goals: Increase Digital Acquisition with Card Members and Prospects.
- Understand who the Prospects and Card Members are.
- Encourage Card members to Upgrade, Add an Additional Card Holder, or get an Additional Card.
- Keep Eligible Prospects engaged and applying for the best card for them and for the business.
- Educate Card Members on how to expand their membership.
- Personalize the shopping experience to illuminate the perfect card / offering to that specific person.
- Create a Template for shop experiences worldwide, understand the different requirements for different markets and how to integrate and adjust for those markets (eg. Japan, Australia).
Existing Understanding: Limited and Underutilized.
- Robust MVT and A/B/n testing produced wins that decided the final outcome of the website. These wins had statistical significance, but no deeper understanding of the entire journey, or why they won.
- Some personas existed, but buried in other teams and unknown to stakeholders. All were aligned to service and not acquisition.
- The external research on the shopping journey was not incorporated to personas or design decisions.
- Sporadic usability testing had offered some insights, but these were largely forgotten and not always acted upon.
Step 3: Propose Effective UXR throughout the Agile Process.
In order to integrate UX research better, I paired closely with the Design Ops Lead and Product Owner of commercial shop to propose a new way to integrate research:
- to win Product more of the budget and process,
- to ensure Design had access to timely usability feedback,
- to collaborate and enhance rapport between CXO and Design.
Step 4: Research Dashboard, Synthesize Research Methods, Prioritize Trends
Many useful insights were overlooked and forgotten. Change came largely from random marketing initiatives guessing how to make the website better.
With the Director of CXO, we created a system to raise consistent observations from many types of research, so that we can act on research.
Qualitative Insights Process
Observations → Hypothesis →Ideas / Solutions
To avoid Hippo (Highest Paid Person in the Room has the loudest voice), we want to clarify that: An idea/solution without Observations = *Opinion.
*Opinions can be tested as well, but it is important to understand where solutions come from and we can prioritize solutions based on this rubric.
- We maintain a research dashboard of past: usability, surveys, external, chat, Analytics, discussion groups, calls, flows, personas, heat maps, etc.
- From this dashboard we prioritized the consistent and impactful observations.
- Then we host a bi-monthly Ideation Day, in which groups of people come up with as many solutions as possible. The backlog of solutions are then prioritized and taken to the iterative Design process.
Step 5: Accelerate Decisions with Research
Some tools I use to facilitate relationships and sharing ideas:
- Weekly research share with Design Team, Keynote/Powerpoint/Sketch.
- Insight Dashboard for each Product of searchable observations, hypotheses, sources.
- Inviting stakeholders at many steps of the process via webex and outlook.
- Synthesizing readouts and offering prioritized recommendations to leverage into design and A/B/N MV Testing.
- Running Ideation workshops with different stakeholders, using whiteboards, printed assets, sticky notes, voting, timer.
- Asana and Jira to handle intake and keep my workflow visible to the design team and other stakeholders.
- Physical wall of Research Assets at the Design Teams workspace.
- Using Oracle to enter Vendor’s SOW into the system.
- Clarifying goals with vendors with inVision, edits in Word, and relaying Qualtrics data.
Example Design Sprint Integration:
Sprint for Card Member (logged-in and cookied)
Prepare Digestible & Tactile* Assets for Sprint (eg. personas, userflows, analytics data, KPI’s, hypothesis, business goals and data, external/competitive research) *Powerpoints are very limited/disallowed — send out background decks in advance of the workshop.
Sprint kickoff Workshop 💡 : userflows, pull out existing personas and align teams with how might we and design workshop, analytics presentation, limitations from business.
Key data: Entry points, NAR, CTA in Banner vs. lower on the page
Business Priorities: Add Supplemental Card Holder, Upgrade, Cross-sell
Limitations: Navigation and some copy and design pattern wins
Assumptions and Problem Statement
Card members have active lives with frequent travel and financial collaboration with friends and family. They appreciate tailored rewards, and are confident with one aspect of their card, but are mystified as to the other benefits the card offers and how to incorporate benefits into their busy life.
How might we educate and encourage Card Members to expand the use of their card within their unique lifestyle?
Ideation and Design Studio 💡 : many stakeholders generate ideas/solutions and vote on best solutions, then design team iterates.
Rounds of Feedback
1. Iterations, usertesting.com
2. Usability session, screener & discussion guide
3. Discussion group feedback session with card members, offering insight into how people think about personalization, what motivates them to add supplemental card members.
Iterations that integrate CXO
MVT testing→Development handoff
Wins from MVT come back into the research pipeline.
With an active dashboard and prioritization model, we created a memory of each product that can track patterns in both user behavioral trends and stakeholder hypotheses.
Now, it is more clear when there is confirming findings, leading to easier decisions, validating/breaking assumptions and solutions.
Creating a platform anyone can contribute to, the dashboard acts as an open platform for brainstorming, when a stakeholder has an idea in casual conversation or in an ideation meeting, they can put it into the dashboard and see if there is any feedback to support it. Perhaps it is supported by research that is not in the dashboard yet; they can put that in as well. Good ideas can come from many places!
Increasing 25–50% of all A/B/N and MV Testing hypotheses to directly relate to observations from the dashboard.
Bi-Monthly Ideation sessions that generate hypotheses and design iterations that directly address observations that are impactful and overlooked.
1 PI sprint ahead to allow for discovery and research before kick-off.