My Journey into Research Ops: Integrating UX Research into Agile and Leading Change with Synthesized Qual and Quant Research
Joining robust CXO, Analytics, and Design Teams as a sole UX Researcher
Banking and Credit Card Company (Fortune 100)
Role: sole UX Researcher on 3 products within Digital Acquisition
Within: Digital Acquisition, Global Website for both Personal & Business Credit Cards
Working with: Product Owners, CXO, Directors, Marketing, Analytics, Data Personalization, and Design Teams
Managing: Scope of Work for Ongoing UX Research, Vendor relationships & Services
Research Ops manages many relationships, priorities, timelines, and processes. It is a challenge. Teamwork, templates and clear methods help keep the most urgent questions and long term goals progressing toward more understanding.
Here are 5 steps and one example of how I approached the challenge of integrating UX research into more and less agile product teams on digital acquisition.
Step 1: Researching the company.
I reached out to other UX Researchers within the company to understand best practices, vetted vendors, tools, and share strategies and applicable research assets.
There was little community within the UX Researchers dispersed within Amex and a meager central hub for sharing assets. We set up a monthly gathering, a slack channel, and started working toward a central location for sharable assets. It was a start to unify knowledge across UXR community.
Step 2: Discovering Stakeholder’s Questions, Goals, and Understanding.
I met with different roles within Product, CXO, Analytics, Design and Marketing to understand their persistent questions and their pressing goals for the quarter and beyond.
Questions: Even though they had numbers as to what people were doing on the shop website, they did not know WHY the people were behaving that way or WHO the prospects were.
- Why did certain design components “win” on A/B or MVT?
- Why did people drop off the funnel at certain spots in the journey?
- Who were the prospects that came to the website?
- What motivated the journey to get a credit card?
- Which eligible prospects dropped off, and what were they looking for?
Goals: Increase Digital Acquisition with Card Members and Prospects.
- Understand who the Prospects and Card Members are.
- Encourage Card members to Upgrade, Add an Additional Card Holder, or get an Additional Card.
- Keep Eligible Prospects engaged and applying for the best card for them and for the business.
- Educate Card Members on how to expand their membership.
- Personalize the shopping experience to illuminate the perfect card / offering to that specific person.
- Create a Template for shop experiences worldwide, understand the different requirements for different markets and how to integrate and adjust for those markets (eg. Japan, Australia).
Existing Understanding: Limited and Underutilized.
- Robust MVT and A/B/n testing produced wins that decided the final outcome of the website. These wins had statistical significance, but no deeper understanding of the entire journey, or why they won.
- Some personas existed, but buried in other teams and unknown to stakeholders. All were aligned to service and not acquisition.
- The external research on the shopping journey was not incorporated to personas or design decisions.
- Sporadic usability testing had offered some insights, but these were largely forgotten and not always acted upon.
Step 3: Proposal for Expanding UX Research throughout the Agile Process.
In order to integrate UX research better, I paired closely with the Design Ops Lead and Product Owner of commercial shop to propose a new way to integrate research:
- to win Product more of the budget and process,
- to ensure Design had access to timely usability feedback,
- to collaborate and enhance rapport between CXO and Design.
Step 4: Synthesizing Research Methods by Collaborating with CXO, Analytics, Quantitative Data, and External Research
Many useful insights were overlooked and forgotten. Change came largely from random marketing initiatives guessing how to make the website better.
With the Director of CXO, we created a system to raise consistent observations from many types of research, so that we can act on research.
Qualitative Insights Process
Observations → Hypothesis →Ideas / Solutions
To avoid Hippo (Highest Paid Person in the Room has the loudest voice), we want to clarify that: An idea/solution without Observations = *Opinion.
*Opinions can be tested as well, but it is important to understand where solutions come from and we can prioritize solutions based on this rubric.
- We maintain a research tracker of past: usability, surveys, external, chat, Analytics, discussion groups, calls, flows, personas, heat maps, etc.
- From this tracker we prioritized the consistent and impactful observations.
- Then we host a bi-annual Ideation Day, in which groups of people come up with as many solutions as possible. The backlog of solutions are then prioritized and taken to the iterative Design process.
Step 5: Managing Relationships and Many Timelines and Details
Some tools I use to facilitate relationships and sharing ideas:
- Weekly research share with Design Team, Keynote/Powerpoint/Sketch.
- Confluence page of searchable observations, hypothesis.
- Creating Confluence Pages for Each Product Team, and Usability Test Logs.
- Asana and Jira to handle intake and keep my workflow visible to the design team and other stakeholders.
- Physical wall of Research Assets at the Design Teams workspace.
- Using Oracle to enter Vendor’s SOW into the system.
- Inviting stakeholders at many steps of the process via webex and outlook.
- Synthesizing especially readouts and offering feedback as to how they can leverage the take-aways into their design process.
- Running Ideation workshops with many different stakeholders, using whiteboards, printed assets, sticky notes, timer.
- Clarifying goals with vendors with inVision, edits in Word, and relaying Qualtrics data.
Example Design Integration:
Sprint for Card Member (logged-in and cookied)
Prepare Digestible & Tactile* Assets for Sprint (eg. personas, userflows, analytics data, KPI’s, hypothesis, business goals and data, external/competitive research) *Powerpoints are very limited/disallowed — send out background decks in advance of the workshop.
Sprint kickoff Workshop 💡 : userflows, pull out existing personas and align teams with how might we and design workshop, analytics presentation, limitations from business.
Key data: Entry points, NAR, CTA in Banner vs. lower on the page
Business Priorities: Add Supplemental Card Holder, Upgrade, Cross-sell
Limitations: Navigation and some copy and design pattern wins
Assumptions and Hypothesis: discover with research and stakeholders
eg. Card members have active lives with frequent travel and financial collaboration with friends and family. They appreciate tailored rewards, and are confident with one aspect of their card, but are mystified as to the other benefits the card offers or how to incorporate that into their busy life. How might we educate and encourage Card Members to expand the use of their card within their unique lifestyle?
Ideation and Design Studio 💡 : many stakeholders generate ideas/solutions and vote on best solutions, then design team iterates (using usertesting.com).
Rounds of Feedback
1. Daily iterations, usertesting.com
2. Usability session, screener & discussion guide
3. Discussion group feedback session with card members, offering insight into how people think about personalization, what motivates them to add supplemental card members.
Iterations that integrate CXO
MVT testing→Development handoff
Wins from MVT come back into the research pipeline.
Overall, there is a memory of the product and research (!) validating/breaking assumptions and solutions.