Uncovering a Backup Admin’s Journey through a Diary Study (User Research Case Study)
The Background
Here at Rubrik, the User Research team is driven by user insights to build better products for our customers. This means truly understanding who our users are, including their needs, motivations, and behaviors over time. When it came to one of Rubrik’s core products, SLA domains, we had a solid understanding of how users were setting them up, but we didn’t know as much about how users were interacting with the SLAs and backups after the initial onboarding. Some of the questions the product team had included: What are these users’ key tasks? How do they behave? What are their challenges? To gain more insight into these areas, I launched a discovery research project to learn more about the daily activities of the backup admins using Rubrik. Put simply, we wanted to paint a picture of the Day 2 Customer Journey.
As a researcher, I found it to be the perfect opportunity to not only conduct foundational research that would help drive UX improvements but also help build greater empathy for our users within our company. This was the first time our research team was using this methodology at Rubrik so we were excited to see what we could learn from the results.
If you’re wondering whether a diary study is the right methodology for your discovery research project, what you need to plan for, and what you’ll get out of it, read on to find out!
The Research Methodology
Since we wanted to explore the behavior of our users remotely over multiple days, a diary study was the clear research methodology of choice. A diary study captures qualitative information about users’ behavior by having participants record their activities each day over a period of time. If you’re looking to gain clarity around a particular persona and their user journey, a diary study is a fantastic way to aggregate this information because you get both the breadth of a longitudinal study and the depth of individual participant responses.
Since the scope of the project was broad, I decided to first determine what categories would be most essential to focus on. This helped me target specific areas where the team wanted to learn more and better structure the diary study questions to get more focused, in-depth responses. In order to determine the topics of interest, I interviewed both internal and external experts to identify the categories of tasks and common challenges when it comes to interacting with our backup product. Rubrik’s Sales Engineers, Technical Product Managers, and Customer Success Engineers are frequently on the front-line with customers, so they were able to share their knowledge and insights with me. I also spoke with 3 customers who were in a senior or management role at their company, who offered a big picture view of how their teams use Rubrik’s products. Using the information I gathered from these interviews, I identified the major areas that we would probe into for the next phase: Maintenance, Monitoring, Reporting, Troubleshooting, and Optimization.
Planning the Diary Study
Since diary studies typically take place over several weeks and are larger in scale, your life as a researcher will be much easier if you thoroughly plan the logistics ahead of time. This means making sure that the timing of the schedule is solid and most tasks are set up to run as automatically as possible. Here are the logistics I considered and my decisions for each aspect of the study:
Length of the diary study: Consider your persona and assumptions around what a typical week or month looks like for them.
- Based on previous research and tribal knowledge, our hypothesis was that most Backup Admins would have a daily routine, which may sometimes be interrupted by unexpected incidents that required their attention. With that, I decided that three weeks (15 workdays) would be an adequate amount of time to understand users’ habits and usage scenarios.
Timing of diary study prompt: Figure out when your users will have the time and capacity to provide solid responses.
- Backup admins are busy people who sometimes have to address unexpected incidents, so I didn’t want to interrupt them with a prompt sent randomly during their day and risk them not being able to fill out their response. Instead, I decided to send out the prompt at a set time near the end of their workday, according to each participant’s preference and timezone. I used Gmail’s “Schedule Send” feature to schedule the daily emails for each person.
Diary medium: Determine what format(s) best allows your participants to convey their responses.
- In terms of the format, I settled on a Google Form since it’s a medium familiar and accessible to everyone and allows for the submission of text as well as photos, screenshots, videos, and audio files. I encouraged participants to submit visuals if it helped illustrate what they were describing and, in the last few days, I gave them the option to submit an audio note as an alternative to writing about their day.
- I didn’t consider the audio responses until later on during the study, but this format was greatly appreciated by some participants who preferred to talk about their day over writing about it. Hearing their voices also led to more natural, personal responses. This will definitely be a technique I employ in future diary studies.
Diary questions: Write questions that will prompt your participants to reflect and encourage answers below the surface level.
- Each day’s diary form started off broadly by asking the user to walk through their day, followed by more specific questions in each of the categories discovered from the first phase of research. Many of the questions were the same across most days so that we could see how their responses changed (or didn’t change) over time, allowing us to identify patterns more easily. I made sure to also include some questions just for fun (like asking them to pick an emoji that best described how they were feeling) to help them open up in their writing and tap into their emotional state.
Recruiting participants: Decide on your target persona and consider diversity in attributes such as industry, company size, etc.
- Because a diary study is a record of daily activity, I decided to recruit users who use Rubrik on a nearly daily basis. Most of these users fall under the Backup Admin persona, although with the blurred lines between roles these days, some also hold responsibilities of Cloud Admins and Security Admins as well. Overall, we had 9 participants for the diary study spanning multiple countries (U.S., U.K., Belgium, Sweden) as well as industries (including manufacturing, education, and IT tools) and size (<100–50,000+ employees). As with our other research projects, we also provided a monetary incentive for all participants and confirmed their commitment to the 3-week long study.
- One thing I didn’t consider at the time of the study but would for future diary studies is the stage of life of the customer, as this can affect their interactions with the product. For example, a customer in their first few months of using Rubrik will likely need to interface more with the product while a customer who has settled into more of a maintenance mode will have a different routine.
Communication with participants: Determine touchpoints throughout the project and maintain an open line of communication to make your users feel they are being heard.
- It was important that our customers experienced a personal connection to this diary study so that they could provide truly reflective answers and feel that they were making a valuable contribution. I provided the participants with a one-pager on the details of the diary study as well as set up introductory calls with each user with the goal of explaining the project and to get to know them more on a personal level. I also scheduled closing calls with each participant after the final day to follow up on any comments or open questions.
Running the Diary Study
One major piece of advice my manager gave to me before starting the project was to not wait until the conclusion of the study to start performing analysis. Instead, on each day of the diary study, I allotted time to read through the previous day’s entries. By reviewing and tagging the responses on a daily basis, I broke down the large amounts of data into more consumable chunks. This method also allowed me to follow up with participants in a timely manner if I needed to ask for additional details or clarification. To easily identify trends and similarities, I compiled all the entries into a single spreadsheet, where I would add relevant tags for each response.
It only took a few days before patterns began emerging from the data. Almost all the backup admins started off with a morning routine that involved checking on the backups that ran overnight and triaging issues and requests from their colleagues. What they were met with in the morning could dictate how they spent the rest of their day. If there weren’t any pressing issues, some of their work would range from maintenance (like performing upgrades to software or hardware) to exploratory learning (like testing out a new feature). If there was an immediate issue to address, then their day was dedicated to investigation and remediation.
In some cases, I heard from users describing how they tackled a problem over several days, attempting to solve it based on previous experience or counting on their team members or Rubrik support to help troubleshoot. It was clear that there wasn’t enough guidance in the product itself to help them resolve the issue. While we have heard users report bugs in interview sessions before, it was a different perspective to read about it in (nearly) real time, as they worked through issues over multiple days.
The diary study also provided a zoomed-out perspective, allowing us to understand how Rubrik fits into the macrocosm of tools backup admins use to protect their company’s data. Across several customers, I saw how Rubrik was just one part of their extensive IT ecosystem, yet it was vital to supporting ongoing business. I learned just how busy and complicated a Backup Admin’s job might be, interfacing with different team members and hopping from one tool to the next, not to mention putting out the occasional fire outside of work hours. I heard from multiple participants that what they appreciated the most about Rubrik was how it distilled difficult tasks into simple and manageable ones. Especially for backup admins who are responsible for multiple workloads and are often jumping from one tool to another, having a user-friendly and reliable tool saved them time and gave them peace of mind. As one user put it,
“That’s what I like about Rubrik — you don’t need to spend too much time on it.”
While this wasn’t an insight specific to SLAs or backups, it was a golden nugget that highlighted what customers find valuable about our product compared to other backup products — saving them time and effort.
Data Synthesis and Deliverables
After the closing calls with the participants, it was time to fully analyze the assortment of data I had gathered over the previous weeks. Working in the tool FigJam, I clustered the notes into Jobs-to-be-Done (with the personas who performed these jobs) as well as feature-specific findings and IT tools.
From there, I began outlining a journey map to detail the story of our backup admin user, highlighting the high and lows in particular. The journey map included the following sections: Touchpoints, Actions, Quotes, Experiences, Pain Points, and Opportunities. Some of these areas warranted more details, so I also created supplementary materials, such as a persona card and an example of a typical day-in-the-life of a backup admin. The study also revealed users’ interactions with Rubrik beyond just SLAs and backups, including insights into areas of the project such as Reports and Errors, so I included separate sections with detailed findings in those product areas as well.
After creating these detailed deliverables in Figma, I put together a slide deck that I presented to the immediate stakeholders as well as during a design team sync and a product and design all-hands meeting. For these share-outs, I needed to distill a large amount of data into a format that was easy to consume and highlighted the most significant insights. It was crucial to convey the most important takeaways in a visual and engaging way. To bring the user to life on screen, I embedded audio clips of the participants describing points of friction along the journey, which were taken from their audio entries or closing calls. Hearing the voice of the users (literally) helped build customer empathy among an audience who don’t often interact with them directly. In the months following the share-out, I continued to receive engagement and inquiries from product management stakeholders about the project, signaling that this research study will continue to serve as a foundational research artifact at Rubrik.
What I Learned
Conducting the Day 2 SLAs and Backup research was a unique learning opportunity not just for me but also our users. At the end of the diary study, participants remarked on how the experience gave them the chance to reflect upon their day and better understand how they were spending their time at work. They also appreciated the opportunity to contribute to a research study where they really felt heard and valued. One participant noted in their final entry:
“Thank you for your time and engagement, it’s always appreciated. It’s nice to feel like we play a part in changing things for the better”.
Ultimately, this diary study proved to be a great way to get a glimpse into our users’ day-to-day lives. The two major benefits I experienced as a result of this project were 1) understanding our users on a deeper level and making them “real” to our stakeholders, 2) stepping outside of the box of a typical time-constrained interview to gain a wider perspective of the role Rubrik plays in our users’ lives. If you work at a B2B company, it can be tricky to know who your users are and what motivates them since their employer is the purchaser, but this makes it all the more important to conduct research early to truly understand the people who will actually be using your platform. With this in mind, a diary study is a cost- and time-effective tool that will help set the foundations for building a product for the right users.
If you found this article helpful, don’t forget to follow Rubrik Design Page for more design insights!
