My journey of working across divisions at Microsoft
If you ever worked with cross-functional teams in a large enterprise and found it challenging to coordinate efforts and create consensus, you will find this article interesting.
My first study at my new team
I work as a UX researcher at Microsoft Developer Division, which focuses on developer tools and services. Back in Jan 2019, I was still new in the division and my first study was about evaluating a documentation article that explained users how to create a function and deploy it to Azure using Azure Command-line Interface (CLI).
One of the hypotheses the stakeholders had was that new developers could easily follow this article to create the function in 20 mins. We did a usability study with 8 participants, and we observed that it took much longer! On average, it took about 45 mins to create the function in Azure.
Problems observed in the study
One of the main learnings from the usability study was that developers expected to quickly scan the article and follow the steps to create the function. However, we observed the following issues:
1) Unclear steps: It was hard to clearly distinguish the steps required to complete the task.
2) Too much text: It was hard to distinguish text that was required to complete the task from supplementary information (such as troubleshooting, concept definition or future reference).
3) Lack of visual distinction: There was no clear visual distinction of key information, such as code that a developer needed to input in the system, and the output that a developer could expect to see when the code ran successfully.
A misstep in communicating study results
I had tried to involve all parties as much as I could prior to the study, but I only got traction with some of the stakeholders. One of the teams involved was not able to participate and observe the studies. I assumed they were not interested to be involved any further. So, we wrapped up the study and presented the results within our division. In a final email that communicated the results, we included this team to keep them informed. However, they were not happy with the results! They felt criticized and underappreciated! I was surprised, I thought I did everything I was supposed to do…
Upon reflecting on the matter, I realized that although I had reached out and invited them to the planning meetings and study sessions, I hadn’t taken the time to build a relationship with them since I was new in my team. I also didn’t know that their division was understaffed and were already receiving so many requests. As I learned more, I understood why they didn’t have the time to participate in the study in the first place. It probably got lost among other high priority tasks!
Resolving the communication issue
I decided to take a step back, talk to their leadership, explain my intent, and develop a mutual agreement for our collaboration. I committed to inform and involve them on any upcoming research on developer documentation, and they committed to participate in at least two sessions of any upcoming usability studies to see and hear customers firsthand as they use our documentation articles. I could finally see light at the end of the tunnel!
- Identifying the problem is a key first step to improving product experience, but when the problem involves a different team or division you need to be careful not to come across as pointing fingers.
- It is essential that you form a partnership with all the parties involved and establish a common goal, so no one feels being attacked or unappreciated.
I must admit that I initially just thought they were reacting to the results because they felt their work was being criticized, but then once we established a more effective communication channel, I realized they were asking a good question: where those issues specific to the article or were they more common across developer documentation?
They informed me that documentation involves both content and platform. Any recommendation for change, had to be implemented across the documentation platform and required extensive engineering effort, because the platform had limited capabilities. The team owning the platform was also very much under-resourced. So, we needed more evidence to prove this was a problem worth solving.
The conversation with the content team was very enlightening. They were the experts on documentation, and they knew best the challenges of making changes. This was a big aha-moment for me and the rest of people in our division! We didn’t know there are so many pieces of the puzzle, and we were thinking initially that it should be relatively easy to change things!
- It is critical to take advantage of the expertise of your stakeholders to make your proposal for change more realistic and actionable.
Gathering more evidence for the problems
To give more confidence to stakeholders that these problems are common across developer documentation and worth solving, I started a synthesis to identify systematic issues related to developer documentation across several product areas in Microsoft Developer Division. The synthesis helped me build consensus among the various stakeholders on the customer problem.
- Is it really a problem? How big of a problem is it? Provide evidence to shed light on these questions.
I talked to researchers and product managers who had previously done research involving developer documentation, reviewed their findings and identified common patterns. Besides content-specific issues, such as outdated or missing information, I found several recurring problems that a designer familiar with a developer audience could help us solve: 1) Discoverability and 2) Information organization.
Building partnerships and allyships
I brought in an extremely talented designer, Marisa Parker, who became my partner, advocate, and ally throughout this project. She had empathy for the problems identified and was motivated to help create best-in-class experience for our customers who need documentation to use our products.
I also had allies from product management leadership in our division and Content Experiences leadership outside of our division, who advocated for the importance of these problems and basically sponsored the effort of tackling them. We all agreed these were relevant problems to solve.
- Building consensus is important, especially when working across divisions. It ensures everyone is onboard and the investment is worth making.
- Sometimes you need both bottom-up and top-down support to build consensus across multiple teams or divisions.
Developing solutions
My partnership with Marisa was crucial. As a researcher, I could convince everyone that we have discoverability and information organization problems, but I needed design expertise to propose appropriate solutions. Marisa carefully studied the problems identified and designed specific solutions to address them. She then created a prototype that demonstrated a new approach to organizing a documentation article for developers.
- Proving there was a problem wasn’t enough. You will need the right expertise to craft solutions and create prototypes to demonstrate how things can be different.
Experimenting with the proposed solutions
After all stakeholders got to see the prototype and gave a thumbs-up to the new design, we repeated the usability study this time with Marisa’ prototype with 8 participants. We reduced average completion time by 15 mins! Participants were happier, and less errors happened along the way. Below you can see some quotes from the participants in the study.
“When I use other docs like AWS, it’s a bit harder to read, this is clear on what I am supposed to do and what is expected of you.”
“Very simplified could make one with no experience to use Functions!”
“Easy to follow.. Straight forward.. Learning on the fly.. No hiccup except the invalid name and recovered quickly..”
This was great evidence that the proposed design changes provided a better experience for developers. We demonstrated that UX is as important for documentation as it is for our products.
- Experiment your prototyped solutions with potential customers and iterate on them to get desired results.
Operationalizing evidence-based solutions
We divided the changes into two groups: 1) content changes, and 2) platform changes requiring engineering work. We held a series of extensive workshops with our content writers on the best practices that proved to improve the customer experience, and how to apply those practices to their articles. We also got buy-in from the engineering team to allocate resources for making the changes in the platform by the end of the year.
- Work with stakeholders to operationalize the solutions you proved to be effective; you may need to fine tune things a bit. This will help the adoption of the solution.
- Follow through with the various teams and divisions to get commitment for implementing the solutions.
Culture change
This was overall, a slower process than running a typical usability study and following through to improve a product, because it involved multiple teams and divisions with slightly different cultures. Changing a culture to be more customer-driven, open to experimentation and implementing new ideas, takes a bit more time and effort. Providing data, facts, and evidence alone was not enough. We needed partnerships and allyships to build trust, pitch new ideas and cultivate new beliefs.
- Changing culture takes time, but it is powerful and inspires promising actions.
Today we involve stakeholders from across all relevant divisions more closely, especially for research and design studies. Our stakeholders are more eager to take a customer-drive approach involving experimentation to assess and improve developer documentation.
The common goal that brought us all together, was creating best-in-class experiences for our customers. We just needed to find a way to achieve that goal together. This required empathy and growth mindset among all divisions involved and I think we all learned how to achieve it together.
- If you are asking for change, take a moment to empathize and understand why the change hasn’t been made already or how challenging would it be to make that change.
- If you are on the side of receiving a request for change, be open to it and assess its potential value. It might be an opportunity to be part of an impactful initiative.
Acknowledgements
I couldn’t have gone through this journey without the 20+ people across Microsoft Developer Division and Content Experiences Division that were involved in this project end-to-end. I specifically wanted to acknowledge a few key partners:
· Marisa Parker for her leadership, creativity, and dedication in creating solutions.
· Dan Taylor and Joe Binder for advocating of the problem and creating leadership awareness.
· Barbara Kess and Dina Berry for their support in advocating for the problem and their dedication to experimenting and implementing solutions.
And many more colleagues who helped us all learn and grow throughout this journey…
Last but not least, I wanted to thank my UX Research managers at Microsoft Developer Division Kelly Krout and Monty Hammontree for encouraging me to write this article and my fellow UX Researchers Travis Lowdermilk, JP Carrascal, and Jake Freiberg, for taking the time to review this article and provide feedback.
