Maintaining design systems with proper user research

By the beginning of the last year, Elisa already had a mature design system which included several products such as shared Sketch UI kit, publicly available library of CSS components and several libraries of React components. These products have ongoing living cycles supported by shared practices and thanks to shared tools. Open design and development community in Elisa made it possible to implement “open source within the organization”, and this was the model for the design system.

By the beginning of the year, Elisa design system was a software product developed by a group of developers using developer-driven model. This means that the development team itself plays a role of product owner when deciding what the potential users would need. Such approach made us contribute with a lot of enthusiasm because we emotionally consider ourselves as owners of the product, but sometimes we were not sure that we were doing right things.

We have a successful product but also faced the need to improve it. At the same time, we could not simply copy approaches of other companies. Even though there is plenty of design systems by other companies, they all address certain business needs which most likely differ from ours. Our path should be designed specifically for our case. But how exactly?

Also, we must admit that previously we were mostly focused on “how we are going to do the things”. In this year, we changed our focus and decided to find an answer to “what we are going to do?” question. For that, we decided to act as designers and go through the classic design process with gathering the requirements, formalizing them, ideating over the possible solution and future implementing.

Image source: http://4smartminds.com/en/about-us/forschung/

There is a general tendency that design nowadays requires more intimate understanding of users. This becomes even more crucial when developing a product used in small communities. For inner organization products, such as design systems, the users are employees of this organization whose amount is limited. In our case, the amount of users is between 50 and 100 people. On the other hand, these people know each other and the creators of the solution. This makes their expectations higher and so results in the need to design the solution the most accurate up to the users’ needs.

For the data collection phase, we applied the following methods:

1. Automatic collection of quantitative data;
2. Questionnaires for quantitative and qualitative data collection;
3. Qualitative data collection with user interviews;
4. Participatory workshops for qualitative data collection;
5. Triangulation or using multiple methods for the research.

Automatic collection of quantitative data

The nature of software product makes possible automatically collect a lot of data about users’ interaction with the product. At the same time, if the product is for an organization inner use, it makes easy to get users’ permission for collecting all the needed data. No public relations effects need to be considered as it might happen in the case of public product. This helped the us get comprehensive data about which parts of the provided product were used the most frequently by different teams and what were technical difficulties in such usage. Out of that, we could get a lot of data about which team uses specific components and what is the volume of such a usage.

Example of gathered data about the components’ volume of usage.

Questionnaires for quantitative and qualitative data collection

We began our direct interaction with the users by asking them to fill a questionnaire. The gathered questions were designed by the whole design system team together. Questions were kept simple and limited by the amount. Mostly, the users were asked about quantitative metrics which could not be gotten automatically. Such as “How long it takes to adopt the new version of the product?”. The last question was for open feedback and enabled users to write text answers. Even though it was optional, all the users provided their answers which were mostly qualitative data. When analyzing these answers, the we evidenced the value of qualitative data and decided to proceed with user interviews in order to get more insights.

Qualitative data collection with user interviews

To get such data, we organized separate meetings with every team using their product. There was one meeting per team where most of the research group and most of the user group were presented at the same time and had a discussion. These discussions did not have a strict agenda and were made in a flexible manner with the research team facilitating the talk. There was a list of mandatory questions to ask but the order was not specified.

Interviewees were asked for both quantitative and qualitative data with the focus onto the second one, so the interviews were semi-structured and they are known as helping people to provide richer descriptions. At the same time, less structured interviews seem to be less informal and more comfortable for the interviewees.

Participatory methods for qualitative data collection

After the interviews, we were amazed how much valuable data we managed to gather from just talking to the people. We were looking to increase the involvement of the users even more and so decided to run a workshop.

We asked Kauri Salonen, a well known specialist in the field, to facilitate such a workshop. The main topic there was to outline the model of using the product in the daily routine. The participants were invited to provide their view onto the process at three dimensions:

1. How the process happens, what are its stages;
2. Who are the actors at each of the stages;
3. What are the product features and complementary tools these actors use to run the process.

The workshop participants were offered a blank canvas to fill in, and the expected results were the outlined process within this canvas.

Schema of the canvas offered to the workshop participants.

For more accurate results, the workshop was held two times, separately for each of two groups of users. The first group consisted of employees from different teams which are not only the users of the product but also contribute to it as developers. The second group included employees from various teams which do not participate in the developing of the product but are its active users. The research group expected different outcomes from these two workshop sessions which could help to articulate their own bias. As expected, the results demonstrated the significant gap between those who are co-authors of the product and those who are not.

In particular, this gave us a lot of understanding what we could improve in managing our design system so that information about its processes as options of usage would be available for all the potential users. Applying proper research methods, we learned what precisely should get our focus and managed to improve these specific things during 2018. Design thinking was promising us better understanding of the user needs and so it worked.