The Porsche Usability Testing Marketplace: Creating Products that People Love
Product Design relies heavily on user feedback. The UX Design Ops team at Porsche has created a marketplace to test products and features in a short time.
In recent years, the digital product organization at Porsche has changed rapidly: Agile product teams within the Porsche ecosystem work very closely with user experience designers, together as one team. The UXDs (User Experience Designers) in return, exchange information in guild meetings about the topics they and their team are currently working on. They get feedback and input from their colleagues. Discuss, think and help each other to find optimal solutions and products for our users.
But we wanted to make it possible that not only UXDs talk about user experience, but also other roles. For this, it is important to us that we see ourselves as a whole team and integrate everyone into the development process. And we want to enable all team members to understand how users interact with their product. No matter if you are UXD, Product Owner or developer.
Creating a marketplace for product testing
After some research and the discussion about interesting articles and tips, we thought about creating an open marketplace for our digital products. Because a marketplace is a place that is open to everyone. Everyone is invited to discover and get to know the variety of products. Environments like this encourage people to have conversations in a casual way and taking time for each other. That’s exactly is the spirit we wanted to capture.
Since our teams already have a lot of regular meetings in their calendar, we want to take as little of their time as possible, but still have a high level of participation. We can achieve this by presenting them exciting content, requiring only short time slots to participate and allowing them to learn something quickly and easily.
We have given our concept the name “Usability Testing Marketplace”. At this event, various product teams from the digital Porsche ecosystem are invited to bring their scribbles, wireframes, prototypes, implementations etc. and test them in short flexible sessions with test persons. This way the products can be presented in detail and a qualitative exchange with test persons and potential users takes place. And all this in a pleasant atmosphere, without appointments, open for all who are interested in user experience in the context of the Porsche brand.
How does our “Usability Testing Marketplace” look like?
Our marketplace consists of several small booths, which are set up in a large and easily accessible location. At these booths, you can get to know and test different products from the digital Porsche ecosystem within a certain period of time, e.g. over 4 hours on one day a week. This means that the teams realize suitable user testing for their specific scribbles, prototypes etc. and offer them at their booths. The test persons can stroll through the marketplace and freely choose which product they want to test. Also methodically, there are a lot of offers, for example, haptic card sorting, preference tests, user interviews, UX writing variants and navigation tests.
Why is it worth it?
Behind the planning, realization and follow-up of the Usability Testing Marketplace, there is a lot of work in the context of general and testing organization. Any feedback, no matter how small, can be helpful for the teams and their product development. Optimizations can be derived from the results of the testing and individually planned for further development of ideas or concrete technical implementation. So, this is why we want to continue doing it.
It is also important for us to talk about the insights and results of the Marketplace and share them in our chapter and guild meetings. After all, many of the products are closely connected to each other and should appear as one unit to the customer. Therefore, changes in one place often have an impact on another. Communication before, during and after the Marketplace is also crucial in order to reach many stakeholders from the ecosystem, to invite them and to gain their attention for the topic user experience. The knowledge gained from the testing should then be used to support decisions on product development and prioritization of tasks.
Don’t get it wrong: the marketplace is primarily used to reveal tendencies, wow moments and pain points to determine whether this solution is going in the right direction. Very large and detailed tests can and must follow in some cases. Fewer assumptions, more valid data is key.
Special times require special measures: The marketplace as a remote version
Initially, we planned to host the Usability Testing Marketplace once a quarter. We were already preparing the second Marketplace when the consequences of coronavirus hit us, and home office became obligatory. Accordingly, the marketplace had to be arranged in a way that it can be done online.
Luckily, we already had many tools and channels in our repertoire, so that we could also work collaboratively with each other when we were not in the office (e.g. Microsoft Teams, Miro, Confluence). Many methods of user testing can also be performed online and shared via links (e.g. with UsabilityHub, Maze, Qualtrics, Optimal Workshop). Thus, the selection of test objects, the choice of methodology, the creation of tests and the setup of additional surveys could be easily done remotely. The coordination and support of the participating teams were also possible.
The biggest challenge was to acquire enough test persons for the event. Unfortunately, we couldn’t avoid scheduling the tests so that the teams could meet with the test persons. We proceeded according to the principle “first-come, first-serve”: Participating teams described their test objects briefly and appealingly and provided possible dates for the online test session via Google Docs. Test persons could scroll through the offer accordingly and register for as many sessions as desired. Subsequently, the test persons received individual invitations with necessary links and information for the test. The sessions were then mostly designed in such a way that the test could be carried out via video conference, screen sharing and online surveys and exchange between team and respondent could take place.
A short look ahead
We also collect feedback on an organizational level and consider how we can optimize the Usability Testing Marketplace and how we can shape it in the future. No matter whether it is a physical or an online marketplace. Our goal is to establish this type of exchange, cooperation, transparency and user testing in a way that it becomes an integral part of the product development process. It should be fun, inspiring and open and sharpen the view on user experience. Through all stakeholder levels, with a positive effect on our customers.
Diana Breternitz is Senior UX Consultant at MHP — A Porsche Company.
Marcel Bertram (left) and Daniel Diener (right) are UX Specialist Marketing Communications at Porsche AG and work as UX Design Operations Team to orchestrate and optimize UX Design within the digital product organization.
About this publication: Where innovation meets tradition. There’s more to Porsche than sports cars — we’re tackling new challenges, develop digital products and think digital with a focus on the customer. On our Medium blog, we tell these stories. It’s about our #nextvisions, smart technologies and the people that drive our digital journey. Please follow us on Twitter (Porsche Digital, Next Visions), Instagram (Porsche Digital, Next Visions, Porsche Newsroom) and LinkedIn (Porsche AG, Porsche Digital) for more.
