testaffairs — streamlined UX research & evaluation

How do you get real insights and emotional feedback from 500.000 people regarding the user experience of transportation in a city like Munich? Sounds difficult? designaffairs has created the basic requirements and developed a unique app to achieve this difficult task: testaffairs. This article will detail how easy it is to set up, conduct, and evaluate a user experience test with testaffairs, while showcasing the benefits of using this app for ux evaluations.

“An urban commute is an end-to end user journey from mobile apps or public architectures to buses, bikes and sidewalks”

Imagine you are an urban planner and your current project is in Munich, “the smart city”. Since Munich is growing fast, transportation is particularly interesting for the future of a smart city. To create a prospective smart-city concept, it is important to identify important factors for users of that transportation. Therefore, you want to know what the users like and dislike about the current transport in Munich.

1. Problem scenario in UX evaluation

So, let’s face the facts: To get representative feedback and insights for your smart city concept, you need a large sample size to participate in your study. Note: Munich natives are not the only users relevant for your concept. Since Munich is such a popular city for tourism, visitors in Munich are an important factor to consider, as well as their opinions about the city. Thus, the data collected must represent a wide range of users. For example, some people always drive their own car, some people might not even own a car but use car-sharing and public transportation. This means that your transportation solution is heavily contextual and must address the needs of every user.

“How do you evaluate UX across multiple digital & physical platforms?”

User-centered design developments like this pose many challenges: you must recruit as many users as possible, define what observations and insights you need, interview and observe them while they are interacting with the product or service (maybe even a long period of time), conduct these tests in an appropriate environment, all while ensuring that the feedback you receive is honest and trustworthy. These findings must then be analyzed and synthesized further to be used for further development.

Figure 1: Evaluating the user experience in the train with the tetsafffairs app.
“How do you design the UX of UX research?”

For the smart city concept, this means observing users while they are using public transportation for work, interviewing people when they reserve a carsharing car at home or while they are using a ticket machine in a hurry. It’s hard to imagine that these users would be willing to give authentic feedback or be observed while they are struggling already. This seems even less likely for the tourists traveling abroad. User feedback like this is complex, difficult, and time-consuming to collect. In other words, it’s expensive.

2. testaffairs solution

With our UX competency gained through 20 years of experience of developing holistic product solutions, we created a tool that helps to overcome these UX research obstacles. testaffairs is an app that allows users to test & evaluate products or services by themselves with their own device wherever and whenever they want. It is a remote evaluation tool — like a digital diary. That means no need for a testing room and several people conducting the test. Additionally, users don’t need to visit you or use your product in an unfamiliar context. With the testaffairs app, it is possible to examine contexts you would never be able to observe with traditional evaluation methods, since the app follows wherever the user goes. The user can give feedback at any time. Once the data is collected, testaffairs reduces the complexity of data evalution by automatically quantifying the qualitative data from a wide range of users.

Figure 2.a: The testaffairs app runs on mobile devices with iOS and Android.

The app is suitable for any kind of product or service and is highly scalable regarding the number of users and the time span of the evaluation without a lot of extra work. As a strategic design agency, we strive for holistic user experience. This is not only true for users who evaluate products with the testaffairs app, but also for owners of the tested product. If an appropriate IT infrastructure is required, testaffairs not only offers better insights from users to improve products and user experience, but it also automates data collection and analysis to increase efficiency, and decrease cost. testaffairs is a comprehensive tool that encompasses three important aspects of product development: efficient research preparation, real-time data collection, and interactive data evaluation.

“To innovate holistic experience, you need reliable holistic data”
Figure 2.b: The three important steps of evaluating a product or service that testaffairs covers.

3. Easy test setup

With testaffairs, it’s easy for you to create new evaluation projects. With the web-based application, you can select or define which categories should be evaluated by the users. In the city example, categories like public transportation, cycle tracks, traffic, etc. are relevant. Then you can define which persons should participate in your test and give them access to the app. That’s all. You are ready to conduct your test!

Figure 3.a: With a quick glance you get a nice and good overview of your projects.
“Seamless scaling from qualitative to quantitative data is key to agile product development and testing across complex systems”
Figure 3.b: Detailview of one project.

4. Contextual test execution

Now, chosen users for the study can just open the project in the testaffairs app. There is no need for the users to conduct a usability test in an unfamiliar environment, being observed, and asked personal questions. testaffairs makes it possible to give feedback every time and everywhere the users choose, allowing them to actively shape the development of the product or service.

“Effortless input, similar to social media & communication, makes testaffairs a joyful experience for testers”

When users interacting with your product or service are experiencing something special, all they need to do is open the testaffairs app on their mobile device, and choose the respective category to evaluate. testaffairs offers flexible and fast data inputs like photos, voice recognition, and text to evaluate topics such as material quality, usability, etc.

testaffairs feedback is simple and easy. Users can give feedback when the train to work is late again, they can positively highlight a well-designed map for Chinese speakers in pedestrian zones, or complain about the availability of bike-sharing services.

“Staying in touch with our testers is essential to generating authentic insights”
Figure 4: Evaluating a product is easy and joyful for the users and only takes a few moments.

Everything inside testaffairs is anonymous, so you get direct and honest input from the users. You get empathic insights into user behaviours and emotions, especially in intimate and inaccessible contexts. This is especially precious for sensible products where users don’t want to be observed during the interaction (e.g. medical products).

5. Interactive test evaluation

Meanwhile, the data is automatically collected real-time in the online database. You can open the testaffairs back-end in your browser and get figures and diagrams of the user feedback automatically. With the interactive filters, you can choose the most important data for you and generate comprehensive, compelling reports for your management or your team. For example, let’s say you are interested in the feedback of women older than 30, and their opinion about bike sharing prices in the first two weeks of the test period? Just click all the filters that apply to create convincing presentations of data through authentic user statements.

“We leverage the speed of AI & neural networks to identify remarkable insights from visual, verbal & geographical data — and flexibly readjust our research”
Figure 5.a: Monitor evaluation results in real-time with automate analysis, diagrams and visualisations.

Since testaffairs does a lot of the analysis work for you, there is no need for intensive data preparation or manually creating complex diagrams and figures. The app can, for example, cluster user feedback by topic. Consequently, the otherwise time-consuming and elaborate process of finding patterns in feedback and analysing specific content can be reduced to a minimum. This means getting rich and valuable feedback directly from your users, saving time and cost, while getting automatized, pre-structured findings, and direct mapping of context, user comments, and metrics.

Figure 5.b: Monitor evaluation results in real-time with automate analysis, diagrams and visualisations.

6. Further case examples

testaffairs is suitable for any kind of product or service. What we described can easily be applied to other products: Just think of sensitive products like the smart bathroom of the future. To get good insights you would need to accompany the user during their daily routine in the morning in the bathroom. It doesn’t take much imagination to realize that this would be a very uncomfortable situation for both users and observers. But with the remote evaluating tool, testaffairs, this isn’t a problem anymore. Another almost inaccessible context, for example, is an alarm clock. To observe the user in his real environment you would need to be at the user’s home during times when really no one likes to be watched.

If this sounds like the perfect tool for your project — don’t hesitate and feel free to contact us at designaffairs!

Editors: Johannes Huber, Evelyn Schweiger, Muya Hong, Guoshu Qin, Max Münster, Vivian Qiu & Kim Moquenco