Dig Deeper in Planet 4 Prototype Tests: Helping Out with Moderated Testing
How Greenpeace tested new engagement features for the future websites (Part 3 of 4)
👈 Part 1 | 👈 Part 2 | Part 3| Part 4 👉
UX studio is a Budapest based 40-person company passionate about creating enjoyable digital products. Specialized in both UX design and UX research, we don’t focus on one particular industry and are proud of the fact that we have collaborated with a wide range of organizations all over the world.
One of the strongest perks of working for UX studio is our company culture: we care about who we work for. When we were approached by Greenpeace to help them with Planet 4, we did not need to think twice — this was a project we could be proud to take part of!
Greenpeace envisions a Planet 4 platform that will engage people from all over the world in taking actions repeatedly. User Experience lies in the principles that, if you want your product to be used, it needs not only to be useful but also to have a clear meaning.
As presented in previous chapters, the P4 team created two interactive prototypes to validate the concept and feature ideas with user research. We, UX studio, jumped in to support with qualitative research: exploring what type of value the proposed features create for future users, how they create this value, and why.
Turning Requirements Into Action
We started by having several online workshops with the P4 product team to understand what had been done to date, what direction the team wanted to go next, and how the work of UX studio could contribute to the bigger picture of Planet 4.
Thanks to the tremendous work that Greenpeace is doing by documenting all the processes and a great collaborative atmosphere we quickly got up to speed with Planet 4 and its mission.
Once the scoping phase was complete, the team decided to carry out moderated usability testing for both prototypes as a method to collect, first-hand, reactions, and insights from users. The focus of these tests has been (in)validating the concept ideas and identifying any early usability problems. We collected a list of important questions that our study would seek to answer, such as:
- Is there a need and interest among interviewed users for this type of content/tool?
- Do they think this platform will encourage them to take more action in daily life?
- Are users interested in seeing their impact? And in sharing their environmental actions on social media?
- What types of actions are more interesting for users?
- Which features do users highlight from either prototype?
Based on these questions, we worked with the P4 team to prepare a script for the tests, including scenarios and specific tasks that each participant was asked to perform. Both scenarios and tasks were highly dependent on the prototypes’ designs since not all actions and paths were clickable. While working on the script, we started looking for test participants.
Testers Demographics
Finding the right participants to carry on the tests plays a key role in validating a design. Each study should ideally involve a relevant sample of the target audience, aiming for diversity but trying to maintain scope. For this concept study the P4 team decided to look for:
- a rather young audience: Generations Z (<24 years old) and late Y (24 to 32 y.o.),
- with a balanced mix in gender,
- across regions (Americas, Asia, and Europe),
- with a mixed level of engagement with environmental issues: from not engaged yet but would like to get involved in environmental issues (with Greenpeace or other organizations), to already highly engaged, and
- with a mixed level of social media usage.
Our recruitment strategy involved multiple channels, including youth organizations, educational institutions, social media, online communities, and organic search.
Through a screening form to identify the best candidates,14 selected users were then contacted to arrange the tests, 12 of which were carried via video conference, and 2 in person at our Budapest office. In-person tests offered the possibility to record not only the screens but also the physical reactions (e.g. facial expressions and body language) of the users towards the prototypes.
Key Takeaways — Concept
In terms of validation of the Planet 4 concept:
- Participants with high environmental engagement are regularly looking for initiatives and good practices, as well as new channels to share information about these with their communities.
- Interviewees with a lower engagement expressed that they would be interested in receiving information about simple, quick ways to contribute.
- All participants expressed the need to receive concrete, actionable tips to incorporate in their daily lives, rather than vague proposals such as ‘become a volunteer’. Most people reacted with enthusiasm to the daily challenges and highlighted the possibility to visualize and track their progress.
A specific call to action (“Participate in a plastic cleaning event”) proved to be more appealing over generic ones
- The number one constraint mentioned by most interviewees is time; they want to contribute but don’t always find the necessary time in their routine. This means that P4 challenges should include both quick actions to take on in-between daily duties and longer tasks when users have more time to give.
- Participants confirmed that it’s a great idea to start by asking them how much time they are willing to spend, as the prototypes do.
Key Takeaways — Usability
The tests also provided interesting inputs on usability:
- Percentages (%) to track users’ personal progress created confusion and were perceived as not relevant. Some users said that seeing 0% when first logging in made them feel guilty, while others questioned the fact that anyone could reach 100% after doing a number of challenges (“and what does 100% mean in this context?”).
- “Being sustainable” is not about reaching a certain % or score, but about constancy — creating a habit. Like many running or language learning apps, that use long-term statistics in a more or less visual manner, incorporating sustainable challenges and practices is more like a personal journey.
- Success and failure of the signup conversion is the moment in which users are asked to create an account. Participants were more likely to create an account in P4 after having explored several features and being introduced to specific benefits of doing so, such as being able to save their progress.
- Our prototypes had been designed with a focus on showcasing new features. Some interviewees described them as “a bit of a labyrinth” because, after going through a few of the given tasks, they could not tell where they were on the platform. These comments appeared recurrently at the end of the tests when we asked what they liked and did not like. We take this as a learning point: navigation cues, even at the prototyping stage, are important for users.
Suggested Actions & Next Steps
All the findings were stored on a database and presented to the P4 team, together with a collection of suggested actions and next steps that included:
- exploring new ways to measure impact,
- review the gamification mechanics (P4 can go much deeper than badges and points!)
- ideas on how to increase retention of users over time — the number 1 killer of many Apps and digital platforms.
Combining these qualitative findings with the quantitative data obtained from the unmoderated tests (see previous chapter) is key to make decisions on how to further develop Planet 4 based on facts and not assumptions.
That’s what the next chapter will be about: a consolidation of the results from the two testing methodologies.
At UX studio, we never take on design projects without research, because user insights guarantee quality and usability. The opportunity to use our expertise to build a platform with such potential for positive impact was very rewarding.
Getting to know young activists from around the world filled us up with optimism, and working with Greenpeace was inspiring from the very first minute because of their enthusiasm and the ‘can-do’ attitude.
Kudos to the Planet 4 team for this approach!
At UX studio we believe that our knowledge and expertise is something worth sharing with others. That’s why we also carry out educational activities, such as Product Design courses (both in English and Hungarian) and UX training for organizations. We also published a book and regularly write a blog, which is read by tens of thousands of readers every month.
