Iterating the P4 prototype with on-site feedback
Feedback can be daunting for users to give and for project team to parse, here's how we'll do it.
With a site like greenpeace.org, generic, unstructured feedback may become quite a lot of data! A good way to minimize this is to propose multiple, segmented, contextual calls for feedback right where it makes sense in the user’s experience, instead of a (usually long) form to asking all we want to know, which is most likely to scare users away.
Exactly like the prototype, feedback gathering is a continuously evolving process, so our strategies and surveys will change and evolve as Planet 4 improves.
Whichever option you choose, the feedback tool will only track from which city/country you will submit, the type of device (mobile, tablet or laptop) and its Operating System, and on which type of browser (Firefox, Chrome, etc) is being used. Aside from that, all feedback is completely anonymous
To gather actionable, on-site feedback, we will be using Usabilla, which tools’ suite allows users to provide feedback directly on page-level. Usabilla’s name should not be confused with usability testing, which involves a determined number of users performing a set of controlled and predefined tasks. The first does not substitute the second, but they are good complementary methods.
Feedback button (passive feedback)
Also referred to as passive feedback, a feedback button is a sticky (or floating) button on the screen. Using this tool, visitors can either point out specific areas in a page to give feedback to by highlighting and screen-shooting it (specific feedback) or give generic feedback to the entire page.
Goals
- Enable users to provide qualitative feedback on each page of the P4 website.
- Facilitate bug reporting
- Integrate user stories and facilitate population of backlog
Implementation
- Visible in all pages
- On M, L and XL screens: sticky button positioned by the scrollbar on the right-hand side, towards the top
- On S screens: sticky button positioned at the bottom of the screen towards the right
Strategy
Short and concise: a single screen with a couple of questions to let users directly reach out anytime. Starting with an emotional rating scale, users can quickly check-in with their overall sentiment and give more candid feedback. Additionally, folks can leave 3 actionable types of feedback (suggestion, compliment or bug), available from a dropdown button. Lastly, a single open-ended text field allows generic comment.
Processing methodology
Organizing feedback into categories right away (suggestion, compliment, bug) makes processing and sorting smoother, allowing the P4 team to focus on consolidation and turn feedback into reports and stories to further explore. Since the last field allows generic text, we may use text-analysis tools to go through data and then draw patterns from what users are telling us.
The next step will be to probe deeper in what folks are saying, by launching specific on site feedback campaigns to validate suggestions/concerns in a qualitative way (this will also help defining which hypothesis to test in future usability studies).
Campaigns (active feedback)
Usabilla refers to “campaigns” as tools to actively ask visitors for feedback. 2 campaigns will be live with the prototype launch: a boost feedback and a survey.
1. Boost feedback
A boost feedback is a slide-out notification originally intended to prompt users to give feedback at a certain point of their browsing experience.
Goals
- Notify users of P4’s “beta” launch condition
- Raise awareness of the feedback button
Implementation
- Slide-out box coming from the right side of the screen
- To be shown only once to each new user, when he/she first opens P4 (unless browser cache is cleared or a different browser is used, it will never bother the user again).
Strategy
The intention is to use the “boost” feature as a notification feature at first (“Hello, welcome to the new site!”), and ask users to navigate further through the site. Essentially, we’re using it as attention-grabber for the call to action.
Content
Welcome to our beta launch!
Our new website is finally here and we would love to hear from you as we fine-tune the details. Give us feedback anytime by using our floating button on the right and help us improve.
2. Evaluation Survey
Survey campaigns in Usabilla can be slide-out, which takes a smaller portion of the screen while still allowing users to browse; and full screen, which interrupts the user flow entirely by dragging their attention to a screen long survey that pops up. For launch, we plan to use the same survey form in both formats: as a slide-out on smaller screens (mobile and tablet) and as a full survey on desktop. We can benefit from a more subtle survey on limited screen estates (S and M) and still take advantage of the standalone link for us to share externally that only a full screen format allows.
Goal
- Collect an overall assessment of P4 (information quality, interface quality and overall satisfaction)
- Gather qualitative feedback to give us an indicator to which areas of the website we need to dig deeper in future usability studies
- Segment the feedback collected based on user type (GP staff, “experienced” non-staff user and “novice” non-staff user)
Implementation
- On S and M screens: slide-out box coming from the right side of the screen, triggered after users having visited 2 pages, so they have experienced enough of the website to be able to evaluate it AND after scrolling to at least 600 pixels from the bottom (this is near the “Happy point’s” position).
- On L and XL screens: centered popup full screen survey, triggered after users having visited 2 pages AND after scrolling to at least 500 pixels from the bottom.
Strategy
Since we’re asking for feedback based on a free exploration, each individual will have unique perceptions based on the viewed pages, consumed content and personal goals. Therefore, the initial survey was designed to get a sense on the overall evaluation of the website without diving into specifics (which will be dealt in the future by contextual based campaigns and usability testing). In other words, the survey will serve as an indicator to what we should put our efforts on the next rounds of feedback collection.
As mentioned, feedback is anonymous. In the survey method, however, there is a further profiling option, asking whether users are Greenpeace staff members or not and (in case of the second) which activities they have been involved with Greenpeace. The reason for this is that we want to make sure that Planet 4 engagement goals are actually met by non-staff or novice users’, and if not, why not and for which goal. Also, asking if users are Greenpeace employees or not will make segmentation easier and custom reporting more accurate.
The main technique to collect overall assessment of the P4 prototype will be to use semantic differential scales, presenting opposite adjectives at either ends of a 1–5 scale.
Users will be able to rate 4 key aspects of the website in a broader sense: content (unappealing/appealing), design (ugly/beautiful), ease of use (hard/easy) and performance (slow/fast).
Should respondents rate any aspect as very low (1 or 2 on a 5-point scale), an open-ended field will appear for users to expand on their frustration or suggest an improvement.
Processing methodology
Based on the data collected with this survey (or campaign), we will be able to map out the prototype’s biggest deficiencies from the lenses of the different type of users (staff/non-staff, engaged/novice), and prioritize an action plan for improvement.
Final considerations
One important point to consider in feedback gathering tools like Usabilla is that, since respondents make the decision about whether or not to complete the survey, unhappy users are more likely to respond than those who are satisfied. Thus we should expect, especially through the Feedback Button form, more critical/negative responses, which is fine since we do want to uncover critical bugs as soon as possible.
We’re launching our prototype this week. Head over to greenpeace.org and let us know what you think!
