Better Internal Product Feedback — The Survey is Dead, Long Live the Survey

Most companies spend the majority of their UX-allocated time thinking about how their clients interact with them, either through a specific product or through the company’s website, mobile app, or other digital touch-points. Which makes sense, these touch-points either directly or indirectly drive revenue. And yet the quality of internal-only software UX can also make a surprising impact on a company’s bottom line, mostly because:

1. Employees don’t like dealing with bad software on a daily basis, regardless of whether they’re being paid to use it. Depending on how many hours per day they spend in your product, a poor experience can lead directly into higher turnover, a.k.a. more domain knowledge flying out the door and more time dumped by management into hiring + training replacements.
2. Poor user experience and low productivity are often correlated. System slowness, frequent crashes, annoying bugs, and poor fit between what needs to be done and how users accomplish it means your great people accomplish less than what they’re capable of, regardless of their efforts.

With that in mind, when it comes to creating great internal software products the good news is that most software teams use an agile methodology, which comes with the distinct benefits of iterative development and the replacement of projects with products. With these a product can gradually evolve from a first version “best guess” (which is almost never perfect, and often barely resembles the final form of a product that evolved well) into something specifically tailored to users’ needs.

The bad news is that the single most useful tool available to the navigators guiding this agile-driven progression is timely, effective user feedback, something most companies don’t facilitate or gather well when it’s about their internal-facing software. Why is that, and what can internal-facing software teams do differently to gather more effective feedback that they can convert into software that enables and delights employees?

Challenges with Emotion-Driven Feedback

Feedback’s either objective (in the case of page load times, task duration, or number of crashes per day) or subjective (in the case of overall user experience and the net promoter score). So long as your collecting and displaying a sufficiently sized and diversified sample correctly, objective feedback itself isn’t subject to bias. The data is the data.

Subjective feedback on the other hand is mostly emotion driven, which often leads to significant bias challenges in the feedback itself. For one, many people aren’t great at separating their feelings about the product itself from other influences affecting their current emotional state. If someone’s car broke down the same day that you ask them about their product experience, you’re a lot more likely to get negative feedback than if you ask them on the same day that they receive a promotion. Barring 1984-style surveillance, this bias can be nearly impossible to untangle from their “average” feelings on the product.

There’s also the question of true scope, a.k.a. how much of our user-base feels the same way as this feedback-giver? One-off messages and bug reports from clients tend to necessitate broader feedback initiatives to see if others feel the same way, while even quantified subjective feedback like user ratings begs the question of who’s actually represented in this feedback. Is it just power users, die-hard company fans, or people who’ve experienced strange bugs that are providing extreme (positive or negative) feedback? And if so, how many users fall into less-vocal groups that feel differently? Baring significant time and investment, it can be tough to answer these questions with any form of accuracy, but answered they must be to get a full picture of how your product’s being experienced.

Internal Software Feedback’s Additional Hurdles

On top of the subjectivity and bias challenges found in the feedback for any product, internal-only product teams face a few additional hurdles, including:

1. Fewer Users — Successful external-facing products usually have more users than even the most broadly-used internal-only products. Therefore it can be tough to get a sufficient sample-size of feedback on internal-only software. A 1% response rate on a survey that targets two million customers yields 20,000 data points to draw conclusions from, while a significantly higher 10% response rate targeting two thousand employees only yields 200 data points, which may or may not be enough to dive into details on what areas of the product can be improved, and in which way.
2. No Customer Support Team — Most companies producing externally-facing software have anywhere from a person to a department focused on customer support, that can place significant focus on gathering and making sense of user feedback. This doesn’t usually exist for internal-only software. Instead the product owner/manager’s often responsible for gathering and organizing feedback on their software, on top of their other visioning, direction setting, communication, and change management duties.This product owner usually doesn’t have enough time to gather good feedback that’s appropriately sampled and scrubbed of bias, meaning the downstream decisions they make are often based on incomplete and/or inaccurate data.
3. Fewer Tools — Social media sites like Facebook and Twitter allow users of externally-facing software to shout their feedback to the world, often while directly tagging the company’s customer support team. By tracking mentions and general posting trends, companies can get a surprisingly robust picture of what people like and don’t like about their software. These sites are usually devoid of internal product mentions (unless the product’s legendarily bad and/or the person posting fears no pink slip), meaning an increasingly useful tool in the broader feedback-tracking world is practically useless to teams building internal products.
4. Potential for Apathy — Regardless of the arguments above for why internal software quality matters, many development teams, users, and even management place fewer expectations on the quality of internal software. If it’s not customer facing, it’s generally considered to be an operational expense that the team has to build, users have to use, and the company has to pay for, instead of a tool capable of yielding greater revenues (ex. by displaying a full customer profile with next-step recommendations to salespeople when one of their clients call in) or reducing costs (through automation or producing the same result with fewer clicks). Less focus is placed on gathering and adapting product strategy based on effective feedback as a result.

The Golden Hammer

Software developers love to call something a “golden hammer” when it’s a tool their peer (unwisely) uses to solve practically any problem, regardless of whether it’s the right fit. The survey is often an internal product team’s golden feedback hammer. They’re easy to make, distribute to large swaths of users, and aggregate results from. All the product owner has to do is slap a couple questions together (1–5 rating, what do you like, what do you not like, etc.), send it to a rolodex of email groups, and wait for their survey tool of choice to receive and aggregate responses.

But due to the sheer number of surveys that most employees are subjected to on a weekly basis, and the impersonal, low-context nature of them, response rates are often low, untangling the emotional from the valuable elements of a piece of feedback can be nearly impossible, and ideas that users have for impactful improvements can be hard to communicate in a four-line text box. While still useful, surveys shouldn’t be the only method in which teams gather feedback on their internal software.

Ways to Gather Effective Feedback

So what else can you do to fully understand pain points and opportunities in your internal software? Here’s a few ideas:

1. Sit-Withs and Shoulder Look-Overs — Have anywhere from one member to the full team sit with a user and watch them perform a task in the system. Good software makers know inefficiencies and annoyances as soon as they see them (often because they feel them deep in their soul), so they’ll know pretty much immediately when a task takes too many clicks, a page loads too slowly, or a button is in a non-optimal place. Understanding the impact of these opportunity areas becomes easier when the team sees the results with their own eyes, instead of just reading written feedback about them.
2. Dogfooding and SMEs — It’s one thing for the dev team to have direct exposure to users, it’s another to have users on the dev team itself. If the product’s targeted user-base includes development teams, why shouldn’t the product’s own development team also use it? That way the people who build the product are also users of the product, and fully understand the pain points and opportunities available since they themselves experience them daily. The fun name for this practice is dogfooding. If development teams aren’t included in the product’s target user-base, it’s often still useful to have users at least partially allocated to the development team, to gather peer feedback and represent the voice of the user. This can take a significant amount of load off of the product owner’s shoulders, allowing them to place more focus on visioning and general product direction, based in-part on the feedback filtered up by these product-using Subject Matter Experts (SMEs).
3. Systematic Feedback — Often forgotten are the many ways that teams can systematically gather useful, objective feedback. This can entail anything from simple measurements like page load times or the average distance of mouse movement on data entry forms, to more domain-specific feedback like the average time taken for common tasks or the average number of “mistakes” users make that require a different person to later correct them. While setting these up can get time consuming, once they’re working they can provide a flood of useful, drillable data available 24/7 for the team to craft product improvements off of.
4. Office Hours — Filling out a feedback survey can feel like bottling up your urgent needs and good ideas in a bottle, throwing it out to sea, and hoping that eventually it’ll float onto the development team’s shores. Setting up an open-invite once per week where users can join and talk directly with members of the development team about their pain points and ideas can greatly improve the fidelity of user to development team communication. Just note that effective moderation and meeting facilitation’s a must for these to work. The team must ensure that only one user has the floor at a time, that they don’t take the entire timebox, and that the conversation remains constructive for these to provide value.
5. Intake Forum — Beyond surveys pushed to users’ emails, an open feedback form easily accessible within the system means that as soon as someone has a good idea, they can send it to the development team, without waiting for the next survey to be sent or the next office hours meeting.

Keys to Making Internal Feedback Work

Gathering effective feedback on internal software requires more than just selecting the right tool(s). The proper leveraging of said tools will make or break how abundant and useful the resulting feedback is. Here’s a few ideas for making internal feedback more effective

1. Incentivize Feedback — Human beings operate off of incentives, it’s how we operate 98% of the time. Want more people to give better feedback on your internal software? You can try appealing to the heart strings, nagging, or threatening to talk to their bosses, but often the best way to get it is by providing a good answer to their question: “What’s in it for me?”. Recognition and rewards like gift cards are a good option. Another good option can be showcasing how a steady flow of great ideas can place someone on a path towards a creative, interesting role in software development. One of the product owners close to me got his role this exact way. He was a user in an operational role who signed up to be a SME (spending part of his day working with the development team as a resident user), and through his good ideas and strong partnerships in both departments he was eventually promoted into leading a team in the same area as the one he advised.
2. Coach on Giving Good Feedback — Like writing a screenplay, giving good feedback is something that most people feel like they they can do well, but are proven wrong once they actually try to do it. Giving feedback is a skill, and while it takes time to master, it’s pretty easy to learn the basics of. Depending on the user-base, teaching users how to give better feedback may mean that the team gets more actionable feedback, users feel heard and understood more often, and the users in question get the added bonus of improving on a skill that they’ll use throughout their entire career.
3. KISS — Besides being the name of a 70s hair metal band, KISS stands for “keep it simple stupid”, and it’s good to keep in mind when trying to get more feedback. People are busy, so whenever a sufficient amount of feedback can be gathered in a 3 minute survey choose that over a 20 minute one. You can always request to dive deeper 1–1 with someone if a piece of feedback catches your eye.
4. Trumpet Results — There are times where providing feedback feels less like throwing a message-filled bottle into the sea, and more like chucking it into the heart of a black hole. Since most development teams simply don’t have the time to address every piece of feedback delivered from a large user base, for submitters it can feel like their voice is being ignored, as their problems remain unresolved and their ideas unimplemented. Therefore to keep the good ideas coming, the development team should frequently showcase features and changes that came as a direct result from user feedback. This shows users that their voice is still being heard, and that the system is becoming better as a result of their feedback, but this cycle may take some time for all but the most important wants and needs to be addressed.
5. Have Leadership Emphasize the Employee Experience — Beyond knowing that their feedback is being heard, it’s worth reminding users that their feedback is actually wanted and valued. This can be messaged by their leadership team, by emphasizing how close the employee experience lives to their hearts, and showcasing what they’ve done to improve it. At the end of the day most people want to do their jobs well, and it’s incredibly frustrating when technology stands in the way of that. Leadership should show that they understand this, and that they’re taking action to enable their people.
6. Emphasize Growth on the Development Team — On the flip side, a constant bombardment of bug reports, complaints, and feature requests can start to demoralize any good dev team. It can start to feel like for every one thing they fix or improve, another ten complaints or “need to haves” come through the door. This hurts morale, and team members can even start feeling apathetic about the quality of their product as a protection mechanism. To prevent this, IT leadership and leadership from the users’ business lines should emphasize the positive impacts that the software brings to users, the company, and ultimately customers, alongside the opportunities. They should also look to provide examples of positive feedback from users (especially the unsolicited kind) and emphasize how iterative development means that no product is perfect, but every product can get better over time. Development team retention is critical to the long-term success of an application, and so keeping the software makers happy and inspired should live at the top of leadership’s todo list.

Conclusion

While internal product feedback seems less important and more challenging to effectively gather than its external cousin, it’s still critical for making great employee-directed software, which is in turn important for keeping your best people and allowing them to achieve their full potential. Let me know your thoughts on the above advice, and if you have any additional tips to enable better feedback that leads to better software!