Impact needs to be measured for it to exist: a reflection on school projects

This relates to something very central in psychological research: measuring the observables against the operational definition of a concept.

And yet I never link it to the concept of impact we always mention in Human-Computer Interaction and Product Design — which by itself, might be an indicator that these concepts are really hard to grasp and transfer.

Apologize for the above nonsense if it doesn’t make sense to you; after all, I wanna share my very rough and immature thoughts with you in my own newsletter (that has no subscriber for now)[wink].

What impact is not

Impact is not “I’ve launched the product”

I was trying to articulate the impact of my school and volunteer projects the other day. But I couldn’t articulate anything convincing. The impact I can articulate is —

“The designer took my research results and designed a new game”

“I contributed to end-to-end design and shipped this product”

“I launched the redesign here”

Why these “impact” don’t seem to be convincing? Because they only state what you have done and do not tell how successful you are with your design or research.

Imagine you are creating a new onboarding experience for a fitness app. You have delivered the prototypes to the team. Mission complete, right? But how would the team (and yourself) know if the redesign is an improvement?

Imagine an extreme case in this scenario. With the redesign, the user bounce rate is 100% while with the old design it is 50%. Can you still say it is an impactful design?

Impact is not just “the usability testing suggested …”

The usability testing scores can tell you success rate, user error and time spent. But they won’t tell you what positive changes will happen for the users after they use this feature or app. In other words, numbers in usability labs do not translate into real-world impact.

Impact is the measurement against goals and metrics

So you need design goals and metrics first. They answer the following questions:

How do I know the redesign is an improvement over the existing experience?

How do I convince the team that my work is valuable and impactful?

If after the redesign, 80% of users went through the onboarding compared with only 60% before the redesign, your redesign is worthwhile.

Having the goal and metrics in mind can prevent designing for the sake of design. Designing hypothetical questions provides you with opportunities to hone your skills. But in the workplace, designing takes up more resources (and gets more serious). Without goals and metrics in mind, designers might end up putting resources in the wrong basket, and the design might end up not aligning with the product goals.

Impact in school projects

“There is no way to measure impact. The project is not launched”

“There are many restrictions in school/volunteer projects that I couldn’t measure my impact…”

In school projects, because most projects emphasize on design thinking and processes, and because most projects won’t launch or enter into the commercial world, it is hard to measure the impact against specific metrics.

But first, have you tried? This is a question I was asking myself and the answer is NO — no, I never even tried to measure success in school projects. Surprisingly, I found myself neglecting this important step because of the anticipation of the restriction of “we are not going to launch it”.

On a retrospective matter, something I could have done is to set up some metrics around usability, engagement, branding, service speed, and try to measure them in small qualitative testings.