Accidental bias & lived experience

How common current practices might be leading social service providers amiss

A few days ago the National Alliance to End Homelessness released guidance on how homeless service providers, continuums of care, and other groups serving individuals experiencing homelessness can react to the special needs that have emerged from COVID-19.

In A Framework for COVID-19 Homelessness Response: Responding to the Intersecting Crises of Homelessness and COVID-19 guidance is broken down into four categories. One of the recommended items in the first section is:

Engage people with lived expertise of homelessness* to ensure plans for this population are adequate and implementable.

*Emphasis was added by the author

It also was recently backed up by Funders Together to End Homelessness’ own guidelines — found here: http://www.funderstogether.org/covid_19_response_and_recovery_recommendations_for_philanthropy

Create space to learn about authentic collaboration in policy and funding decisions [by] engaging people with lived expertise.

This guidance is unsurprising.

Over the last half-decade, there’s been a push to involve individuals who have experienced being homeless in the planning and implementation of programs. This leads to better outcomes — it has become a best practice that nearly every homeless service provider has at least started to implement in some way.

Listening to people is transformational

The guidance to gather feedback from individuals with the expertise lived homeless experience is transformational. It requires more than a token effort. It requires a shift in organizational philosophy from one of expertise to one of humility.

Consider the Housing First movement, originally created by Sam Tsemberis, and now adopted by organizations around the world (with great success). When you discuss with Sam client feedback he will tell you that the birth of the Housing First model came from trusting the individual who is experiencing homelessness. Trusting their opinion and respecting their wishes was the start of housing first.

Traditional ways of collecting feedback from individuals with lived experience

The most common way homeless service providers gather feedback from individuals with lived experience is by forming an advisory board of individuals who have experienced homelessness.

These boards tend to meet monthly and give feedback on policies, programs, and events. The best providers then use this feedback and implement more client-friendly opportunities.

These boards are often made up of people who have experienced homelessness in the past but are no longer currently homeless. Occasionally, a homeless service provider will include individuals that are currently homeless. These individuals tend to be the ones that the providers have a good relationship with.

These board formulations are natural — organization invite those that they trust and that they know can attend the board meetings but this natural formation also lends itself to a sneaking type of bias.

The accidental bias of those who have successfully navigated the system

As already mentioned, lived experience advisory boards are often full of individuals who are no longer homeless. They have received services from a homeless service provider, often from the organization on whose board they serve, and using these services navigated their way out of homelessness. This is no easy feat and their experience can lead to a lot of valuable insights.

It does, however, introduce an accidental bias as these groups discuss and seek to improve programs that have already worked for them. While they seek to improve the program it’s tremendously hard for any individual to see a way that it absolutely wouldn’t have worked for them — we are after all reflections of our own experience, if it worked for us it will work for everyone else with some tweaking.

These board members make suggestions and these suggestions do improve services but they might just improve the services for the group of individuals for which those services were already going to work.

What is lost in this is the idea that it might not work for everyone else. You might have an entire subset of your population for which these services do not work and by continuously asking the people it did work for how to improve it you end up in a cycle of improving the thing that works for some and not others.

This type of bias actually has a specific name: Selection Bias

Here’s a quick definition of selection bias -

Selection bias occurs when you are selecting your sample or your data wrong. Usually this means accidentally working with a specific subset of your audience instead of the whole, rendering your sample unrepresentative of the whole population. There are many underlying reasons, but by far the most typical I see is collecting and working only with data that is easy to access.

Sound familiar?

How to avoid accidental selection bias

So how do you ensure you are getting feedback and guidance from individuals with lived experience without falling into this particular accidental selection bias?

The easiest way is to involve those individuals that are currently experiencing homelessness and, importantly, not just those that the organization has a good relationship with.

This is a difficult task — people in vulnerable situations are often reluctant to share feedback, especially negative feedback, because of their legitimate fear of losing out on services. This puts providers in an interesting conundrum. How do you involve someone who doesn’t trust you in the planning and feedback process?

Providing a safe place for your clients to share how they feel

There are various ways to handle this situation and while I myself have a strong bias towards a particular way that uses self-service kiosks there are other ways to engage clients on their terms.

Paper surveys, third party reviews, private advisory board meetings, group town halls, or even online surveys can provide a meaningful way for individuals to give feedback. Any source of feedback it should be noted comes with a form of selection bias — so it is best to try and utilize multiple strategies to gather feedback.

The main thing to consider in each of these ways is ‘How can I protect the anonymity of the individual?’. Just saying ‘I will protect them — no one else will see it and I know that I won’t use this against them’ is common but isn’t enough and is a shockingly easy avenue for abuse.

You not only need to provide the safety that comes from an organization's staff being good and ethical people but you need to provide systematic safety that protects the individual's identity.

Organizations must protect and project that anonymity. If someone doesn’t feel safe in giving feedback, no matter that actual safety mechanisms are in place, then clients will never share how they truly feel.

(Amidst all of this, someone might ask — is this worth it? Does it work? Yes — it does and I can share with you from personal experience how it does.)

Listening to people is transformational

Organizations have applied the outcomes of Sam Tsemberis' hard work to great success with the housing first model but have ignored the origin of his work like humans accepting Prometheus’ gift of fire without ever wondering how to make more of it.

When we build a culture of listening we do not just adopt the best practices of others but we discover best practices for ourselves. We discover our own personalized best practices, our transformational programs, we build our own wells of empathy. We will discover the truest ‘one-sized fit’s all’ type of solution is listening and responding to what vulnerable individuals need.

A culture of listening will not come without some investment in time or money. However, the invested time or money will be the best investment one can make in protecting an organization from the easily overlooked, and accidentally implemented, selection bias that might be holding an organization back from reaching its ultimate potential.