Lessons from Tulsa, OK: Sending text messages to encourage residents to connect with support services during a crisis

Learn how Tulsa helped more households across the city access critical financial supports during the COVID-19 pandemic

Case study: Help More Residents Access 211 Helpline Services with Text Message Reminders

Project Summary
City: Tulsa, OK
Goal: Increase access to financial support services during times of crisis
Department Lead: Office of Performance Strategy & Innovation (OPSI)
Timeline: 3–5 Months
Intervention: Text messages urging residents to call 211 to check their eligibility for financial assistance
Evaluation Method: Randomized Control Trial
Result: Individuals receiving the text messages were 17.6 percent more likely to call 211 and 18.5 percent more likely to be connected to financial services compared to those who did not receive the messages. In sum, 144 households that received the text message called 211 and got connected to housing services, resulting in an additional $16,500 of financial assistance provide to the households that ultimately received support.

Want to try this yourself?

Read an overview of the project below and then download the evaluation replication worksheet.

NOTE: This worksheet includes detailed steps that you can follow to implement this project in your city and design a rigorous evaluation. This resource is best suited for city staff working in Innovation & Performance or Communications / Resident Outreach.

Housing instability is heightened in times of crisis. Text messages can facilitate resident connection with support services.

Before the COVID-19 pandemic, the U.S. was already dealing with a fully-fledged eviction crisis, with an average of 3.6 million eviction cases filed each year. Getting evicted, or being threatened with eviction, is an extremely taxing endeavor for the people involved, psychologically and financially. Evictions also take a toll on local governments which may need to bear the costs to provide supportive services to individuals who have been evicted.

Since the onset of the COVID-19 pandemic — and the ensuing public health and economic crisis that have been made obvious in its wake — financial instability has spread throughout the country, disproportionately affecting low-income families. In 27 U.S. cities that are being tracked by the Eviction Lab, over 300,000 landlords have filed to evict their tenants, despite a CDC moratorium on evictions as well as other local protections, and an estimate of 30 percent of all renters are at risk of eviction.

Local helplines, like 211, connect residents with support services ranging from legal and financial assistance to nutrition and housing support. In times of crisis, connecting residents to a helpline (or an online platform) can be a powerful solution for people in need of support who may not be aware of the services available to them.

In Tulsa, OK, housing insecurity has pre-dated the COVID eviction crisis: in 2016, the city had the 11th-highest eviction rate in the country. In the fall of 2019, in an effort to rectify this issue, the City of Tulsa enacted the Affordable Housing Strategy, which outlines goals and action steps to address housing inequities. The city’s Office of Performance Strategy & Innovation worked with What Works Cities and the Behavioral Insights Team (BIT) to develop an intervention that would encourage residents at risk of eviction to call the local 211 helpline — a directory of 7,600 vetted services available to Oklahoma residents, including rent payment support services. When the COVID-19 pandemic hit, it became clear that this intervention would be particularly beneficial to Tulsans who were seeking support even beyond housing.

Increasing traffic on helplines is an easy and inexpensive way to make sure that people’s needs are efficiently met. While not all cities have a 211 service as comprehensive as Tulsa’s, over 95 percent of the U.S. population has access to 211 services via phone. Helplines streamline the service referral process by providing an easily accessible system, available 24/7, that connects callers with trained professionals who can dispense up-to-date information and refer them to a suitable supportive service.

For Tulsa’s leaders, the question then became: How might the city encourage more residents in need to call this helpline — and ultimately secure the assistance they needed?

Use learnings from behavioral science to send timely text message reminders that let residents know about 211 services, and nudge them to call if needed.

In order to increase traffic to the 211 helpline in Tulsa, the Office of Performance Strategy & Innovation designed a text message notification to make residents aware of the 211 service and remind them to call the helpline if they were in need.

To design an effective solution for this group, the project team first mapped the process of utilizing the helpline service to explore why more Tulsans were not using the helpline to access financial support services. By talking to eviction experts at the housing council and turning to the data, the team learned that the biggest barrier may be that residents are simply unaware of the existence of the 211 information and referral service. In addition, residents at risk of eviction may be facing additional demands on their mental “bandwidth” due to the increased financial strain and stress of the pandemic, and may not seek out additional supports, like calling 211. (This is a concept known in the behavioral science literature as cognitive load.)

As a result of these factors, individuals experiencing financial distress may be even less likely to follow through on tasks such as pursuing available assistance options, especially for those who have never used the 211 service before.

The city concluded that a text message campaign could help increase awareness of the 211 service and, with the right messaging, it could convey how straightforward it is to access the information/referral service and also reduce the stigma associated with accessing these services. In fact, texting reminders is an excellent way to get more people to call a service, as they are conveniently reaching the targeted individuals on the phone from which they are likely to place the service call.

[Check out this worksheet to replicate the evaluation that Tulsa conducted to see if their text message awareness campaign was successful.]

See below for an example text message and an explanation of the relevant behavioral science concepts that were leveraged in its design.

How the message leverages Behavioral Insights:

How do we know if this text message campaign worked and how cities can run their own similar evaluation and project

Tulsa designed and launched a randomized control trial to evaluate the impact of this text alert on people who were considered at risk of eviction because they were experiencing financial distress. The at-risk population was identified by an in-house predictive model based on water billing payment data: the model selects people who are considered more vulnerable to eviction based on whether they have a history of missing utility payments or have had their utilities shut off.

Note: Your city does not need to have a predictive model to replicate this trial! If you have access to a database that contains information indicating the financial status of residents and their phone numbers you can use that information to target recipients. Another option could be to use census level data or a resource like the Opportunity Atlas to identify individuals with higher needs. Or, you can roll out the intervention to a broader group of residents, as long as you can get contact information for all the recipients.

Once the city had identified the recipient group and assembled its phone contact details from the water billing information, the team used Twilio, a cloud communications platform, to send a text message to a random selection of half the people in the sample, while the other half (which served as a control group) was only exposed to the 211 material they would have normally encountered, such as emails or print advertisements.

One of the best parts? The trial was inexpensive to run. Sending messages for this trial cost only $0.0075 per message, for a total expense of about $40 to reach over 5,100 residents.

The city was able to measure the impact of the text message by comparing the number of calls, uptake in support, and service connections from the group that received the message to the group that did not (i.e., the control group).

Tulsa found significant differences between the two groups: text recipients were 17.6 percent more likely to call the helpline and 18.5 percent more likely to be connected to financial services. The success of this intervention showed that text messages can be an effective and cheap method of outreach to at-risk residents. They can help lower the barriers for residents to connect with available financial services — a key step in increasing the uptake of these services for eligible residents. Over the five-month trial, 144 households that received the text message called 211 and got connected to housing services. When compared to the expected rate for the uptake of these services in this period, the text message resulted in an additional 11 households receiving assistance and an additional $16,500 of housing assistance provided.

The Evaluation Protocol

The remaining section of this guide will walk you through a step-by-step on how to adapt this intervention and evaluate its impact in your community.

An evaluation project has four key steps:

• Scoping Your Evaluation: Identify your goals, target population, and approach
• Designing Your Evaluation: Develop your evaluation method, outcome measures, and randomization approach
• Implementing Your Evaluation: Deliver your solution, ensure everything goes to plan, and collect data
• Analyzing Your Results: Conduct data analysis based on your collected data

To design an effective evaluation, Tulsa developed an “Evaluation Protocol” to guide their evaluation design and implementation. While sending a text message may seem straightforward and might not feel like it needs a formal protocol, evaluations need to be implemented in a careful and precise way to generate results that you (and others) can learn from. A protocol can help you to document the decisions and intentions around your intervention. It’s a forward-looking plan that will help you down the road — and also make it easy for people to replicate your work in the future (like we are doing now). We’ve made it easy for you to implement a similar project in your city by giving you Tulsa’s evaluation protocol to follow along.

Download the evaluation replication worksheet to see Tulsa’s evaluation protocol and develop your own approach!

How to use your evaluation results

Congrats — you’ve implemented your evaluation, analyzed your results, and (hopefully) you have lowered your city’s eviction rate by increasing the number of callers to the support services helpline for your community. We also hope you learned about evaluation too — and have a new set of tools you can use to design evaluations in the future! The most important part of running an evaluation is using the results to make decisions and scaling up what works.

After Tulsa’s evaluation, they had a few ideas of what they could do using what they learned.

• Documenting the Learning: They kept a record of the project report for internal use
• Scaling the Program: The City, and in particular, the team at OPSI, are working with the managers of the housing assistance programs to set up the SMS directory and design the messaging for future interventions.
• Using the Model: They plan on using this trial as a model for future housing insecurity interventions: they intend to conduct text message campaigns to alert residents of opportunities for the CARES housing and utilities assistance programs.

In 2020, Tulsa achieved What Works Cities Certification at the Silver level for data-driven excellence. The city is also a member of the cohort of 9 cities that are participating in the What Works Cities Economic Mobility Initative.

Launched by Bloomberg Philanthropies in April 2015, What Works Cities (WWC) is one of the largest-ever philanthropic efforts to enhance cities’ use of data and evidence. Want to learn more about What Works Cities? Contact certification@whatworkscities.org.

The Behavioral Insights Team (BIT) is a social purpose organization that helps governments around the world boost their impact through the application of behavioral science and rigorous evaluation techniques. As part of What Works Cities, BIT helps cities develop and rigorously evaluate ideas for improving government services. We define rigorous evaluations as evaluations that utilize experimental or quasi-experimental methods. We believe this is the best way to help understand the impact of a program or policy.

Want to learn more about the Behavioral Insights Team and services available to cities? Contact info@bi.team.