Helping People Succeed on Court-Ordered Supervision
Welcome messages that promote client/supervisor interaction
This piece was co-authored with my colleague Eric Giannella
Overview
People on community supervision (probation, parole, or pretrial supervision) are in a high-stakes situation. Their freedom is literally on the line: if they are not successful in meeting the terms of their court-ordered supervision, they may be incarcerated. Our research team at Code for America studied text messaging between people on supervision and their case managers to understand how texting might help supervisees avoid technical violations of the terms of their supervision. We learned that clients who engage in a texting relationship with their case manager do significantly better on supervision than their peers who do not. Furthermore, we learned that client engagement increases when the first message they receive from their case manager includes easily replicated qualities of personalization, politeness, and clarity about the purpose of the message.
The scope of the problem
1 in 55 adults in the United States is on community supervision. In 2016, over 4.5 million people were on community supervision for probation, parole, or pretrial supervision. That’s more than twice the size of the incarcerated population.
In 2016, at least 168,000 Americans were incarcerated for technical violations of the terms of their community supervision. About one-fifth of parolees released from state prisons are reincarcerated for technical violations, and as much as a quarter of the U.S. jail population is incarcerated for similar reasons.
Texting clients on court-ordered supervision
Some U.S. jurisdictions use text messages to remind people on community supervision about upcoming court dates, often with very encouraging results. One recent randomized controlled trial with pretrial clients in New York City found that reminder messages reduced the rate of failure to appear for a court date by as much as 10 percentage points.
Code for America researchers have built on this work to study the effect of two-way texting between supervision clients and their case managers. We have learned that when a person on community supervision engages in a texting relationship with their case manager, they are significantly more likely to succeed on supervision. And furthermore, the first text message the case manager sends to a new client can significantly increase the client’s engagement for months to come.
Our research grows out of our work on ClientComm. ClientComm is a two-way communication tool which allows people on community supervision to text with their case managers. Since 2017, we have run eight ClientComm pilots with county and state community supervision offices.
We built ClientComm to address a problem: community supervision is meant to serve as an alternative to incarceration, but too often it has the opposite effect as a “revolving door” to jail or prison. This is because people on supervision must meet numerous requirements: curfews, drug tests, travel restrictions, appointments, classes/trainings, etc. Failure to complete one of these requirements is called a “technical violation” and can lead to arrest, a court hearing, and incarceration, even though it is not in and of itself a criminal offense.
The idea behind ClientComm is that convenient, real-time two-way communication will help people on supervision get the information and support they need to avoid technical violations. The goal is to keep people from being incarcerated for reasons that have no bearing on public safety.
The text messages sent back and forth through ClientComm, when matched with outcome data from our government partners, are a treasure trove of information. Code for America researchers are able to study how case managers and clients communicate, seeking patterns associated with client success on supervision.
Clients who send texts to their case manager are more successful on supervision
One of our most important findings is that clients who use ClientComm to text their case managers are more successful on supervision than those who don’t. The association is strong and statistically significant in jurisdictions across the country, from Pima, Arizona to Salt Lake City, Utah to Baltimore, Maryland.
Table 1 below compares the outcomes of 3,825 pretrial supervision clients in Baltimore, 2,168 of whom sent at least one message to their case manager and 1,657 of whom did not. The clients who sent at least one message had a statistically significantly lower rate of supervision failure than the clients who sent no messages (p < 0.001).
Furthermore, the probability of supervision failure decreases for clients who send more texts, as shown in Figure 1, also based on data from Baltimore.
In fact, the relationship is similar across jurisdictions and holds regardless of whether the case manager tends to send a low or high volume of messages [Figure 2]. Departments and case managers may differ in the risk level of the clients they see. According to evidence-based practices, higher risk clients ought to receive more communication, which we see reflected in the figure below (higher levels of supervisor engagement are associated with higher failure rates). Nonetheless, regardless of the department or level of supervisor engagement, clients who send messages to supervisors also have a lower likelihood of failure.
Of course, this association doesn’t imply causality. Perhaps clients who are inclined to send messages to their case managers also have other characteristics that make them more likely to succeed on supervision. But we wonder whether there might be more to it than that. Perhaps when clients send messages, they (and their case managers) commit to the supervisory relationship — and its outcome — in a deeper way. Perhaps they are more likely to notice critical reminders when sent by a case manager with whom they have a two-way texting relationship.
An investigation of case managers’ welcome messages
Of all the messages case managers send to their clients through ClientComm, the most likely to receive a response is the welcome message. (About 26% of welcome messages receive a response, while only about 5% of all supervisor messages do.) Therefore, we focused our research on the content and language of the welcome texts to try to identify qualities of messages that clients are likely to respond to.
We started by reading anonymized welcome messages, making sure to sample across case managers, offices, and outcomes. We inductively identified categories such as “begins with a greeting,” “uses polite language,” “includes the client’s name,” “written in all caps,” etc. We hand coded hundreds of messages. We then used pattern matching to tag all Baltimore welcome messages true or false for 13 different qualities that we had identified. The tags became variables in regression models that allowed us to determine which message qualities were associated with receiving a response. The results are in Table 2 below.
Experimental confirmation: Welcome texts with certain qualities encourage client response
To test our findings, we designed an experiment with our partners in the Baltimore pretrial supervision department. We wrote a templated welcome message that includes qualities associated with higher rates of response. When creating a new client in the ClientComm system, case managers in the treatment group were invited to send the templated message (which they could modify or ignore if they so chose). The treatment also included information about the message qualities that encourage client response. Meanwhile, case managers in the control group had the status quo experience: when creating a new client, they were prompted to write a welcome message with no template or coaching.
We didn’t want the treatment group’s experience to influence the control group, so we assigned case managers to the two groups by office — that way colleagues who worked closely together would all have the same experience. Tests for “cross-contamination” of template use by the control group over the course of the experiment were negative. We also manually reviewed messages from both groups at the beginning and end of the experiment and did not see any spread of practices from the treatment office to the control offices.
The experiment results provide robust evidence that the templated message used by the case managers in the treatment group does promote client response, as shown in Table 4 below.
51.2% of clients in the treatment group replied to the welcome message, compared to 19.5% of clients in the control group (p < 0.001).
Clients in the treatment group continued to text their case manager at higher rates than the control
Our experiment proved that using the favorable qualities in a welcome message increases the likelihood that clients will respond to the message. But we were surprised and pleased to discover another difference between the clients in the treatment and control groups: the clients in the treatment group didn’t just respond at a higher rate to the welcome message, but they also sent significantly more messages overall.
This finding suggests that a well-crafted welcome message not only inspires an immediate response, but also encourages clients to engage in the texting relationship in an ongoing way. See Table 5 below.
48.7% of clients in the treatment group sent two or more messages overall, compared to 18.3% of clients in the control group (p < 0.001). 31.8% of clients in the treatment group sent four or more messages overall, compared to 10.0% of clients in the control group (p < 0.001)
Importantly, this distinction between the behavior of the clients in the two experimental conditions is apparent even when we only consider those clients who replied to the first message. This suggests that a well-crafted welcome message has a positive effect on clients’ ongoing engagement regardless of their individual inclination to respond to any welcome message. See Table 6 below.
Among clients who responded to the welcome message, 68.1% of clients in the treatment group sent two or more messages overall, compared to 41.9% of clients in the control group (p < 0.001). Among clients who responded to the welcome message, 44.9% of clients in the treatment group sent four or more messages overall, compared to 24.1% of clients in the control group (p < 0.001).
One possible explanation of these findings is that clients in the control group assume that the sms messaging from their case manager is one-way. Some of these clients may try replying to the first message with “Thanks,” or “Okay,” but the case manager is unlikely to respond to such a message. Therefore, clients might conclude that they can receive texts but not send them. In contrast, clients in the treatment group are explicitly told in the welcome message that they can text their case manager.
Any effect of the treatment on outcomes was not enough to overcome natural variation in failure rates
Given the treatment’s success in increasing communication between clients and case managers, it’s natural to wonder whether an approach like this could lower the rate of technical violations. Baltimore already had very low failure rates, which meant that there was little room for improvement. While we did observe a decrease in the failure rates if we compared the treatment and control groups before and after the beginning of the experiment, in exploring the data, we found that the changes could easily be attributed to trends that preceded the experiment. As a result, we do not feel confident drawing any conclusions about the efficacy of our messaging template in reducing failure rates. We hope that future research can measure the effect of higher levels of client engagement on the rate of technical violations.
Conclusion
Our learning from ClientComm started with an analysis of the conditions of client success, continued with a qualitative investigation of the characteristics of case managers’ messages, depended on modeling to identify which of those qualities were associated with client responses, and concluded with a randomized controlled trial.
We discovered that clients who engage in a texting relationship with their case manager do significantly better on supervision than their peers who do not, and that more client engagement is associated with higher rates of success. Furthermore, we learned that a well-crafted welcome message prompts significantly more client engagement in the texting relationship, and that such a message can be presented as a template to case managers.
Building on these findings, we recommend that community supervision departments that utilize two-way texting pay special attention to how clients are introduced to the texting tool. A welcome message that includes an explicit, personal invitation to engage in a two-way texting relationship can have a long-lasting impact on client engagement throughout the period of supervision.
