Models and obstacles for targeting vulnerable groups — an economics and computer science perspective

Reflections on “Provision and Targeting for Vulnerable Populations” Tutorial at EC’20

EAAMO

Follow

Published in

EAAMO

13 min readDec 19, 2020

--

This post by Nicolas Pastrian Contador is a brief (biased) summary of the Provision and Targeting for Vulnerable Populations tutorial, presented by Prof. Sera Linardi and Prof. Sam Taggart at the Economics and Computation Conference in 2020. This blog series represents our members’ reflections on the various MD4SG activities related to mechanism design and access to opportunity. Our previous talks can be found on our Youtube channel.

If you don’t have previous exposure to the work of the Mechanism Design community, this post is a great place to start.

The issue we are concerned with in this tutorial is: you wish to transfer goods, services, or cash to those with most need. How do you identify these individuals?

Applications range from social safety nets (housing, education, income support, healthcare, job search) to development programs (asset transfers, health interventions, training). There are high stakes involved in these types of programs: expenditures on health programs comprised 6.6% of U.S. GDP in 2002, and ranged from 4.6% to 8.4% 17 OECD countries. For education related programs it was 5.3% in the USA and ranged from 3.3% to 7.3% for the same group of countries in the same year (Currie and Ghavari, (2008)). Moreover, those types of programs had a huge impact on reducing poverty in the USA (see figure below).

The classic toolbox for targeting involves 4 main techniques: proxy means tests, community-based targeting, ordeal targeting, and categorical targeting. In practice, these techniques are not implemented in isolation, and mixed implementations are not uncommon. This tutorial reviews the first three of these techniques below.

Why is targeting necessary?

Wealth differences lead to a suboptimal consumption from individuals since some of them consume too much and others too little from a social welfare perspective. Markets alone cannot solve the problem of redistribution. Moreover, in-kind transfers would be necessary if other market distortions (i.e., externalities, market power) or limitations (i.e., substance addiction, lack of information) are present (Nichols and Zeckhauser (1989), Currie and Ghavari (2008)). Hence, we will need a way to identify poor individuals in order to improve efficiency. As U.S. Census bureau reports, governmental programs are often helpful in alleviating poverty, but efficient targeting remains an open question.

Method 1 of 3: Proxy Means Testing (PMT)

The simplest technique to identify vulnerable populations is trying to classify them according to their income or current consumption level. So, what we want to achieve is binary classification between eligible (“poor”) and ineligible (“non-poor”) people. This classification is easy if tax data is available since we have a direct measure of their income/consumption. However, if such data is unavailable or it is not reliable (e.g. informal sector is important), then we will need to use other observable features in order to predict people’s consumption and classify them accordingly. This is what we call proxy means testing (PMT).

An example of the use of PMT for targeting is the PKG program in Indonesia studied by Alatas et al. (2016). This program offered cash transfers of approximately $130 or 10% yearly income to individuals below 80% of the poverty line, which in 2018 was around $29USD/month. In order to identify recipients, first a survey on consumption and household characteristics was administered to a subpopulation. From the data collected, a subset of variables were chosen which were easy to measure, costly to manipulate and which have high explanatory power. These variables were used to construct a regression model. Finally, this model was used to determine eligibility on the whole population.

Challenge: prediction error

However, Hanna and Olken (2018) have shown that the prediction errors both in terms of exclusion of eligible individuals (Type I error), as of inclusion of ineligible individuals (Type II error) are present. Hence, this approach is not absent of problems.

Manipulation in PMT

Manipulation is an actual threat to these designs. Ads that provide resources and help for applying to some of the programs could be easily found across the web. This could become a threat to classification (i.e. identification of the poor) if non-vulnerable people wanting more resources change some of their characteristics in order to get included in the program. This is a reason why PMT usually aims for using features which are not only easy to measure but also hard to manipulate.

Strategic classification

An interesting approach to analyze manipulation is to look at targeting as a learning problem, as the one studied by Hardt et al. (2016). There the authors study a learning problem where training data is honest, but testing is performed on manipulated data. Here, the designer goal is to correctly classify individuals knowing that they will manipulate their characteristics in order to be selected, while individuals want to maximize their expected net gain from the classification procedure. For example, there is evidence that the number of books is a good predictor of academic performance. However, if we try to use it to help us to classify potential school or college applicants it would induce families to acquire more books only to “game” the algorithm in order to be selected, reducing the efficacy to better discriminate between individuals using this information. The proposed solution for this issue involves increasing the selection criteria, i.e., “move the goalpost”, in order to countervail the potential manipulation carried over by people.

Figure from Session 2B: increasing classification threshold to overcome manipulation

Does strategic classification treat vulnerable populations fairly?

However, there is an important concern regarding inequality in access to resources or services with this approach since the impact of increasing requirements and the ability to manipulate could vary across groups. Indeed, it seems that this is the case since by comparing classification with and without the increased selection criteria: the welfare disparity between non vulnerable and vulnerable population increases if there are differences either on the access (cost) to manipulation technology or underlying differences on the distribution of features on both groups (Milli et al. (2019)).

A potential solution to this problem is introducing subsidies to overcome the differences between manipulation costs so the vulnerable population gets a better chance to access the technology or resource provided. However, Hu et al. (2019) shows that it is possible to improve the designer objective -improve classification adjusted by the expected cost of the subsidies- but at expense of the utility of both groups. That is, while classification is improved, both vulnerable and non-vulnerable populations are worse off.

Method 2 of 3: Community-Based Targeting (CBT)

An alternative technique involves delegating targeting tasks to local agents instead of carrying it over in a completely centralized way. The main idea is that a community has more information available about individuals due to the proximity of their interactions. In practice, there are various ways to aggregate this information, including through local leaders, informant-based approaches, and community meetings. The last two involve participatory wealth ranking, in which members of the community decide how to classify individuals in the community according to their characteristics: the group agrees on a poverty definition and then ranks members according to wealth.

Each of these methods has its advantages and limitations. However, as we will see below, there are potential gains from using a community approach to the targeting problem.

Learning from Community Data

Alatas et al. (2012) compares the performance of CBT and PMT. They issued an open invitation to the community members, and then participatory wealth rankings were used to classify the members of the community. While detailed, this procedure was time-consuming: more than 1 hour and a half on average.

By collecting rich data on consumption and individual characteristics, the authors were able to compare the classification accuracy induced by three different approaches: direct classification (from consumption data), proxy means tests based on characteristics, and the community-based approach.

They show that classification accuracy reduced over the meeting, that is, in the beginning community-based classification performed better than PMT but provide a worse classification as times passes. That is, fatigue has an impact on the classification procedure. Hence, a better classification could indeed be obtained by including communities in the process but the protocol used for classification is quite important to determine the performance of this process.

Who is considered to be ‘in poverty’ within the communities?

This comparison between the classification PMT and CBT also shows that there are variables that the first approach could completely ignore but that seems relevant for communities when they are ranking their members. In particular, Alatas et al show that PMT does not consider whether individuals are part of a minority group, but it is a relevant variable for communities. Hence, the whole definition of poverty could be augmented by the involvement of the community in the process of classification.

Method 3 of 3: Ordeal Targeting

Both methods above involve active targeting the population of interest, that is, actively looking to identify the ones that must be served. An alternative approach is called ordeal targeting or self-targeting.

Here, the idea comes from analyzing the problem from another perspective: we want to give benefits to impoverished communities through some program, but such benefits could also be attractive for the non-poor. Self-targeting deals with this problem by imposing costs that would reduce the attractiveness of the program for the non-poor, so only the ones in need get access to these services. Hence, by designing a cost scheme which has a different impact on poor and non-poor, the targeting efficiency is improved. Long applications, waiting times and required visits on working times are examples of cost schemes that will in theory have different impact to each group, and are expected to make programs less attractive for the non-vulnerable population.

What is the problem with ordeal targeting?

One of the problems with ordeal targeting is that ordeals are usually unproductive, they generate little or no value. The improved accuracy on targeting should balance against such waste and loss on productive efficiency. A related concern is that the poor and vulnerable populations are the ones actually paying such costs, while it is the behavior of the non-poor driving the necessity to rely on such distortions. Moreover, such costs could discourage applications from the poorest, which is probably the population we should care more about.

Limits of policy implementation

There are some theoretical and empirical considerations that could make self-targeting fail. For example, if there is a technology available that could allow some people to avoid or reduce the costs imposed, then the procedure could be ineffective. Another reason that could make ordeal targeting unsuccessful is the implementation of exemptions, that is, even if an incentive scheme is properly designed so that the poor would self-select into the program but we allow people to skip such requirements then the targeting will fail anyways, allowing non-vulnerable population to easily apply. Indeed, Alatas et al. (2016) shows that the PKG program contained exemptions that made increasing targeting requirements unsuccessful.

Trade-offs in ordeal targeting

Another related aspect discussed by Kleven and Kopczuk (2011) is how stricter selection criteria and the amount of benefits interact with both types of selection errors. On one hand, more complexity or screening intensity could improve targeting accuracy, reducing both false positives and increasing true positives among the ones that apply but could at the same time reduce applications from the vulnerable population, which is undesirable. On the other hand, increasing the program benefit would increase the number of applications but could potentially increase false positives. Hence, there is a trade-off between these two dimensions that we need to balance in order to implement the best targeting policy possible.

Behavioral considerations

The game-theoretic or traditional approach to self-targeting basically concludes that increasing screening rigor improves provision to vulnerable populations since it discourages those defined as non-poor from applying for benefits. However, in practice, several programs present a problem of under-coverage, i.e. only a fraction of the eligible group defined as ‘poor’ is being served. Therefore, something is missing, and so increasing complexity and screening rigor could indeed have the unintended consequence of providing less resources to ones in need.

Figure from Session 2A: a summary of the traditional and behavioral approaches to self-targeting.

Why are eligible individuals not applying for benefits?

Poverty could impact the performance of people, hence negatively impacting their capacity to respond to the incentives and benefits available from certain programs. Indeed, Mani et al. (2013) showed that for farmers in India, decision-making was impacted by whether they were before (low income state) or after the harvest (higher income state). This suggests that the actual income level could impact the quality of people’s decision making, in that there are cognitive costs related to poverty. However, the general evidence on the impact of these costs is still mixed.

Another reason why people are not applying could be impatience. That is, impatient people could be less willing to apply for benefits because the waiting time is restrictively long for them. While a correlation between income and impatience is documented in the literature (Tanaka et al. (2010), Dohmen et al. (2018), whether there is a causal relationship and what is the mechanism behind this relation are still open questions.

Deservingness: are the poor culturally perceived as lazy or unlucky?

Depending on what is our perspective on why a particular population is poor, we will be more or less willing to provide support. When people misperceive the poor as lazy they are less likely to provide assistance. For example, we will be less willing to support people living in poverty if we think that there are job opportunities available to them that they are willfully not taking, or because of stereotypes and stigmatization about substance abuse. In contrast, we will be more willing to help the poor and support social programs designed for vulnerable populations when we perceive them as unlucky. For instance, we would be more willing to help if we perceive their poverty condition coming from an accident, an underlying health condition or a disability. Moreover, some authors argue that such differences could explain the differences in redistributive policies across countries (Piketty (1995), Alesina et al. (2001); Alesina and Angeletos (2005)).

In this sense, the way a situation is framed could impact the public’s willingness to support a program or population.

Final remarks

The theory used for rationalizing individual behavior is key to computing the expected consequences of a policy. Different reasons behind the vulnerability of individuals, as well as different evaluations of such circumstances, would determine both the effectiveness of the implementation of certain policies on reducing poverty, as well as determine the willingness of the public or authorities to implement such policies.

More accuracy is not necessarily better: if the poorest face any type of limitation, then improving accuracy could have the unintended consequence of excluding the ones most in need. Moreover, the possibility of manipulation in order to be eligible could even increase the differences between poor and non-poor populations.

There could be an important gap between theory and practice: sometimes carefully planned policies do not work due to unexpected obstacles, from individuals or from institutions, that distort the outcome predicted by the theory.

All the different approaches to targeting have their pros and cons, and they should be seen as complementary in the goal of helping the poor.

The discourse used about impoverished communities is fundamental to obtain support for certain programs or populations. An example is the way people with drug or alcohol abuse are seen: if they are seen as people having bad habits and making bad decisions, then the public will be less willing to help them. However, if we recognize that indeed those people have an underlying health condition, and are experiencing something outside of their control, then people could start acting more empathetic and supportive to them.

Moreover, a particular way of thinking about the poor would be strongly influenced by the norms and politics of a particular society, and not necessarily have to do with any intrinsic characteristic of a vulnerable population.

Due to the limited resources available to help people in need, targeting is crucial in order to focus such resources on helping the most vulnerable population. This is especially true in the current pandemic situation where not only is this group the one that suffers more, but also that the pandemic crisis has exposed even more people to being part of this disadvantaged group. Hence, knowing the targeting tools available and their limitations have become even more important today for helping the disadvantaged to succeed.

Our working group is currently considering theoretical and practical approaches to some of these questions — we welcome thoughts and suggestions!

Written by Nicolas Pastrian from University of Pittsburgh. This is part of an ongoing conversation within the MD4SG Inequality Working Group — if you would like to join this conversation, please feel free to reach out to us at organizers@md4sg.com.