GiveCrypto Monthly Update — August 2019 : Phase 2 Impact Numbers, Fraud Detection and a Post Mortem
Phase 2 Impact Results
Phase 2 of GiveCrypto’s Venezuelan Ambassador Program ended in mid-August. This program leverages local ambassadors to identify people in need and gives all participants (recipients and ambassadors) $10 per week for 6 weeks. As part of the program, we also recruit stores to accept payments in crypto. Here are the summary numbers for phase 2:
Charities should be able to measure their impact (are they helping their beneficiaries?) and be willing to share their impact data publicly. Impact is a vague term and each charity will define it differently. We believe that GiveCrypto will have the greatest impact along two dimensions: food security and psychological well-being. We measure this impact through a “pre-post” evaluation, which asks participants questions before and after they receive our help. The difference between their before and after answers gives us a directional sense of our impact.
While a pre-post evaluation gives us a directional sense of impact, we are not running an experimental evaluation like a Randomized Controlled Trial, which would enable causal claims about the relationship between changes that we measured and our program. For example, it is possible that other changes were occurring in Venezuela (e.g. delivery of food aid) that caused the differences we measured. That said, we still believe that there are important learnings and indications of impact in our data that we are excited to share:
Our impact analysis shows a statistically significant reduction in:
- People skipping meals
- People going a full day without eating
- People worried about having enough money for food
- People worried about health problems
There were two areas where we saw improvements that were not statistically significant:
- People worried about their children not being able to attend school
- People worried about the affordability of medicine
The charts below summarize the pre-post responses to the food security questions:
These data suggest that our aid leads to less people skipping meals (10 percentage point reduction) or going a whole day without eating (16 percentage point reduction). Both of these results are statistically significant at p=0.01, meaning that there is less than 1% likelihood that this happened by chance.
The charts below summarize the pre-post responses to the psychological well-being questions:
We see decreases across all categories, with a statistically significant reduction in worry over having enough money for food (9 percentage points) and worry over health problems (15 percentage points). There are also decreases in worry over children not being able to attend school (1 percentage point) and worry over access/affordability of medicine (7 percentage points) but these are not statistically significant, meaning that they are more likely to have occurred by chance alone.
Overall: We are excited by the findings from our impact analysis. We found statistically significant reductions in food insecurity and several measures of psychological well-being, which suggests that our program is working. While we are cognizant of the limitations with our data collection, these results are encouraging and illustrate the potential impact of our work.
Automated Fraud Detection
Due to the fact that we are giving away money, we are an attractive target for fraud. We have already implemented a number of fraud prevention mechanisms from relying on trusted ambassadors to invite new recipients, to SMS verification and ID upload. But, aside from sporadic back checks and manual blockchain analysis, we’ve done limited fraud detection. As we scale our operations, we will need to improve and automate these efforts.
Initially, ambassadors will be the focus of our proactive fraud detection. Ambassadors are given a lot of responsibility, which means that if they are acting in bad faith, they can do significant harm. We will start by considering three types of suspicious behavior; invite location, wallet addresses and recipient activity. Ambassadors that trigger any of these categories will be manually reviewed by the field operations contractor.
Invite Location
We ask ambassadors to be in/near the recipient’s home when they send an invite to the recipient. Part of the invite flow requires the ambassador to provide their GPS coordinates. If we notice an ambassador with suspicious location information, they will be flagged for review.
Wallet Addresses
The ambassador and their invited recipients should have unique wallet addresses. Additionally, there shouldn’t be significant amounts of crypto being sent between ambassadors and recipients. We will monitor wallet addresses on the blockchain and, if we see any of this happening, the ambassador will be flagged for review.
Recipient Behavior
We are already collecting a number of signals that could help us identify potentially suspicious recipient behavior. Some examples include where recipients spend their funds (i.e. known stores vs. unknown addresses) and how accurately the recipients answer the targeting questions (i.e. how the recipient and ambassador know each other). We will use these signals to calculate a ‘risk score’ for each recipient. Ambassadors whose recipients have an average risk score that is above a certain threshold will be flagged for review by field operations.
Additionally, we have recently hired a contractor in Venezuela to perform telephone back-checks. This person has no previous contact with field operation contractors or ambassadors and will randomly call participants asking them questions about their experience with the program. The questions will revolve around fraudulent behavior by other participants in the program and should provide another way to identify bad actors.
Community Currency Post Mortem
Community currencies are an intriguing area of research. Although there are various ways to implement a community currency, the basic idea is that you issue a new currency that is specific to a geographic area. The currency is distributed to individuals within the area and local stores agree to accept payment in the currency. This increase in monetary supply should theoretically increase the productive capacity of the economy, which should increase income and consumption. Community currencies are attractive because they can provide outsized impact compared to an organization’s funding. Cryptocurrency is a great tool for community currency implementation because it makes it relatively easy to create and distribute a new currency.
Our initial community currency experiment started during a Coinbase hackathon. We created an ERC20 token (DigiDolar — which is ‘pegged’ to the US dollar) and had one of our field operations contractors in Venezuela recruit vendors to accept payment in the token. The distribution scheme was deliberately simple; if the vendor agreed to accept payment in the token, we gave them $50 of DigiDolar. Over the course of the three day hackathon, we were able to sign-up four vendors and get one transaction completed.
Encouraged by these results, we decided to expand the scope of the community currency experiment. The goal was to recruit twenty vendors in the same town.
Thanks to the hard work of our field operations contractor, we were able to recruit sixteen vendors. However, we struggled to get the vendors to actually use the currency. I personally spoke with most of the vendors and got a myriad of excuses. These discussions made me realize that a successful trial would require significantly more effort than we could dedicate, as we are currently focused on the Venezuelan Ambassador Program. For this reason, we have decided to suspend the community currency trial.
I am still bullish on the potential of community currencies. However, we need to find the right formula and dedicate the necessary resources for a successful implementation.
