SVB UWV Hackathon 2017: Blockchain + Facial Recognition + 24h? No problem!
6AM, Saturday morning. Half-asleep I start gathering my things (“what do I need to bring?!”) before quickly getting in my car: It’s Hackathon day! On the drive over to Eindhoven I start thinking about the e-mail I received last night, informing us on which case we were assigned.
What would the best approach be to tackle the issue?
Which techniques can we use?
Will it be feasible in 24 short hours?
We were assigned case two (out of five), asking us to provide an answer to the ‘alternative proof of life’ challenge. The government institution(s) at hand has the following problem:
Every day, thousands of people are entitled to various government benefits. When these people reside in the Netherlands, it is easy to check whether or not a person is still alive and should still be receiving his or her payments. In our surrounding countries this gets more difficult, as not every country can or wants to open up their registers to our government to check for this data. If you however know that this organization serves people in over 200 countries in every ‘corner’ of the globe, you see where a problem may arise. How do we know this person is still alive?
At this moment recipients are required to, once a year, go to a government official in their respective country. Who this government official is? We don’t know. What institution or organization? We don’t know that either. One of the ‘subject matter experts’ present at the hackathon could tell me there are several countries with surprisingly high numbers of clients that are well over 120 year old, and still receiving payments. Of course I believe the local climate could do wonders for a person’s health, but these numbers are understandably viewed with some skepticism.
In other words, fraud is a big problem. People pass away, and their partner/relatives/neighbors keep collecting their payments en masse.
Aside from this, there are more issues. People often have to spend much effort to visit such a government official, both time- and resource-wise. The process is cumbersome for all parties involved and imperfect at best. In this modern golden age of information technology, data revolutions and RAD development, surely there must be something better we can think of?
>> ENTER UWV/SVB HACKATHON <<
So that’s what we set out to fix. Easy enough, right?
Filled up on espresso and chicken wings –I’m taking a moment here to express my full-blown love for the cantina-chef and his crew because wow, that was some good chicken. Tender, great crust, delicious seasoning, not too thick: just right — me and my teammates from TimeSeries (Larn van de Vries, Richard Edens and Adnan Ramlawi) gathered ‘round a flipover and put on our thinking caps.
We all agreed we needed to solve the issues for both ends of the process: the end-user and the government employee. Employees now on average spend 6 minutes per case: a number we believe can be decreased significantly, while providing better information. A clear, easy-to-use screen on which reviewers get all relevant information and can efficiently make their decision. But what information would you want on this screen? Of course we have some personal details and perhaps even a phone number, but what would you need to know if someone is still in the land of the living?
>>HERE’S WHAT WE CAME UP WITH<<
According to us, there are two main concerns when requesting, receiving and reviewing think kind of information digitally.
1. Is this really the actual person I have to look into?
2. How do I know this has not been pre-recorded/prepared/digitally altered, i.e. how do I know it’s recent?
In our solution we addressed both questions. We created a personal environment for every end-user the issuing government wants a PoL (Proof of Life) from, in which he or she can see what the current status of the case is, and what is expected of them.
After agreeing to the terms, a ‘wizard’ processflow starts, in which the user is asked to follow the instructions and perform some actions. We first ask the user to show his passport(-photo) and take a picture. In the next step a visual overlay helps the user to successfully perform the next action: take a selfie.
Wait, but why?
Fair question: I’ll explain. After taking these two pictures, we send them to a cognitive facial recognition and comparison API. Using these verification processing services, we can see the outcome of a number of questions, of which to us the most relevant are: Is what I am seeing in the second picture a real person (and not a photograph) and are these two photos depicting the same person (opposed to a sibling, offspring, crabperson or whatnot). This results in a score (0–100) that tells us how likely it this that the person in the two photos is in fact the same. What you can do with this score will be addressed later, but wait…
That’s not all of it.
Our third step deals with the question of how recent this material is. The person in front of the webcam might as well be an illicit illusional intellect, a digital deceptive desperado, a genuine- oh well you get the picture (pun intended).
To deal with this, we ask the user to record a short movie. In this he or she needs to do two things: read a randomly generated sentence out loud, while performing a (as far as the user knows) equally random task. This sentence and task is what tells us this recording is recent: there are so many variations, this cannot be pre-recorded. The combination is tied to today’s timestamp.
Record, press, send: that’s it! Using an internet-enabled device with camera, the user can do in 2 minutes what previously would take him a day (or more): how’s that for convenience!
Now it’s up to the government employee to review the case.
Using a personalized dashboard and workload (I’m not going to keep spamming pictures, just believe me it exists) he can search for and start the review process of the supplied PoL. In this case detail screen they see the person’s personal details and contact data, a timeline of events, and more importantly: the sent information.
Here he instantly sees the result of the cognitive service, the video (which can be watched as well of course) and the text-to-speech transcription of the recorded sentence. Here he can proceed to Deny, Place in Review (which allows him to send a request to the user, to re-record the movie, provide additional information, etc.) or Approve the PoL case. Pretty cool right?
But wait, where’s the Blockchain?
You want Blockchain? We give you Blockchain! But first I’ll explain why it’s the perfect usecase, and not something we incorporated just because it ‘sounds cool’.
As you may have figured out, this process (and thus the application) deals with highly sensitive personal data. Privacy and Security are major, if not the biggest, worries surrounding this discussion. All of the above depicted and described information can be asked for by an agency and may be used, but can only remain in its possession and be stored as long as it is absolutely necessary. That is why immediately after a ‘decision’ has been made on a case, all sensitive information (recording, photo’s, etc.) is deleted. Gone! How then, you ask me, can we make use and share the gathered information? Well, with -wait for it- Blockchain of course! In our private hyperledger blockchain, a new entry is made with a timestamp, identification number, and a Yes or No on whether this person is still alive. This way at any point this chain can be consulted using our secure API, and it will tell you the moment this person was last verified to be alive (or not). A de-personalized entry, only containing derived information.
And to finish with our so-called ‘deal-maker’: This API can be made accessible to all other (selected/approved) government agencies which in their processes need this information as well, and now do not need to start a separate similar (costly) process! Clients do not have to go through the same steps for every institution over and over again, resulting in a happy client and a happy government.
Ow, and a happy TimeSeries Hackathon team, as this proposed (and fully functional) solution gave us the (almost unanimous — 6/7) win. Cheers!
