This Robot Fights Back Against Robocalls

How I used Twilio and a few lines of Python to stop all unwanted callers permanently.

John Coogan
Sep 19, 2017 · 6 min read

I, like most people, am annoyed by telemarketers. But recently the problem has gotten so much worse due to the automation of scam calls. This was covered in detail by NPR’s Planet Money and investigated extensively by Reply All in an amazing 2 part podcast (available below). The Planet Money piece shows that these robocalls are unlikely to end anytime soon, since the FTC is moving very slowly on the issue. Then, right in the middle of the piece, they played an ad for Google Cloud Platform and I got an idea. I could make a bot that would screen for unwanted calls and only let quality calls through. Like a digital personal assistant.

When it comes to blocking just Robocalls, there are several apps that do a decent job at an affordable cost. This NYTimes article highlights some of the best options. I wanted to go a step further though and put a firm line of defense in between myself and unwanted calls.

How the bot works:

  1. Calls come in to my normal phone line.

Breaking Down Each Function:

Step 1: Maintaining a normal phone line.

Image for post
Image for post

Putting software between your phone and the world is a lot harder than it should be. Twilio offers a great interface for creating programatic phone systems, but porting a phone number to Twilio is a six step process that can take up to four weeks.

A better option, in my opinion (if you have an iPhone), involves creating a new number for Twilio, which you will use publicly, and then restrict your personal phone line to only accept calls from numbers in your contact book. You can restrict unknown numbers from making your phone ring by setting Do Not Disturb mode to allow calls from your contacts and silence all other numbers. The key here is to set the schedule from 10:00PM to 9:59PM so Do Not Disturb mode will stay on permanently.

This, in and of itself, is a good start and will help clean up unwanted calls. But it’s a bit of overkill, so setting up a new number that you can use publicly to forward calls to you based on criteria you set is a good next step.

Step 2: Check Contacts via Google API

If you sync your iPhone contacts with Google, accessing your contact listing via API is relatively simple. You just need to get the people resource from the Google People API. I find it best to crawl this for updates daily and save all the “known numbers” to a JSON file for easy access (it’s just a list of phone numbers that will be passed through to your real number).

When a new call comes in to the Twilio number you have setup (and given out freely), Python opens the JSON file with all the stored numbers and checks if the caller is in the list. If so, the call is automatically forwarded and the caller is unaware of this process, if not though, the caller will be prompted to explain who they are and why they are calling.

Step 3: Prompt caller for explanation via Twilio Speech Recognition

Image for post
Image for post

Twilio offers great out-of-the-box speech recognition that allows you to prompt a caller with a message and then automatically record and transcribe their response.

from twilio.twiml.voice_response import Gather, VoiceResponse, Say

speech_result = VoiceResponse()
gather = Gather(input='speech', action='/completed')
gather.say('You have reached John's automated answering service, please state your name and why you are calling.')
speech_result.append(gather)

This will return a transcribed SpeechResult object for further handling. There is the opportunity to inject some machine learning into this step to understand why someone is calling, maybe correlate that with my calendar or recent emails and handle the call accordingly, but that is out of scope of this post.

Step 4: Text me the transcription via Programmable SMS

Sending this result to me via text is trivial with Twilio. The python code looks like this:

client = Client(account_sid, auth_token)client.messages.create(
to="YOUR PHONE NUMBER HERE",
from_="YOUR TWILIO NUMBER HERE",
body="Call {}. Message: {}".format(caller_number, speech_result))

Then, based on the number and transcribed message, I can text back “yes” to be connected if it looks like a call worth taking.

Step 5: Forward the call to me if I approve.

Once a positive response is received the call needs to be patched through to my real number. Since I have my Twilio number saved in my contact book, the call will go through and ring on my personal line. Twilio Labs hosts a small app that allows for simple call forwarding. The URL structure is simple, I just substituted my own phone number and recorded a voicemail in case the call fails.

from twilio.twiml.voice_response import Dial, Number, VoiceResponse

response = VoiceResponse()
dial = Dial()
dial.number('YOUR PHONE NUMBER HERE')
response.append(dial)

print(response)

Fortunately, very little customization is needed for this step.

Step 6: Ask to take a message if I don’t want to talk.

If I’m busy and don’t respond to the text, Twilio then prompts the caller to leave a message. I think 60 seconds is the longest amount of time you want to put an unknown caller on hold, so the Twilio script waits 60 seconds for a response before asking to take a message.

Step 7: Transcribe the message and send to my email.

Just like before, the voice message is transcribed using Twilio’s speech recognition service and then emailed to me using Python’s simple smtplib. Sending email with Python is well documented, so I won’t go into detail about how to do that here.

Next Steps:

I would love to turn this into a self-service app in the future and share it with other people who are interested, but there’s a bit too much on my plate at the moment and I’m sure companies will start adding these features to phone systems in the near future.

If you want a more basic version of this, you can use Google Voice to screen calls and get some of the same functionality, although it’s far less customizable. The Google Voice call screening solution still causes your phone to actually ring (vs. a much less invasive text message).

I can definitely see how some people might consider this overkill, but I think it’s appropriate for our current societal communication patterns. Increasingly, the legacy phone system is being abused for spam and most people have already shifted to texting or a variety of social platforms as the dominant form of communication. I would love for the FTC to roll out a great solution for this problem, but I just don’t see it happening fast enough. Until then, I’ll probably keep hacking my phone until I have the least amount of tolerable distraction in my life.

Image for post
Image for post

Thanks for reading! Please hit the clap button if you found this interesting!

You can connect with me on Twitter: www.twitter.com/johncoogan

HackerNoon.com

#BlackLivesMatter

Sign up for Get Better Tech Emails via HackerNoon.com

By HackerNoon.com

how hackers start their afternoons. the real shit is on hackernoon.com. Take a look

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

John Coogan

Written by

Co-founder of @Soylent, was CTO there for ~5 years. I write about technology, marketing, ecommerce, food, productivity, and anything that can be hacked.

HackerNoon.com

Elijah McClain, George Floyd, Eric Garner, Breonna Taylor, Ahmaud Arbery, Michael Brown, Oscar Grant, Atatiana Jefferson, Tamir Rice, Bettie Jones, Botham Jean

John Coogan

Written by

Co-founder of @Soylent, was CTO there for ~5 years. I write about technology, marketing, ecommerce, food, productivity, and anything that can be hacked.

HackerNoon.com

Elijah McClain, George Floyd, Eric Garner, Breonna Taylor, Ahmaud Arbery, Michael Brown, Oscar Grant, Atatiana Jefferson, Tamir Rice, Bettie Jones, Botham Jean

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store