This Robot Fights Back Against Robocalls

How I used Twilio and a few lines of Python to stop all unwanted callers permanently.

John Coogan
Sep 19, 2017 · 6 min read

I, like most people, am annoyed by telemarketers. But recently the problem has gotten so much worse due to the automation of scam calls. This was and investigated extensively by Reply All in an amazing 2 part podcast (available below). The Planet Money piece shows that these robocalls are unlikely to end anytime soon, since the FTC is moving very slowly on the issue. Then, right in the middle of the piece, they played an ad for Google Cloud Platform and I got an idea. I could make a bot that would screen for unwanted calls and only let quality calls through. Like a digital personal assistant.

When it comes to blocking just Robocalls, there are several apps that do a decent job at an affordable cost. I wanted to go a step further though and put a firm line of defense in between myself and unwanted calls.

How the bot works:

  1. Calls come in to my normal phone line.
  2. If the number is in my contacts the call goes through automatically.
  3. If not, the bot answers and asks what the caller is calling about.
  4. The caller is then placed on hold, while their answer is texted to me.
  5. If the call looks legit, I text back “ok”, and the bot connects me.
  6. If I don’t respond, the caller is asked to leave a message.
  7. Messages are transcribed and sent to my email.

Breaking Down Each Function:

Step 1: Maintaining a normal phone line.

Image for post
Image for post

Putting software between your phone and the world is a lot harder than it should be. offers a great interface for creating programatic phone systems, but is a six step process that can take up to four weeks.

A better option, in my opinion (if you have an iPhone), involves creating a new number for Twilio, which you will use publicly, and then restrict your personal phone line to only accept calls from numbers in your contact book. You can restrict unknown numbers from making your phone ring by setting Do Not Disturb mode to allow calls from your contacts and silence all other numbers. The key here is to set the schedule from 10:00PM to 9:59PM so Do Not Disturb mode will stay on permanently.

This, in and of itself, is a good start and will help clean up unwanted calls. But it’s a bit of overkill, so setting up a new number that you can use publicly to forward calls to you based on criteria you set is a good next step.

Step 2: Check Contacts via Google API

If you sync your iPhone contacts with Google, accessing your contact listing via API is relatively simple. You just need to from the Google People API. I find it best to crawl this for updates daily and save all the “known numbers” to a JSON file for easy access (it’s just a list of phone numbers that will be passed through to your real number).

When a new call comes in to the Twilio number you have setup (and given out freely), Python opens the JSON file with all the stored numbers and checks if the caller is in the list. If so, the call is automatically forwarded and the caller is unaware of this process, if not though, the caller will be prompted to explain who they are and why they are calling.

Step 3: Prompt caller for explanation via Twilio Speech Recognition

Image for post
Image for post

Twilio offers that allows you to prompt a caller with a message and then automatically record and transcribe their response.

from twilio.twiml.voice_response import Gather, VoiceResponse, Say

speech_result = VoiceResponse()
gather = Gather(input='speech', action='/completed')
gather.say('You have reached John's automated answering service, please state your name and why you are calling.')

This will return a transcribed SpeechResult object for further handling. There is the opportunity to inject some machine learning into this step to understand why someone is calling, maybe correlate that with my calendar or recent emails and handle the call accordingly, but that is out of scope of this post.

Step 4: Text me the transcription via Programmable SMS

Sending this result to me via text is trivial with Twilio. The python code looks like this:

client = Client(account_sid, auth_token)client.messages.create(
body="Call {}. Message: {}".format(caller_number, speech_result))

Then, based on the number and transcribed message, I can text back “yes” to be connected if it looks like a call worth taking.

Step 5: Forward the call to me if I approve.

Once a positive response is received the call needs to be patched through to my real number. Since I have my Twilio number saved in my contact book, the call will go through and ring on my personal line. The URL structure is simple, I just substituted my own phone number and recorded a voicemail in case the call fails.

from twilio.twiml.voice_response import Dial, Number, VoiceResponse

response = VoiceResponse()
dial = Dial()


Fortunately, very little customization is needed for this step.

Step 6: Ask to take a message if I don’t want to talk.

If I’m busy and don’t respond to the text, Twilio then prompts the caller to leave a message. I think 60 seconds is the longest amount of time you want to put an unknown caller on hold, so the Twilio script waits 60 seconds for a response before asking to take a message.

Step 7: Transcribe the message and send to my email.

Just like before, the voice message is transcribed using Twilio’s speech recognition service and then emailed to me using Python’s simple smtplib. , so I won’t go into detail about how to do that here.

Next Steps:

I would love to turn this into a self-service app in the future and share it with other people who are interested, but there’s a bit too much on my plate at the moment and I’m sure companies will start adding these features to phone systems in the near future.

If you want a more basic version of this, you can use to screen calls and get some of the same functionality, although it’s far less customizable. The Google Voice call screening solution still causes your phone to actually ring (vs. a much less invasive text message).

I can definitely see how some people might consider this overkill, but I think it’s appropriate for our current societal communication patterns. Increasingly, the legacy phone system is being abused for spam and most people have already shifted to texting or a variety of social platforms as the dominant form of communication. I would love for the FTC to roll out a great solution for this problem, but I just don’t see it happening fast enough. Until then, I’ll probably keep my phone until I have the least amount of tolerable distraction in my life.

Image for post
Image for post

Thanks for reading! Please hit the clap button if you found this interesting!

You can connect with me on Twitter:




how hackers start their afternoons. the real shit is on Take a look

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

John Coogan

Written by

Co-founder of @Soylent, was CTO there for ~5 years. I write about technology, marketing, ecommerce, food, productivity, and anything that can be hacked.

Elijah McClain, George Floyd, Eric Garner, Breonna Taylor, Ahmaud Arbery, Michael Brown, Oscar Grant, Atatiana Jefferson, Tamir Rice, Bettie Jones, Botham Jean

John Coogan

Written by

Co-founder of @Soylent, was CTO there for ~5 years. I write about technology, marketing, ecommerce, food, productivity, and anything that can be hacked.

Elijah McClain, George Floyd, Eric Garner, Breonna Taylor, Ahmaud Arbery, Michael Brown, Oscar Grant, Atatiana Jefferson, Tamir Rice, Bettie Jones, Botham Jean

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store