Weekend Project: Building an SMS-based grocery list using Twilio and Firebase
Whenever I’ve tried to use my phone as a grocery list it’s never really worked out. Using the iPhone notes app was fine for a while but it was surprisingly annoying to select a specific item to delete (among other tiny problems). I didn’t want to download yet another app or open Chrome/Safari. I wanted something cross-platform. I wanted something that I could share with my girlfriend and my roommate(s) and my parents.
I settled on using text messages to create, read, and delete items from a persisted list (my grocery list). I used Twilio for messaging and Firebase for data storage.
For those not familiar with these services and APIs do:
- Twilio: A service that lets you control a phone number and pass through messages
- Firebase: a NoSQL database that lets you persist information
An early iteration involved using the following set-up:
When I sent a text message to the Twilio-created phone number, Twilio would process through that message and send it over to my Express server. My Express application would then communicate with a local JSON file and perform the appropriate action.
This worked fine, but the better solution would be to use serverless functions, functions that were able to be scaled and therefore allow multiple people to access the service without lag or crashing the server.
My next iteration used AWS Lambda serverless functions to handle each message and Firebase to persist my grocery list.
So now Twilio would send over the text message to my Lambda function which would process it and communicate with the Firebase database.
Literally less than a week after I had this setup running Twilio released Twilio functions, their version of Lambda functions. This greatly simplified my architecture and made it easier to debug as well, since I didn’t have to fiddle with headers and protocols for AWS Lambda.
This is the current set-up, and it’s worked for the past couple of months.
By the way, let me know if you’d be interested in using this service. The way it is set up is that I’ll add your phone number to a group, and this group shares a list. You can only be part of one group at a time. You can then text Trader Jay and automatically only edit the list you’re assigned to. If a random number tries to text Trader Jay they get added to a separate list and can’t use the functionality at all.
Pitfalls & Design Decisions
AWS Lambdas are notorious for cold starts where it takes a while for the function to be provisioned and start up. Thankfully there are a few ways to address this: you can upgrade the RAM or increase the time-out time for the request response.
One other option is the keep the functions warm by querying them every 20 minutes. But that would then take another function like a cron job and part of the reason for using serverless functions is to not worry about that. Increasing the RAM worked well enough.
Setting up the REST API properly on AWS Lambda took some twiddling. Twilio POSTs data in a certain format, and Lambda needs to handle that. The Twilio API also expects data back in a certain specific format (TwiML, similar to XML).
Using Firebase within the Lambda function out-of-the-box also posed a problem. Usually when a Twilio message came in you open up a connection to Firebase, run your function, and then respond to the Twilio message. What was happening though was that the connection to Firebase would stay open, which meant that the Lambda function wouldn’t close and would be unable to send a response back to Twilio within the allotted response time. I had to explicitly close the connection to Firebase after I had used its data.
I’m not sure how Twilio exactly handles its serverless functions, but I haven’t experienced any cold starts with it, and they were a breeze to set up.
There are a few things I’d like to improve:
- Message parsing. Currently, you need to say explicitly “add ___” for each item. I’d like to implement a better way to add multiple items while still allowing for commas, and be able to remove items without having to spell the name exactly as entered.
- Initialization procedures. Instead of manually editing the Firebase database with new phone numbers I should be able to have a form that lets people sign up automatically.
- Think a bit more on why text messaging is a good idea and if I really shouldn’t just use an app with things like checkboxes. I’ve heard that Google has good built-in support for family notes, etc.