Reliable, Fail Safe and Fast Campaigns
We at ShoutOUT Labs recently released a new version of our campaign management service. Even if you are an existing customer of ShoutOUT, sometimes you might not have noticed this because this is more of a back-end update and we didn’t do much of a change in front-end except for adding few icons for the campaign status.
I thought of sharing some technical info about this new release so that you can understand why we claim the new version to be reliable, fail-safe and fast. Before all that, let me walk you through what a campaign is in ShoutOUT in case you haven’t used it.
A campaign is a process where you can send a message to an audience, based on different criteria. Simply put, using campaigns you can type one single message (either sms or email, or both) in one place and send it to a group of recipients instantly (broadcast campaign), on a scheduled time (broadcast scheduled campaign), or each time when a user does a certain action (triggered campaigns). Although this sounds very simple, there is a series of processing happening at the back-end for a single campaign as briefed below.
Validating campaign info
The new campaign request data from the front-end need to conform to a predefined structure and should contain values of allowed types only.
Building the message (if a personalized message, a different message for each recipient)
Generates the message replacing dynamic parameters with parameters of each contact and removing unsupported characters
Charging for the message
Based on the message type (sms or email), debits credits from the merchant account
Sending the message
Based on the transport type and receiver, sends the message through the appropriate sms or email gateway
Logging the message
Combines the charging and sending results, and store in a database for future traceability
Generating the report
Builds the campaign report aggregating all the message sending and charging reports
Above is a series of ordered tasks that we have to run in order to complete a campaign and we can model this as a workflow. There are some important aspects that we have to be concerned about when practically implementing a workflow.
- Coordination between each task
- Persisting messages in case of task failures or unavailability
- Execution time (how long it takes to complete)
- Handling throttling for 3rd party service access and I/O operations
- Monitoring progress of each task (whether started, running, successful or failed)
- Possibility to easily terminate in case something goes wrong
- Support for multiple parallel workloads (parallely executing multiple workflows)
- Optimization of computing resources
In our legacy campaign management service we did not address all above since it is indeed a lot of work when implementing all these from the scratch. Moreover, it was out of the scope of our business purpose to implement all these ourselves. So we looked at several options for frameworks and managed services which implement workflow models, and finally decided to go with Amazon Simple Workflow Service. I am not going to mention why we selected this in this article since it’s a different topic.
Amazon SWF is a fully-managed state tracker and task coordinator in cloud. It is reliable, scalable, and nicely separates the control flow logic. It has a lot of features which took away most of our pain points. Of course there was a component that we had to implement, but it was minimal and contained our business logic only. However, using SWF was not a blue ocean; you probably know this if you have used it before. In our case, because of the complexity and scale we wanted to achieve, it was really challenging. I’ll share more info about our experience with SWF separately in my personal blog. The diagram below shows a very high-level picture of the logical architecture of our campaign workflow architecture.
Finally, what we needed to connect to the campaign workflow service from front-end, was the API endpoint. For this we no longer required to use a hosted API, instead we decided to go serverless with AWS Lambda and AWS API Gateway using the Serverless Framework which is a hot topic these days. This further enhanced the availability and reliability of our campaign back-end since the new campaign requests from the ShoutOUT dashboard directly go to api gateway, lambda and land in swf. So, even if our workflow workers are offline, your messages won’t get lost. The diagram below shows an overview of how each of these components related to the campaign service connects with each other.
I am sure that you feel safer now with ShoutOUT campaigns and probably want to give it a try. The ShoutOUT team understands how important your messages are for you and your customer.
