Handle high load traffic with AWS and Go
Scalability is the one of the major and prevalent concern of modern software industry. There are huge amount of data and traffic and if you want to survive in this continuously growing/changing/evolving industry, you should be equipped with the right tools in your tool bag. Moreover, if your business runs in the Ad Tech industry, huge amount of traffic and data come into tangible existence. That’s why, in this blog post, I would like to give some details about how we made architectural decisions to handle 10Ks of requests per second.
Interceptd is a mobile fraud detection tool in Ad Tech industry. Customers send their clicks/installs/events, etc. via Rest APIs and they are warned via Rest APIs when the Interceptd filters, detects and prevents any type of fraud. Thus, integration between two products should perform well even in the high load traffic.
First Phase(Performance Issues)
At Interceptd, our Rest API returns responses within ~20.0 milliseconds as shown below;
And our customer has lots of tasks implemented in PHP and running on AWS EC2s and whole traffic is drained into that tasks. In our initial step of integration, they had sent their whole traffic to Interceptd’s Rest APIs synchronously in a single thread. After a while, we realized that their task count increased 4 times, due to enabled auto scaling feature. This was not a cost effective and perpetuated in the long term and we did not tend to change tasks from single thread to multi-threaded application, due to the already running tasks in production environment should have worked without being changed its architecture drastically . Nevertheless, we tried the performance of multi-threaded version of the integration and anyway, it did not perform well (~10K/minute).
Second Phase(Loose Coupling with S3/SNS)
The first attempt and decision we were made was separating integration from the core parts of the tasks to assure the loose coupling. Thanks to AWS S3, whose response time is very low and resilient service, we drained whole traffic to seperate AWS S3 buckets and SNS topics have been subscribed to S3 buckets’ creation events (Current architectural diagram shown below). Furthermore, due to the tentativeness of the traffic, we decided to upload traffic 6 times per minute in bulk files. Hence, we could have a chance to manage the flow uniformly.
Final Phase(AWS Lambda/Go)
After decoupling tasks and integration feature, we had a chance to choose appropriate programming language and stack to overcome the performance and cost effective scalability problems. 10Ks of requests per second need concurrency and Go’s builtin concurrency seemed a good choice to tackle such high volumes of data and traffic. Moreover, AWS Lambda can be triggered from SNS events (because of its uncountably large scalability and cost effectiveness, not because of the serverless hype in the software industry). It performs well as shown below diagrams;
And it can be clearly seen in the below diagram, AWS Lambda with Go completes its process (~1200/sec) within ~2 seconds.
One point should be beared in mind that if you invoke all traffic in different goroutines, AWS Lambda throws “socket: too many files” error, because you can not increase ulimit in AWS Lambda (limit of maximum connection). Therefore, you have a limit of goroutines you can create. For example, If you try to send 1500 requests in different goroutines, you will get an error. We send these 1500 requests with dividing it 10 within 150 goroutines.
Conclusion
Go is a powerful programming language with its concurrency support(readability, fast execution time, etc.) and when it is combined with the AWS Lambda, it performs well for tasks have low tolerant to latencies with fine-grained goroutine counts.
You can reach out template Go code which consumes SNS events triggered from S3 via Github link below.
