4 Reasons Why You Should Publish Monitoring Data Async in AWS Lambda

Learn the motivation behind Thundra’s async monitoring data publish structure

Serkan Özal
Jan 4, 2018 · 4 min read
Image for post
Image for post

Longer request duration:

Publishing monitoring data in the request itself in a synchronous way increases the execution duration. It adds an overhead of a few tens of milliseconds to your Lambda function execution duration in the best case. In the worst case, you can’t send your monitoring data if network fails or service outage occurs in the data receiver side. Even if you manage to send the data after retrying a few times, you would lose hundreds of milliseconds or even seconds by doing so. Increase in the duration means spending a lot more money as you pay per 100 ms. you use and degraded user experience which can cost even more in the long run.

Stateless environment:

Lambda functions must be stateless because a container which runs your Lambda function can be destroyed any time when it is not handling a request. Therefore, saving monitoring data in local storage to publish in batch is not a good idea.

Data publish failures:

Monitoring data receiver-side might not be available all the time. In this case, you will need to either retry the monitor data to send the request or just skip it silently.

  • You may skip sending the data if you can tolerate data lost. If the failures end up quickly, the losses might be acceptable depending on your system monitoring needs and expectations. But the failures might continue during hours (or even days). In this case, if there is a problem with your Lambda functions, you will have no idea, and this is not acceptable for most systems.

Access within VPC:

If you put your functions in a VPC and need to access internet to publish monitoring data, you need to define a NAT gateway and connect it to Internet gateway. This means that you need to do additional operations work on your side and NAT gateway might be a bottleneck because it’ll handle all monitoring data.

Publishing monitoring data asynchronously

Because of the reasons mentioned above, Thundra publishes monitoring data through CloudWatch. This is also considered as a best practice by AWS as explained in “Serverless Architectures with AWS Lambda” whitepaper .

Image for post
Image for post


Full Observability For AWS Lambda

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store