How to scale AWS Pinpoint to notify hundreds of customers about HiveNightmare
--
We recently released our mobile app, built with React Native, and are adding push notifications functionality so that our customers can get the info they urgently need on how to stay safe from bad things. This is a quick synopsis of our learning journey into building this functionality.
Our first decision was whether to build or buy a managed service for push notifications. The team had a collective engineering goal: get to market faster whilst focusing on coding customer value, which excluded building. We considered multiple providers (OneSignal and others) and ultimately landed on AWS Pinpoint. We’re a heavy AWS shop, and the AWS SDK integrates with React Native; not needing to integrate yet another SaaS product made choosing Pinpoint easy.
The AWS pinpoint segment limit is 100 custom fields or dimensions (in our case, the company unique identifier is the custom field): https://docs.aws.amazon.com/pinpoint/latest/developerguide/quotas.html#quotas-segment/.
We overlooked the segment limitation (customer unique identifiers) when designing our push notification enqueuing system. The system we used to enqueue notifications to customers lacked a batching mechanism. This potential issue did not surface in testing as the 100 dimensions per segment constraint fell within our bounds for dev/beta tests of a select dozen customers.
When we expanded our implementation to all customers (more than the limit), the push notifications were successful multiple times! This may be due to a soft limit set by AWS — we have a ticket open with AWS to understand why. Our CTO was to perform the push notifications manually (he insisted, and it aligned with our shared goal of getting to market faster) while we developed the automation. Of course, the limit blocked the functionality when our CTO sent a push notification to all customers.
Looking into the logs, we realized that the segment dimensions limit was responsible for blocking the push notifications. We batched the customer unique identifiers segment list into batches of 100 dimensions. We shared the Typescript code here to help others with our lesson learned (read the docs!):
The full AWS Pinpoint code can be found on our Github repo: https://gist.github.com/myousseif/8a6945069f063c9d434ed9a282d0ba11
We love AWS; learn with us, and come join the team!
