Italian | Musician | Traveler | Technical Evangelist @ AWS https://aws.amazon.com/developer/community/evangelists/alex-casalboni/ Opinions are my own.
A very common alternative would be adding the separator directly in the client/producer (not ideal, but free). Or with a velocity template in case you are using API Gateway (with a service integration to put records into Firehose).
I agree with you, Yan Cui.
It’s worth highlighting that a trailing “\n” is needed only with JSON records, and only if you want to use Athena later in your pipeline.
My personal guess is that many other customers are using other data formats such as CSV/Parquet, which don’t require a trailing “\n”…
Nice library :)
I think the Python interface is very intuitive and will simplify a lot of workflows.
Regarding the architecture, have you considered implementing it using AWS Step Functions? Imho, it would be much simpler and cheaper as well.
Hi Kalin :)
Thanks for writing this, Dariusz!
SSM Parameter Store is really powerful, and I’ve recently published a Python wrapper for it: https://github.com/alexcasalboni/ssm-cache-python
(It handles caching, invalidation signals, list conversion, paths, etc.)