Alex Casalboni
Apr 8, 2019 · 1 min read

I agree with you, Yan Cui.

It’s worth highlighting that a trailing “\n” is needed only with JSON records, and only if you want to use Athena later in your pipeline.

My personal guess is that many other customers are using other data formats such as CSV/Parquet, which don’t require a trailing “\n”. And I believe Firehose does not inspect the record content to understand which format is being used.

Plus, some other less common use cases might require a different object separator (see https://en.wikipedia.org/wiki/JSON_streaming).

What you be ok with having a configurable separator option in your Firehose Delivery Stream?

    Alex Casalboni

    Written by

    Italian | Musician | Traveler | Technical Evangelist @ AWS https://aws.amazon.com/developer/community/evangelists/alex-casalboni/ Opinions are my own.