How to implement Node.js to format data sent by Kinesis Firehose with Lambda

Log formatting with Lambda
It is assumed that Kinesis Firehose forwards the log to S3, receives the log at Lambda, and formats the log. I will explain how to write Lambda processing at that time in Node.js.
Source Code
exports.handler = (event, context, callback) => {
const output = event.records.map(record => {
const data = Buffer.from(record.data, ‘base64’).toString(‘utf8’);
const parsedData = JSON.parse(data);parsedData.text = “add text”return {
recordId: record.recordId,
result: ‘Ok’,
data: Buffer.from(parsedData, ‘utf8’).toString(‘base64’)
};
});
callback(null, {records: output});
};
This is the process from receiving data until returning. Since it is written with ES 6, please upload source code after trans-piling with Babel. The function name is handler, but whatever name you use is okay.
Please refer to the following article for Transpail.
How to receive data
It can be obtained from event.records of the first argument. The data received from Kinesis Firehose will be multiple.
Since the base64 encoded data is passed, it is converted to utf8. Also, if you are sending in JSON you have to do JSON.parse.
In summary, the flow of this data conversion is below.
Binary (base64) -> String (UTF8) -> JSONHow to return data
When returning data, overwrite the changed data and set the character string ‘Ok’ to the result key. Is converted into binary data in base64 encoded to return to the same state as when it was last received is complete it will be rewarded by passing the second argument of the callback.
