AWS Kinesis Data Streams using Python — Part 1

Writing into AWS Kinesis Data Stream using Python

How It Works — Amazon Kinesis Data Stream(Picture from aws)

With streaming data widely available, you may often need to store the large volume of incoming data before storing it in the database. AWS Kinesis is a platform for Ingesting and storing data streams before they can be subjected to further processing. Using Python writing into the data stream and reading from it is easier than what you’d think. Let’s find out.

Part 1 — Setting up kinesis data stream in aws

Console Work

Choose Kinesis service from AWS Management Console

At the AWS management console, search for kinesis and choose the option as shown in the image above.

I already have a data stream so it shows total data streams as 1 for me. For you it might be 0. Let us get our data stream up and running. Click on create data stream.

Next give a name to the stream and assign the number of shards that you want. More the number of shards, more data kinesis will be able to…