Building a Serverless Data Pipeline with AWS S3 Lamba and DynamoDB

Yi Ai
Yi Ai
Mar 5, 2019 · 3 min read
Image for post
Image for post
Image for post
Image for post
$ serverless create --template aws-python --path data-pipline
$ serverless plugin install -n serverless-python-requirements
plugins:
- serverless-python-requirements
custom:
pythonRequirements:
dockerizePip: non-linux
layer: true #Put dependencies into a Lambda Layer.
functions:
importCSVToDB:
handler: handler.importCSVToDB
layers:
- {Ref: PythonRequirementsLambdaLayer}
environment:
documentsTable: ${self:custom.documentsTableName}
bucketName: ${self:custom.s3bucketName}
events:
- s3:
bucket: ${self:custom.s3bucketName}
event: s3:ObjectCreated:Put
rules:
- suffix: .csv
$ sls deploy --stage dev
$ aws s3 cp sample.csv s3://dev.document.files

HackerNoon.com

#BlackLivesMatter

By HackerNoon.com

how hackers start their afternoon. the real shit is on hackernoon.com. Take a look

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Yi Ai

Written by

Yi Ai

AWS Community Builder, AWS Certified Data Analytics Specialty, AWS Certified Security Specialty, AWS Solutions Architect Professional, AWS DevOps Professional

HackerNoon.com

Elijah McClain, George Floyd, Eric Garner, Breonna Taylor, Ahmaud Arbery, Michael Brown, Oscar Grant, Atatiana Jefferson, Tamir Rice, Bettie Jones, Botham Jean

Yi Ai

Written by

Yi Ai

AWS Community Builder, AWS Certified Data Analytics Specialty, AWS Certified Security Specialty, AWS Solutions Architect Professional, AWS DevOps Professional

HackerNoon.com

Elijah McClain, George Floyd, Eric Garner, Breonna Taylor, Ahmaud Arbery, Michael Brown, Oscar Grant, Atatiana Jefferson, Tamir Rice, Bettie Jones, Botham Jean

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store