Tweeting Images by Dispatch with AWS Cloud Events

Dispatch supports cloud events, and you can also create your own event driver to trigger functions. Let’s create something to show off those features.

Here are the demo workflows:

User needs to grant secrets to Dispatch event driver, and driver itself will create required aws resources and establish connections. Once user uploads image to s3 bucket, event driver will fetch the event and send it to Dispatch. Then Dispatch will invoke function to watermark the image and tweet it out.

Prerequisites

1. A running Dispatch instance (quickstart)
2. Dispatch CLI (latest release)
3. AWS account and Twitter Apps

Setup AWS S3 Bucket And CloudTrail

The very first thing we need is a s3 bucket. Follow this instructions to create a s3 bucket. And take a note of region and bucket name.

Besides, we also need to create CloudTrail which required by aws for events to be caught. Follow this instructions to setup CloudTrail for your s3 bucket. And make sure that in Data Events section, add the s3 bucket you just created. If you want to upload the images to some specific folder in s3 bucket, for example images folder, remember to put images in prefix for Data Events.

If CloudTrail was setup properly, you should see Object-level logging is enabled in your s3 bucket -> Properties.

Setup Dispatch Eventdriver

What’s next? We need an event driver to fetch aws s3 image uploading event and send it to Dispatch for later processing.

Create AWS Secret

In order to connect to AWS services, we need to retrieve the aws secrets. Put them in aws-secret.json:

Then, create a secret named aws:

$ dispatch create secret aws aws-secret.json
Created secret: aws

Create Event-driver-type

You can find a prebuilt event driver image in Docker Hub dispatchframework/dispatch-events-aws. Let’s create event-driver-type using that prebuilt image:

$ dispatch create eventdrivertype aws dispatchframework/dispatch-events-aws:latest
Created event driver type: aws

Or, if you prefer to build image yourself. The event driver repo contains all driver codes and Dockerfile:

$ docker build -t dispatch-events-aws:latest .

Create Event Driver

Event driver will need some event patterns for CloudWatch rules, details can be found here. We will use this template for our demo:

And make sure to replace <bucket-name> section to the s3 bucket you just created.

Now, we have everything we need to create the real event driver:

$ dispatch create eventdriver aws --secret aws --set clean-up --set region="us-west-2" --set event-patterns='[
    {
        "source": [
            "aws.s3"
        ],
        "detail-type": [
            "AWS API Call via CloudTrail"
        ],
        "detail": {
            "eventSource": [
                "s3.amazonaws.com"
            ],
            "eventName": [
                "PutObject"
            ],
            "requestParameters": {
                "bucketName": [
                    "dispatch-events-aws-demo"
                ]
            }
        }
    }
]'

Some explanations:

• We are creating an event driver of type aws.
• We need the aws secret to talk to aws services.
• We want to clean-up the aws resources after event driver shuts down.
• We use us-west-2 as region, change this to your aws region if different.
• We define event-patterns which is a json array string, if you have more event patterns, put them together in a json-array-style. Examples are here. And the bucket name we used here is dispatch-events-aws-demo. Change it to your bucket.

If everything goes well, we should now see the event driver is ready. That means the event driver finishs initialization of required resources, starts fetching events from aws SQS and sending them to Dispatch.

Setup Dispatch Function

Now we have a running event driver that is sending events to Dispatch, but we don’t have the function to process events yet. So, let’s go to create a function to watermark image and publish it to Twitter.

Create Twitter Secret

To tweet something, we need credentials from Twitter Apps, create a new twitter app and get the credentials. Put them in twitter-secret.json:

Then, create a secret named twitter:

$ dispatch create secret twitter twitter-secret.json
Created secret: twitter

Create Base Image and Image

Before creating the function, we need to create base image and image first.

$ dispatch create base-image python3-base dispatchframework/python3-base:0.0.7 --language python3

After base-image is ready (check by dispatch get base-image), create image:

$ dispatch create image python3-twitter python3-base --runtime-deps requirements.txt

In requirements.txt, we specify the runtime dependencies for our function:

Create Function

Next comes to the real function creation. Download the function code in twitter.py file. After base image and image are both ready, create the function twitter by:

$ dispatch create function twitter --image python3-twitter ./twitter.py --secret twitter --secret aws

Function twitter will consume aws secret for downloading image from s3 and twitter secret for tweeting.

Create Subscription

Now we have event driver and function ready, the last missing piece is that we need to create a subscription in Dispatch to make a bridge between incoming events and function invocation:

$ dispatch create subscription twitter --event-type="aws.s3"

Here, we specify the target function twitter which we just created, also make sure the event-type is aws.s3. After doing this, the last missing piece is done.

Let’s Upload Images

Finally, we finish the workflow. Let’s upload some image to test it.

Go to the s3 bucket we just created, upload some pictures and wait them to be tweeted to your Twitter!

Also remember, if you specified prefix in CloudTrail, make sure your upload matches that prefix, either it is a folder prefix or file name prefix.