Deploying a Node.js Twitter Bot on AWS Lambda using Webpack

I recently had an idea for a Twitter bot for the #LearnInPublic community, which I thought would be a fun project with half a day worth of work. Moreover, I wanted to use AWS Lambda function for the sake of learning. Why AWS Lambda? Because it’s practically free for maintaining a twitter bot and I got hands on experience with a seemingly complex tool used widely in the industry. You get 1,000,000 free requests and 400,000 seconds of compute time per month for AWS Lambda. My bot uses 0.02% of the above numbers.

In the process I found a cool use-case Webpack. I had fun reading the webpack docs and figuring learning about the gotchas. Even though it was a very primitive use-case, felt like a true Sean Larkinn for a day. Here is a detailed article on how you could do the same.

A little about the bot itself.

About half a year ago when Shaun Weng wrote about learning in public, it resonated with me like nothing had ever before. It felt like a nudge in the right direction and I ended up saving the post to read it every now and then for motivation. I thought it’d be cool to write a bot which tweets the same swyx post periodically (so that it keeps resurfacing on my feed) and retweet all the posts with the #LearnInPublic hashtag concentrating the community’s work under a single roof (profile).

Prerequisites

Goes without saying, you’ll need Node.js installed on your computer, a Twitter account for the bot, and an AWS account. This tutorial also assumes you’re familiar with node and won’t go into great depth of the bot’s workings.

The tutorial is divided into three main sections

1. Setting everything up (Twitter, AWS Console, Bot config).
2. Writing the bot and building it with Webpack.
3. Deploy with AWS Lambda!

SETUP

This section handles all of the account setups you need to take care of for each of the platforms that you’ll be using.

• Twitter

You’ve to make a new account for the bot which is going to responsible for posting/retweeting on its profile. Once you’re done with that goto https://developer.twitter.com/ and setup a developer’s account.

Create a new app at https://developer.twitter.com/en/apps/create by filling in the appropriate information. Once the app is successfully created, note down the API keys and access tokens.

• AWS

You should have an AWS Account to proceed, If you don’t, you can create one at https://aws.amazon.com/account/. Once done we can proceed with he setup for the lambda function(s).

I was using AWS for the first time so I’m not sure If I am understood the Identity tools (IAM Users and Roles) correctly or If I used them right, however these are the steps I took from my best understanding of the docs and it does work, so here goes nothing ¯\_(ツ)_/¯ .

When you make an account on AWS and sign in using the email/password, you are logging in as the root user. The root user has complete access to all AWS services and resources, hence it is recommend to not use the root user for everyday tasks. For greater security and organisation, you can give access to your AWS account to specific users — identities that you create with custom permissions. This is where AWS IAM comes into play.

AWS Identity and Access Management (IAM) enables you to manage access to services and resources securely. Using IAM, you can create and manage AWS users and groups, and use permissions to allow and deny their access to AWS resources.

Go to the AWS IAM Console to create a new User. Name it such that you Identify what it’s purpose is (e.g twitter-bot). Make sure the User has AWSLambdaFullAcces policy attached to it.

Additionally also create a IAM Role, IAM roles are a secure way to grant permissions to entities that you trust. You can create a role from the same AWS IAM Console. Select Lambda as the service which will use this role.

You’ve to attach the same permission policy of AWSLambdaFullAcces for this Role as well. Give it an appropriate name (e.g lambda-twitter-bot), description and create the role.

• Twitter Bot

I used the npm package twit which is a Twitter API Client for node to interact with the Twitter API. Once you’ve written the bot, you’ll need to upload a compressed zip file of the folder with code to the AWS Lambda console.

Instead of uploading the entire node_modules which may have dev dependencies and other config folders adding to the bloat, we will bundle the app using Webpack and zip a single file for the upload.

Make a folder for your project and initialize your node project after changing into that directory. Then, install twit as a dependency for your project.

mkdir twitter-bot
cd twitter-bot && npm init -y
npm install --save twit

Next, we need to setup the API Keys we acquired from twitter for both the development and production environments. We are going to maintain a dev.js with local copy of the API keys and prod.js to access the production environment variables bound to the same API keys, that we will eventually set on the AWS Lambda Console. If you’re uploading your code to GitHub, make sure you don’t commit dev.js , add it to .gitignore. Depending on the environment we’ll export the appropriate file.

mkdir config && cd config
touch index.js
touch dev.js
touch prod.jS

index.js

if (process.env.NODE_ENV === 'production') {
module.exports = require('./prod');
} else {
module.exports = require('./dev');
}

dev.js

module.exports = {
consumer_key: 'YOUR CONSUMER KEY HERE',
consumer_secret: 'YOUR CONSUMER SECRET HERE',
access_token: 'YOUR ACCESS TOKEN HERE',
access_token_secret: 'YOUR ACCESS TOKEN  HERE'
};

prod.js

module.exports = {
consumer_key: process.env.CONSUMER_KEY,
consumer_secret: process.env.CONSUMER_SECRET,
access_token: process.env.ACCESS_TOKEN,
access_token_secret: process.env.ACCESS_TOKEN_SECRET
}

Writing the bot and building it with Webpack

For my bot I needed two different functionalities (tweet and retweet) to be triggered by AWS Lambda at different time intervals. Let’s proceed with only one of them as an example. (If you have a use-case similar to mine just deploy two lambda functions like I did). The example is of a bot which retweets posts a certain hashtag, but doesn’t retweet it’s own posts.

The format for an AWS lambda function is as follows, hence you will need to export your function:

exports.handler = (event, context, callback) => {
   //function logic
}

Fun fact: Don’t use a forEach loop as they don’t work with asynchronous code. Here’s a great article on the same.

I’ll not dive too deep into the working of the bot’s code, it’s quite straightforward if you read the docs for twit. You query for all the recent tweets with the given hashtag. On receiving the response, you de-structure out the all the statuses, loop over them and retweet them. The for .. of loop works well with the asynchronous async/await.

You can test the code by entering the Node.js console.

> node
> let retweeter = require('index.js');
> retweeter.retweet();

WEBPACK!!!!!!

Next we bundle the code for easier deployment to AWS Lambda because we don’t need no node_modules in our zip.

Webpack is a static module bundler for modern JavaScript applications. When webpack processes your application, it internally builds a dependency graph which maps every module your project needs and generates one or more bundles.

Webpack v4 is easier to use than ever before, and does not even require a config file by default. It can look daunting to beginners (like it did to me for the longest time) but it is very well documented and it’s easy to find your way around it.

Step 1: Install Webpack as dev dependencies.

npm install --save-dev webpack wepback-cli

Step 2: Modify package.json to add the build script.

//package.json
{
  "name": "twitter-bot", 
  "version": "1.0.0",
  ...
  "scripts":{
    "build": "webpack",
    ....
  },
  ......
  "devDependencies": {
    "webpack": "^4.29.0",
    "webpack-cli": "^3.2.1"
  },  
  "dependencies": {
    "twit": "^2.2.11"
  }
}

Step 3: Add Webpack config.

Add webpack.config.js to the root level of your project directory.

//webpack.config.js
const path = require('path');
module.exports = {
    entry: './index.js',
    output: {
      path: path.resolve(__dirname, './dist'),
      filename: 'index.js',
      libraryTarget: 'commonjs'
    },
    target: 'node',
    mode: 'production'
}

Here’s a list of what all the parameters are for

Note: If you have more than one functions with different configuration (like I did as I needed two different outputs for different lambda functions) you can export multiple configurations by exporting an array of config objects.

module.exports = [{
  //config 1
},{
  //config 2
}]

Entry

An entry point indicates which module webpack should use to begin building out its internal dependency graph. Webpack will figure out which other modules and libraries that entry point depends on (directly and indirectly).

The entry point for this bot is index.js .

Output

The output property tells webpack where to emit the bundles it creates and how to name these files.

output.pathtells webpack where to emit the bundle andoutput.filename is used to specify the bundle name.

Since index.js exports an object with the retweet function as per AWS specification, our bundle should do the same. You can test this behaviour in the node console.

> var library = require('./index.js');
> library
{  retweet: [AsyncFunction: hashtagRetweet]  }

output.libraryTarget: "commonjs" makes sure that in the bundle itself, the function that is generated by webpack from the entry point will be assigned to the exports object.

Target

Instructs webpack to target a specific environment. Use node to compile for usage in a Node.js-like environment

Mode

By setting the mode parameter to either development, production or none, you can enable webpack's built-in optimizations that correspond to each environment. The default value is production.

And that’s about it for webpack! Whip up your terminal and npm run build to generate your production grade bundle in ./dist . Zip up the dist folder as the Lambda console only accepts .zip files.

> npm run build
> cd dist
> zip bot.zip index.js

Deploy using AWS Lambda!

At the final stage of our journey, we’ll make this bad boy live! We want to upload our lambda function and make sure it is triggered in set intervals. The CloudWatch Event which comes with AWS is the perfect solution for it. The CloudWatch Event trigger lets you schedule it’s your function calls. It is basically a cronjob for your lambda function.

• Go to your Lambda Management Console, you can access it from Services in your navbar. Create a new lambda function there.

• Give it a suitable name, choose the Node.js 8.10 Runtime and choose the Role you created earlier in the setup stage. New features such as async/await syntax works because the LATEST node runtime is supported by AWS Lambda. In the lambda console you’ll have various options to trigger your function.

• Upload the zip file from the dist folder. Change the filename to [filename].[function name] In this example, it’ll be index.retweet

• Next we add the production environment variables. Scroll down to find the section to input environment variables. Open you prod.js file and copy the environment variables one by one. Pay special attention and make that sure you copy the exact variable name.

• Finally, add a CloudWatch Event trigger in the function Designer. Select the CloudWatch Event and scroll down to configure it. Create a new rule where you declare the rate at which your function should be called.

//Syntax:
rate(Value Unit)
// Value is a positive number
// Unit can be minute | minutes | hour | hours | day | days

E.g, For a duration between function calls to be 5 minutes, the expression should be rate(5 minutes)

Save the function and your bot should be ready to go! Test out your function with the Test button on top. Out bot’s function doesn’t take any parameters. Your bot will be up and ready, doing its magic and saving you time and money.

In case you wan’t some metrics for your lambda function, the CloudWatch metrics under the monitoring tab provides some cool charts on invocation, duration, errors, availability etc to give you an estimate of your function’s performance. You can also read CloudWatch Logs that captures console.log as well, this might help you debug stuff.

BUT YOUR BOT IS UP AND RUNNING TWEETING WITHOUT ANY HUMAN HELP AT THIS POINT! BE PROUD OF YOURSELF! THANKS FOR READING

I am a web and mobile developer based in India. I love React/React Native/JavaScript. I write about my adventures with tech, projects and Hackathons. If the article helped you, smash that like button (jk but you can clap em).

I’m sumdook on Twitter, if you want to me talk about React, tech, JS, football, anime, eSports and just about anything. I’m also redoing my website and I’ll link it as soon as I finish,Cheers.