Getting Started with the Twitter Account Activity API (Beta)


The Account Activity API will replace User streams and Site streams APIs on Tuesday, June 19, 2018. This process will soon be mandatory in order to keep your applications running!
I was super excited to see that Twitter was coming out with a brand new Account Activity API that follows a webhook push model in favor of the current User streams API. The latter requires you to create an application that maintains a connection with the service for as long as you’d like to receive updates. For me, this meant creating Amazon ECS services using python containers. While it was pretty straightforward to implement, I am very excited at the opportunity to start building serverless applications using the Serverless Framework and further reduce my AWS costs by leveraging the webhook push model and only pay per request, rather than the cost to run a service 24/7.
Unfortunately, I ran into a few hurdles when trying to get up and running with the new API. This new Account Activity API is still in beta, so it is understandably a bit rough around the edges, at least from a documentation standpoint. After scouring the Twitter Community forums, it turns out I wasn’t alone, and I found that many others were facing the same challenges I did. To help out, I thought I’d share my steps (many of which were through trial and error), so forgive me if I’ve missed anything or am misrepresenting things — this is from my own experience and your mileage may vary.
1. Register a Twitter App
This is pretty straightforward, and if you’ve already been building Twitter applications to date, this step is no different than before. You’ll need to go to apps.twitter.com and register an application. Also, make sure you generate all applicable credentials, including an Access Token and Access Token Secret, which can be found on the Keys and Access Tokens tab.
You’ll also want to note the app ID of this application. This can be located in the URI of the application management portal you’re in. It will be in the form of https://apps.twitter.com/app/<app ID>. Copy this down as you’ll need it later!
2. Create a Twitter Developer Account
Next, you need to create a Twitter developer account. This is different than requesting access to the API, although the process looks very similar to one another. This tripped me up my first time around. So fill out that form, being sure to include details around what you’re trying to build, the App ID from step 1, and state that you want access to the Account Activity API beta. I think I left that last part out in my first run, and thus needed to move on to step 3. Note: The turnaround for the developer account was 3 days for me.
3. Apply for Account Activity API (beta) Access
I’m guessing if you requested access in step 2, this step might be moot. However, I did not specify so and needed to explicitly request access by filling out this form. Note: I did not receive any sort of communication regarding the receipt of, or gaining access to, the API. I’m not sure what the turnaround is on this, but it wasn’t until I tried a few business days later that I realized I had access!
4. Build your Webhook CRC (Challenge Response Check)
Before you can successfully register your webhook endpoint with Twitter, you need to make sure you have a function that is capable of handling the Challenge Response Check (CRC). In short, upon registration of your webhook endpoint, Twitter will perform an HTTP GET to your endpoint with a crc_token parameter. Your function will need to parse this parameter, and calculate a response back along with your consumer key.
Below is an example Lambda function (python 3.6 runtime) I wrote that works. I’m storing the consumer key in AWS Systems Manager Parameter Store, and my Lambda function has an IAM role that allows it to use the appropriate KMS key to decrypt kms:decrypt, and access to the parameters ssm:GetParameters.
Also worth mentioning, I created a CloudWatch Event schedule to ping this function, with a short circuit at the top. I did this to keep the Lambda function “warm”, necessary as the CRC will fail if the response takes 1 second or more.
5. Register your Webhook
Now you need to register your webhook endpoint with Twitter. You do this by sending an HTTP POST to the API endpoint https://api.twitter.com/1.1/account_activity/all/:env_name/webhooks.json
You’ll need to pass the URL of your webhook as a parameter. Also, by default, the value of :env_name is env-beta. Here’s a quick and dirty python 3 example:
More details around this API and the possible response codes can be found here.
6. Subscribe your Application
Now that you’ve registered your webhook endpoint with the API, you need to subscribe your application to receive the webhooks from Twitter. Assuming you followed the example code above, subscribing is just as easy (make sure you are still using the same session from above):
7. Secure your Webhook
Before diving too deep into building your app, I recommend following the guidelines around securing your webhooks. I don’t see the need to repeat Twitter’s documentation, so just follow their guidelines!
That’s it! I guess it’s not all that complicated in the end, but the layout of Twitter’s documentation and having any sort of explicit “step 1, step 2, …” instructions are lacking.
To test, simply perform some action on your Twitter account like a DM, follow, Tweet, etc. You should see the webhook posted to your endpoint. To get started, I recommend printing the output of the JSON payload so you have a good idea of what you’re working with.
Hope that helps!