Archive Tweets in Orchestrate with Node-RED

Tejas Manohar
Aug 22, 2015 · 5 min read
Image for post
Image for post

Even those of us who do not code have ideas and should have the ability to create. In October of 2013, IBM Emerging Technology released Node-RED, an open-source visual tool for wiring together the Internet of Things, including hardware devices, APIs, and online services. Node-RED allows programmers and non-programmers alike to produce Node.js, a JavaScript engine with an event-driven, non-blocking IO, services by connecting nodes, or service-specific pieces, together through a visual workflow builder.

In this tutorial, you’ll learn to use Node-RED with the Twitter API to store tweets containing certain keywords/#hashtags/@mentions in Orchestrate.


If you’re not familiar with Node-RED, I recommend that you read the entire “Getting Started” chapter as well as the “Writing Functions” section under the “Using Node-RED” chapter of the Node-RED official documentation. Once you have Node-RED up and running as well as the basics down, move right along.

Streaming Tweets

The first step in developing the service is using the Twitter API to create a stream, or an abstract interface for continuous flow of data, of filtered tweets that match certain parameters. To get started, drag the Twitter input node from the social section of the node palette into your canvas.

Next, double click on the Twitter input node you just added to the canvas and fill out the form — login with twitter, select the range of data you’d like to search through, and enter the list of words, @ids, and hashtags you’d like to filter by.

Debug Output

After saving your configured Twitter input node, drag the debug node from the output section of the node palette into your canvas and configure it to output the complete msg object to the debug tab.

Then, wire the two nodes, deploy your changes, and voilà, JSON objects should be streaming down the debug tab of your sidebar.

Parse Tweets

Key/Values in Orchestrate are how you store most data. At it’s basic level, it’s a JSON object store. You specify a unique key and then the JSON value to store. Thus, you cannot pass the JSON object for an individual Tweet directly into Orchestrate but must convert it into a compatible format first. To do this, drag the function node from the node palette into your canvas and configure it to parse the msg object from Twitter into a JSON object store matching the structure.

{   "data": {     "text": "msg.tweet.text"   } }

If you do not provide a key to Orchestrate, then one will be autogenerated for you. That said, if you’d prefer to explicitly set a key, match the below JSON structure.

{   "key": "...",   "data": {     "text": "msg.tweet.text"   } }

Whatever you do, remember that keys are unique and thus, you should make sure not to set a generic value like a Twitter @handler as the key, for that can be shared among many objects. Below is the code we’ve crafted for the function node.

msg = { data: { text: msg.tweet.text } }; return msg;

Pushing Data to Orchestrate

The final step of the equation is pushing each tweet to your Orchestrate collection. The first to do here is create the collection while logged in to your Orchestrate Dashboard.

If you browse through your node palette, you may notice that there aren’t any nodes related to Orchestrate. If that is the case, deploy your changes and then, run the following command in your project’s directory:

npm install --save node-red-contrib-orchestrate

After you restart your server and refresh your browser, you’ll see some nodes labelled “orchestrate” in your node palette under the storage section.

Then, drag the Orchestrate output node from the social section of the node palette into your canvas and wire it to your function node so the parsed tweets will flow into Orchestrate. Next, configure the node with your Orchestrate credentials and specify the collection you just created by its name.

Now, if you deploy your changes in Node-RED and check the contents of your Orchestrate collection, you should see a series of Tweet objects (assuming Tweets matching your set criteria have been published).

{ "results": [ { "path": { "collection": "tweets", "key": "074d7ddf82206057", "ref": "e9ec76f691f28b0f", "reftime": 1419899265923 }, "value": { "text": "RT @Frankiann_xoxo: What to do for New Years?" }, "reftime": 1419899265923 } ... ], }


This application in full has been released as a public, open-source repo on GitHub. Note that if you clone this locally, you’ll have to login with your own Twitter account and add your own Orchestrate API key in each node’s configuration since credentials.json is purposefully ignored by git.

In the future, you can create additional collections and nodes to interact with them in the same way described in this tutorial. Additionally, you can also utilize the power of Orchestrate and its nodes to execute powerful search queries.

Read more about Orchestrate’s Node-RED Client here.

Originally published at on January 21, 2015.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store