Building a CasperJS Twitter Bot

Simple guide how to do casper js script that will find random item in eshop and than creating some really simple twitter robot that will tweet link to this random item each day.

I love buying random stuff from http://aliexpress.com/, I love that huge selection of really strange stuff there. I recently wrote some code that will find random item on that site and than tweet new rendom item each day. This blog post is simple tutorial how to do simple casper js stuff and simple cron stuff to create this bot.

Preparation

We will need Node.js and npm for all of this. Install it from their website.

Our main tool for this bot will be Casper JS we will need to install through npm, but first we need to install correct version of Phantom JS.

PhantomJS is super power tool that allows us to render web pages and interact with their DOM without needing a browser. Unfortunately latest version of PhantomJS (1.9.8) has a bug that is outputting some strange error messages to standard output. This version is installed automatically with CasperJS so we need to install correct version of PhantomJS before installing Casper. (This bug should be fixed in PhantomJS 2.0).

npm install -g PhantomJS@1.9.7-15
npm install -g casperjs

Next thing we need is to install some tool for tweeting from linux terminal.
I chose Twidge:
sudo apt-get install twidge
You need to setup your twitter account for your bot now (If you did not done it before). And simply run:
twidge setup
On screen tutorial will guide through process of setting up twitter in terminal.

Casper JS code

Now we have everything we need to create this awesome piece of code.
We start by writing simple CasperJS script that will got to alieexpress and find us one random item. Aliexpress do not have RANDOM button, but has pretty powerful search and listing of all of its categories. So we will go to the page with a list of all categories.
var casper = require('casper').create({ verbose: true, logLevel: 'error' });
casper.start('http://www.aliexpress.com/all-wholesale-products.html');
casper.run(function() { this.exit(); });

Next we need to get all categories from this page and choose one random:
var links = [];
var category = "";
function getCategories() {
var links = document.querySelectorAll('#page ul a:not(.more-link)');
return Array.prototype.map.call(links, function(e) {
return e.getAttribute('href');
});
}
casper.then(function() {
// aggregate all alliexpress categories
links = this.evaluate(getCategories);
category = links[Math.floor(Math.random()*links.length)];
category += "?shipCountry=cz&SortType=price_asc&isFreeShip=y";
this.open(category);
});

Get categories functions gets links for all categories on http://www.aliexpress.com/all-wholesale-products.html and returns them as Array. Them we choose some random catgory, we add some search params that are interesting for us (free shiping, sort by price, etc...). And then we open this newly created link. This will gets us to Search page for our category with our search parameters. What we need to do right now is to collect all links to products on this page, choose one randomly and return it.
function getStoreItems() {
var links = document.querySelectorAll('h3 a');
return Array.prototype.map.call(links, function(e) {
return e.getAttribute('href');
});
}
casper.then(function() {
var items = this.evaluate(getStoreItems);
var item = items[Math.floor(Math.random()*items.length)];
this.echo(item);
});
Now if we would run this with casper js code will return link to one random item on aliexpres.
casterjs filename.js

Cron

To complete our twitter bot we need it to tweet new random item every day. We can do complicated bash script that would wait for 24 hours and than ran out casper script, but doing this with cron is much easier.
To open cron settings type
crontab -e
If you are doing this for first time it will ask you what is your default text editor, and then opens it with all the cron settings.
We just need to add one line there:
0 0 * * * text=$(PHANTOMJS_EXECUTABLE=/usr/local/bin/phantomjs /usr/local/bin/casperjs ~/AliexpressBot/javascript/random.js);
twidge update "Today's random item is: $text Enjoy!"; echo $text > ~/log/cron.log

Sources

You can download all code we talked about here athttps://github.com/PavlikPolivka/AliexpressBot. Actual twitter bot can be found herehttps://twitter.com/AliexpressBot
One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.