How to collect data like a spy — Part 7

Creating an interactive dashboard for your data

For this part we are going to need Solr, Banana and NiFi and we are going to create an interactive dashboard of the tweets.

Solr

Download Solr and install it.

After downloading the solr, all you need to do is untar the tar file and move it to /usr/local/solr .

tar xzf solr-5.2.1.tgz solr-5.2.1/bin/install_solr_service.sh –strip-components=2
./install_solr_service.sh solr-5.2.1.tgz -i /usr/local/solr -u $user
// where -i is installation directory and $user is the user you are using now.
After you perform above steps you will get the installed directory at /usr/local/solr/ and now you can start the services
mv /usr/local/solr/solr-5.2.1/* /usr/local/solr/
./bin/solr start -c -z localhost:9983

Once you start the solr you can open the web url at http://hostname:8983/solr

Banana

Banana is a tool to create dashboards to visualize data you have stored in Solr. Commonly used with Logstash for log data. Its a fork of kibana.

Download Banana and install it.

Run Solr at least once to create the webapp directory:
cd $SOLR_HOME/bin/
 ./solr start
Copy banana folder to $SOLR_HOME/server/solr-webapp/webapp/
Browse to http://hostname:8983/solr/banana/src/index.html

NiFi

Now create a data flow in NiFi to send the twitter data to Solr.

Configure the PutSolrContentStream processor as follows, this will push the tweets to Solr.

Enable the flow and then check Solr . Use the following URL and search the Solr index.

http://hostname:8983/solr

Dashboard

Use the following URL to access the dashboard.

http://hostname:8983/solr/banana/src/index.html#/dashboard

You will have a timeline of tweets in the dashboard. The dashboard is highly configurable, have a play.

The Series

Part One — How to collect data like a spy
Part Two — Getting NiFi up and running
Part Three — How to collect social media data like a pro
Part Four — Creating a database with AWS Athena
Part Five — Connecting RStudio to Athena
Part Six — Creating Maps of the Data in RStudio
Part Seven — Creating an interactive dashboard for your data

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.