MapD Tweetmap Installation and Quick Overview

MapD recently released an open source Tweetmap Demo. The code is available on GitHub.

It installs easily on Mac OS X. You can get it running locally in 10 minutes and start modifying the code. It’s a great learning experience.

Installation

git clone https://github.com/mapd/mapd-tweetmap-2.git

npm install

You will see a few warnings. These can be safely ignored for your initial test.

npm start

You will see a message that says, webpack: Compiled successfully

You can now open a local browser to:

http://localhost:8080/

The local test is running a small static dataset of approximately 572,000 Tweets.

It’s still enough to get a taste for the features of the MapD platform. A fun search is to show the number of parties. The sample dataset captures Tweets from October 31st, 2014. What will we find?

At that one point in time, people tweeted the word party 142 times.

I also looked at people that went trick or treating.

By typing in the URL manually, you can see the actual Tweet or Instagram post. This will provide a brief glimpse into the fun and friendships that people share on Twitter.

Code

There are three front-end libraries that you should look at:

  1. mapd-connector
  2. mapd-crossfilter
  3. mapd-charting

You can get a quick understanding of how to use MapD Connector and MapD Crossfilter by looking at the code here.

MapD Connector

In /src/services/connector.js, you can see the connection to the database.

This is a hosted MapD Core database with a small dataset for testing.

MapD Crossfilter

Cross filter allows you to apply a filter to one chart and automatically update the other charts. You can see this in /src/services/crossfilter.js

MapD Charting

The GitHub repository for MapD Charting comes with a number of examples for you to browse and edit.

Streaming Data

The Tweetmap Demo on the MapD site has close to 400 million Tweets in it, much larger than the 0.6 million Tweets in the demo on GitHub.

With 400 million Tweets, I can see that there are 38 ,000 Tweets about hockey.

You can see some of the hotspots and drill down into individual Tweets.

With a larger dataset, you can discover new facts about hockey and the culture of hockey.

Getting Streaming Data

To stream the data you can look at the Twitter API or a service like GNIP, Twitter’s enterprise API platform.

Moving Data From Stream into MapD Core

You can stream the data into the MapD Core with StreamInsert.

Documentation on using StreamInsert is here.

Example:

cat file.tsv | /path/to/mapd/SampleCode/StreamInsert stream_example mapd --host localhost
--port 9091 -u mapd -p MapDRocks! --delim '\t' --batch 1000

The MapD table used in the streaming should be set up with a row_count to automatically manage keeping just the latest records.

Download MapD Tweetmap-2 from GitHub Now

)

Jesse’s Big Data Adventures

Written by

Exploring the potential of blockchain and big data. Trying out blockchain technologies, big data infrastructure, visualization, and storage technology.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade