Social Science — Get Twitter Data in PowerBI
Ever wanted to do some analytics on tweets with a certain hashtag (#) or from a certain user (@)
Don’t look further, their is a great tool called Microsoft Power BI. In following blog post, I’ll be explaining how to get twitter data into Power Bi .
Let’s Go!
First, we need to authorize our self on Twitter. This can be done by creating a Developer Account on dev.twitter.com. Click on Sign Up in the upper right corner.
After creating your account go to apps.twitter.com. And click [Create New App].
Now you need to fill in some details about your application. It does not really matter what you fill in for Name & Description. But name needs to be globally unique. For website you can use your company website. Callback URL is not necessary for this project. Don’t forget to agree with the ‘Twitter Developer Agreement’
Confirm with clicking ‘Create your Twitter application’
When you are on your Application page. Click the tab ‘Keys and Access Tokens’.
Then click on the button (on the bottom) ‘Create my access token’.
Keep this page open because the Consumer Key & Access Token will be needed.
Most work is finished now. Lets load some Data!
Open Power BI and click Get Data and choose ‘R Script’.
Yes we will use some R coding, but don’t worry its very basic and my comments will explain everything.
You will see a window where you can input your R Script.
Paste following code and click ‘OK’ to confirm.
[code]
#Check if all libraries are available.
list.of.packages <- c(“twitteR”,”stringi”)
new.packages <- list.of.packages[!(list.of.packages %in% installed.packages()[,”Package”])]
if(length(new.packages)) install.packages(new.packages)
#Load Twitter Library
library(twitteR)
#Fill in these details with your personal consumer en access keys that you received on apps.twitter.com
consumer_key <- “xxxxx”
consumer_secret <- “xxxxx”
access_token <- “xxxxx”
access_secret <- “xxxxx”
#Initial authentication to twitter
setup_twitter_oauth(consumer_key, consumer_secret, access_token, access_secret)
#What are we looking for on twitter?
search_term <- “#powerbi”
#How many tweets do you want to receive
number_of_tweets <- 1000
#Search on Twitter
tweets <- searchTwitter(search_term, number_of_tweets )
#convert received data to something PowerBi understands
df_tweets <- twListToDF(tweets)
#column cleanup
df_tweets <- df_tweets[,c(“text”,”favorited”,”favoriteCount”,”created”,”retweetCount”,”isRetweet”,”retweeted”)]
df_tweets$text <- stri_encode(df_tweets$text, “”, “UTF-8”)
[/code]
So what does the code do?
First we will install the ‘twitteR’ library. Its a package created by Jeff Gentry.
The package that he has developed will do all the hard word. So we don’t need to bother about it anymore.
Second step is to fill in all the keys that we received from Twitter. After that the code will do some authorization with Twitter, so we are allowed to receive data.
Now its our turn to tell the code what we are looking for and how many tweets we want to receive back.
Next is the real search on Twitter and converting the received data in something readable for Power BI.
Final part is cleaning up some columns and changing the code page of the tweets.
Click ‘OK’ to confirm the code. This might take a while depending on how many tweets you want to be returned.
A ‘Navigator’ window will open. Select the Data Set df_tweets and click OK.
All the next steps are up to you! The data is now available in Power BI
Enjoy querying Twitter!
