Checking on your twitter friends

I’m lazy… I might even be the laziest person on earth but I’m still waiting for a confirmation on that one from Guinness (the book, not the beer…) Like most people, I hate doing repetitive tasks by hand. I think it might be the reason I became a developer in the first place… Anyway, I bet I’m not the only one with this “problem” and as such, this post is meant for everyone who’d like a little bit of automation in their lives. So, let’s automate Twitter!

In this post, I’ll go step by step and explain how to build a script to fetch the list of your followers (the ones who follows you) and the list of friends (the ones you follow). To follow along, you should have a basic understanding of the terminal (link) and python (link). If, by any chance, you get stuck, leave a comment below. So here it goes!

Let’s get ready

To connect to the twitter API, you’ll need proper credentials and fortunately for us, they are very easy to create. To do it, simply go to http://apps.twitter.com/app/new, fill everything and simply create your new app. After the refresh, go to the Key and Access tokens tab, click Create my access token and you should be able to collect everything needed from the screen:

• Consumer Key
• Consumer Secret
• Access Token
• Access Token Secret

Now, we have everything, for the non-code part at least. So, let’s start the coding part, aka, the hardest one. To get started, create a new file and import the two libraries we’ll need:

• tweepy - our script is based on tweepy. This library alone will let you make every allowed request to the twitter API. To install it, simply run pip install tweepy on your terminal and you should be ready to go. As a side note, if you installed python properly, you should have pip too.
• time - the twitter API has limits, so you can't make thousands of requests at once. Since we plan to fetch a good amount of data (depending on your total amount of friends/followers), we'll use time to add some pauses between requests. This library is "free", meaning, it comes installed with python by default.

Since you are preparing everything, grab your credentials and add them to the right spot on the code bellow and don’t forget to change the screen name too, or you’ll be analyzing my followers/friends not yours.

import time
import tweepy

consumerKey = '__________'
consumerSecret = '__________'
accessToken = '__________'
accessTokenSecret = '__________'
screenName = 'mauro_goncalo'

Next, with your keys, we’ll initialise tweepy to be able to interact with twitter.

authTweepy = tweepy.OAuthHandler(consumerKey, consumerSecret)
authTweepy.set_access_token(accessToken, accessTokenSecret)
api = tweepy.API(authTweepy)

Fetching our followers

Now that everything is set up, let’s fetch our followers. To prevent mass extraction of data, the Twitter API is paginated just like its web interface. This means that instead of fetching a long list of followers/friends, we’ll need to fetch multiple pages of users and put everything back together into a big list. In practice, we’ll use a cursor (this comes with tweepy) to fetch all the pages of followers, and for every one we'll keep their screen name.

Note: When making calls to twitter, we need to pace ourselves and limit the number of requests or we’ll end up hitting the limit and be temporarily locked out. To be safe, simply wait a bit before fetching the next page of results. To do it, use time.sleep(60) to pause for 60seconds.

followers = []
for page in tweepy.Cursor(api.followers, screen_name=screenName, count=200).pages():
    print '> Fetching api.followers'
    for user in page:
        followers.append(user.screen_name)
    time.sleep(60)

print 'followers count', len(followers)

Fetching our friends

After getting a list of followers, we’ll now fetch a list of friends. The process is pretty much the same, the only thing that changes is the function used to fetch the data. We use api.friends instead of api.followers.

friends = []
for page in tweepy.Cursor(api.friends, screen_name=screenName, count=200).pages():
    print '> Fetching api.friends'
    for user in page:
        friends.append(user.screen_name)
    time.sleep(60)

print 'friends count', len(friends)

Crossing the finish line

Now that we fetched everything, we’ll use both lists to extract valuable information: by crossing the lists we can check who’s present in one list only. To do it, we need to convert our previous lists to sets. This allows us to subtract two lists and get the elements not present in both lists. In practice, this means we’ll simply see who follows you but you don’t follow back and who you follow but doesn’t follow back.

setFollowers = set(followers)
setFriends = set(friends)

print 'They follow you but you don\'t follow back:'
print setFollowers - setFriends
print
print 'You follow but they don\'t follow back:'
print setFriends - setFollowers

Final code

By now, you should have this final version of the code:

import time
import tweepy

consumerKey = '__________'
consumerSecret = '__________'
accessToken = '__________'
accessTokenSecret = '__________'
screenName = 'mauro_goncalo'
authTweepy = tweepy.OAuthHandler(consumerKey, consumerSecret)
authTweepy.set_access_token(accessToken, accessTokenSecret)
api = tweepy.API(authTweepy)
followers = []
for page in tweepy.Cursor(api.followers, screen_name=screenName, count=200).pages():
    print '> Fetching api.followers'
    for user in page:
        followers.append(user.screen_name)
    time.sleep(60)

print 'followers count', len(followers)
friends = []
for page in tweepy.Cursor(api.friends, screen_name=screenName, count=2000).pages():
    print '> Fetching api.friends'
    for user in page:
        friends.append(user.screen_name)
    time.sleep(60)

print 'friends count', len(friends)
setFollowers = set(followers)
setFriends = set(friends)

print 'They follow you but you don\'t follow back:'
print setFollowers - setFriends
print
print 'You follow but they don\'t follow back:'
print setFriends - setFollowers

A more efficient lazy human

I hope you managed to get your script running and fetching your results without lifting a finger (isn’t that awesome?!). Now you know who follows you but you don’t follow back and the ones you follow but don’t follow back. I believe this information could be put to use in many ways, helping you improve your account(s).

I hope you liked this post and if it was useful, then my mission is complete, and I was able to help you to be a “more efficient lazy human”… And remember, if by chance you get stuck, just let me know and I’ll try to help.

PS1: Quick question: What do you want to learn next? What do you want to see automated? Leave a comment bellow.

PS2: If you need to get something automated, get in touch. I love new challenges and I’ll be happy to help.