When Python meets Twitter: a simple script finds your tech community to follow
Sometimes it is difficult to guess where the key-influencers of a science/dev community are. I don’t want to find the typical old-school expert, because with the pace of discoveries, today’s expert is the enthusiast who re-learns everything every few years, or even months. What I want is to find the people who really are in the first wagon of the state-of-the-art train.
Twitter is an incredible tool to keep you updated on almost any topic, but choosing who to follow is not obvious. Let’s imagine I want to make a list of machine learning experts to follow on Twitter, but I only know a few top ML enthusiasts or key-influencers.
One methodology that has given me some success is to create an intersection between these key influencers on a common topic, and the result I get is a high-quality curated community of enthusiasts/experts in the field.
Finding enthusiasts on machine learning
I always keep an eye on the latest papers and ML Github repos, and for that I have a few favorite Twitter users that I love to follow, I will choose three of them because of the quality of their content, they are @ak92501, @jonathanfly and @quasimondo.
Probably I don’t want to copy-follow everyone they follow, because I could end following their friends and family, so let’s write a script on Python which use the Twitter API to intersect who are they following, in this way probably I will get an auto-curated list of key-influencers.
And here it is, a Twitter list you can follow. Ta-da! 🥳
You can find some well known names and companies on the top, but the interesting part is usually in the ones with less followers, this is a great tool to discover hidden gems 💎 or emerging enthusiasts.
Finding enthusiasts on Quantum
Now that this quick tool seems to work, let’s try the same but in the quantum computing field, I work at IBM Quantum, so I have met a lot of quantum computing enthusiasts, I am going to choose three who I trust more their criteria, and who are not IBM employees, in order to have a less biased list.
And here it is the Quantum Twitter list you can follow in one-click! ⚛️
Of course, there are more enthusiasts on Quantum and AI, but this is a way to find who of them are actively posting on Twitter, which is great when you want to keep up with certain topics.
I wrote 2 scripts, the first one to get the intersection of the enthusiasts, and the second one to extract and store their information in a CSV file. I used Tweepy, which is a Python library for the Twitter API, it works very well for making this kind of small prototypes.
What the code does in less than 90 lines is:
- Authentication on Twitter.
- Check if you already queried before that user, to avoid calling the API if you already have the data.
- Retrieve all the followers_ids or friends_ids depending on what you want to intersect. You could intercept someone friends with someone followers.
- Get usernames, because they are more human-readable than userIDs (Probably I could save some lines, time and API calls by using IDs, by I prefer to understand the results).
- Store them so you can always re-intersect local lists without expending more API calls.
After we got the intersection of the screen_names, we can run another short script to retrieve more info from this list (description, if they are verified, number of followers, statuses, etc.), and again in less than 30 lines. This is why I ❤️ Python :-)
What the code does is:
- Authentication on Twitter.
- Analyze saved intersections from the previous script, but also if you input a username, and is stored, it retrieves the info from that user’s friends/followers.
- Calls the api.get_user function and reads the data.
- Opens a CSV file and stores everything.
I hope you can try these scripts by yourself, let me know your results, and if you liked this topic let me know too with some claps or comments and I will start posting some curated lists of other interesting tech topics: cybersecurity, autonomous vehicles, VR/AR, etc.
Stay safe! :-)