mini tweet bot

a mini tweet bot with multiple features

a python application

:for twitter automation with Cloud Foundry on IBM Bluemix


(0) intro

Mini tweet bot is a web application (accessible in the below link to the working app) for automating twitter functions with a designated synced twitter account. The front page has the feature of allowing anybody with access to the app to tweet to the synced account. This is a useful purpose for any situations in which you would want multiple people tweeting to the same account. Twitter offers the service called Tweetdeck (linked below), which has a similar feature allowing multiple users to tweet to the same account; however Tweetdeck requires every user to have a a personal twitter account, and have the private access information (the password) for the shared account. With mini tweet bot, there are no restrictions, no shared passwords, no login process, and no twitter account required to tweet; users simply need access to a web browser either on their own device or be able to use a public computer device at, for example, an event or Tradeshow. The other extended features of mini tweet bot are not available with twitter’s tweet deck, but are similar to many of the twitter bot features offered from 3rd party services. Those features, in the order they appear in the app, are:

  1. tweet image or selfie: this functionality appears on the home page along with the status update tweet functionality.
  2. translate: mini tweet bot will translate your tweets into L337 or ascii-art text.
  3. retweet and follow: searches twitter for tweets from the results of a query from the search terms from the given user input, then attempts to retweet on of the returned tweets, then attempts to follows the user of the retweet.
  4. Follow followers: a minor feature that follows all of the synced user account’s followers.
  5. Follow X more users: searches twitter for tweets from the results of a query from the search terms from the given user input, and follows X number of the users from the tweets in the results of the search. X is taken from user input or default
  6. Auto retweet: searches twitter for tweets from the results of a query from the search terms from the given user input, then retweets a tweet and follows the user in an automated process and so this cycle is repeated every N seconds for Y intervals. N and Y are taken from user input or default.
  7. Auto tweet file: accepts a file and retweets a newline from the input file in an automated process and so this cycle is repeated every N seconds.

An example tweet, tweeted from mtb.mybluemix.net:

Listed below are some of the environmental variables, the platforms used, and some of the important links referenced for the mini tweet bot.

python:

cloud:

examples:

twitter:


While at the Docker Conference (#DockerCon) in Austin, TX 2017, I was introduced to Cloud Foundry applications on IBM Bluemix. This occurred around the time of my spring break from my studies at Holberton School, which was near the end of our C language curriculum, and the week before we began our python curriculum. Since, I had time to rest during spring break, I was excited for the challenge of learning python, interested in navigating IBM’s Bluemix, and impressed with twitter’s social potential, I made mini tweet bot. This article summarizes some of mini tweet bot’s features and how I built it; it is organized by: (1) python, (2) web integration, (3) cloud infrastructure, and (4) bugs & need for improvement, and (5) build your own bot. Noob disclaimer: this mini tweet bot is my first attempt to build anything with python and to integrate with Cloud Foundry apps.


Holberton School Receives new round of investments

Holberton School

Holberton School is a project-based 2-year full-stack software engineering program. With an income-share agreement, the school has a joint stake in it’s student’s success, and therefore draws many candidates looking for an alternative to the college and university system. Named after Betty Holberton, one of the pioneer engineers to work on ENIAC, Holberton is also especially considerate of the interests of gender- and racially- diverse populations. To learn more about Holberton School, check out their website or follow their twitter account (beside is a recent retweet by @holbertonschool announcing an investment from Ne-Yo).


(1) Python

tweepy

One of the main engines of the mini tweet bot is the tweepy library for python. This library has many custom integrations with twitter that may be utilized by simply configuring a twitter account with Twitter’s development platform, and then calling a function in python. To configure tweepy with your twitter account, first, create a twitter application from the above link, and then second copy your keys, access tokens, and secrets to your python app. Once you have copied your secrets from a twitter app, then there is another set of configuration functions to help tweepy initiate. The below configuration example contains some of code on how I configured tweepy with my twitter app. Note: in order to keep my private information confidential, I stored the data in a separate file that is in my .gitignore file, and separate from inside the code of my python app. In the below example, I have not provided my personal key, token, nor secret information. To customize your own mini tweet bot, you should use your own key, token and secret information and replace your information with the information in the mycredentials.py file; all the rest of the python application will function for whatever twitter account is used with the configuration file.

configuration

$ cat mycredentials.py
consumer_key = '[YOURCOMSUMERKEY]' # (example: '5FmNevRxQriq1')
consumer_secret = '[YOURCOMSUMERSECRET]' # (example: 'pMKxr4ocPV451')
access_token = '[YOURACCESSTOKEN]' # (example: 'ZuPzOYwz2n6o')
access_token_secret = '[YOURACCESSTOKENSECRET]' # (example: 'TCqaOPc8j1qP')
$ cat app.py | head -26
import tweepy
import multiprocessing
import os
import cv2
from flask import Flask, render_template, request, jsonify
from werkzeug import secure_filename
from time import sleep
from credentials import (consumer_key, consumer_secret, access_token,
access_token_secret)
from camera import take_picture
# custom imports
import censorship
from aldict import ascii_dict, leet_dict
# custom variable names from imports
remove_whitespace = censorship.remove_whitespace
censor = censorship.censor
# tweepy
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)

Tweepy comes with many simple functions that can be used to tweet, follow others, and retweet. To read more about the full potential of tweepy, check out their readthedocs.io. For the purposes of this demo, I will explain how to setup tweepy to tweet, retweet, and follow. For more further explanations, you can alway check out the source code in the above link to the mini-tweet-bot github repository!

tweet function

Since this python application interacts with a web browser, I have set up the mini tweet bot to call this function when the user inputs text to be tweeted to the connected account. The web integration with flask is explained further below, this snippet is enough to show how simple it is integrate with tweepy. Once mini tweet bot receives the instructions from the user to make a tweet, the tweet_text() function is called with a variable, which is a string of characters from the user input. In the tweet_text() function, there is a try: except: sequence, because this allows to check if the tweet failed to tweet. A tweet would fail most likely due to the tweet being a repeated tweet, or some other problem with the twitter API. For more information on the twitter API, read the above twitter API limits link. To attempt a tweet, a tweepy function is called with the same input string variable and the api authorization from the information from the configurations file. There is no need to check to ensure that the input text fits the twitter requirements for text size because the HTML5 form used to take user input has size restrictions built into it. The profanity check has also already taken place, so at this point in the code, if a tweet is successful, we hope that it is a positive tweet. Below is the tweet_text() function.

def tweet_text(tweetvar):
""" tweets text from input variable """
try:
api.update_status(tweetvar)
except:
return False
return True

retweet & follow function

The retweet_follow() function is called when the user makes this request. This function uses a forloop to attempt to retweet (tweet.retweet()) the loaded tweets from a twitter search query. A tweepy integration runs the twitter search query of tweets, using the search terms from the mini tweet bot user input (tweepy.Cursor(api.search, q=searchterms)), and returns 10 tweets in the results (.items(10)). Similar to the above function, retweet_follow() function, also uses the try: except:() sequence. If a retweet is not successful, the instructions in the function are to attempt to retweet the next search result. If all 10 attempts are unsuccessful, then the function returns unsuccessful. However, if a retweet is successfully retweeted, the twitter user that made the tweet that mini tweet bot retweeted, is followed (tweet.user.follow()) by the linked twitter account so long as that user is not already being followed. Then, after the follow attempt, the function returns True, meaning the retweet and/ or follow were successful.

def retweet_follow(searchterms):
"""searches tweets with searchterms, retweets, then follows"""
for tweet in tweepy.Cursor(api.search, q=searchterms).items(10):
try:
tweet.retweet()
if not tweet.user.following:
tweet.user.follow()
return True
except tweepy.TweepError as e:
print(e.reason)
pass
return False

censorship

Since this is a public tool, I added my own custom censorship module that stops a user from being able to tweet a phrase that contain at least one of the 700+ strings of profanity words. If a word match is found, the tweet_text() function is never called, and a failure message is output. Below are some of the functions for the censorship; I'll explain how they work below.

from profanity import profanity
def is_clean(inputlist):
for i in range(0, len(inputlist)):
word = inputlist[i]
if word in profanity:
return False
test1 = remove_whitespace(word, "")
if test1 in profanity:
return False
test2 = leet_detect(word)
if test2 in profanity:
return False
test3 = leet_detect(test1)
if test3 in profanity:
return False
return True

The is_clean() function checks if the input list is clean or if it has a word from the profanity set. The set is in the source code (linked above). This is a simple loop that does a string compare function with each word from input and each word from the profanity set. The file profanity.py contains the set of vulgar words, and is imported with the first import line above. I used a set because the way python stores in memory and searches a set is a faster speed of O(log 1) vs. searching through an array or list of strings, which is the speed of O(log n). The list of profanity words is also stored in the profanity.txt file in the support directory, and it has no format, just one word per line. The second conditional check in the loop removes all the whitespace characters from each word just in case there is an input of "bad_word" or "i-am-mean", which would be a user trying to pass in "badword" or "iammean" to tweet. The last two profanity checks translate ascii-art and L337 to alpha characters, and then perform the same checks. So "b@dw0rd" or "b/\|)\^/ord" are translated to "badword" and "badword" and then the profanity check occurs. The ascii and L337 dictionaries can also be viewed in the source code in the file aldict.py.

def censor(tweetvar):
test1 = test2 = tweetvar.lower()
test2 = remove_whitespace(test2, " ")
test1 = test1.split()
test2 = test2.split()
if is_clean(test1):
if is_clean(test2):
return True
return False

The censor() function converts the input string into two test strings, which are then converted into arrays. One string has all the whitespace characters converted to spaces, the other string is left unchanged. The whitespace conversion occurs once here, and then again for each word of the other array that is left unchanged inside the is_clean() function. The first whitespace conversion changes an input string of: "1badword-2badword_3badword" to this: "1badword 2badword 3badword", so that each word is checked to see if has profanity. The second whitespace check is explained above. Finally each array is checked to see if it has profanity with the above is_clean() function. If there is no profanity, the censor functions return True meaning the string is clean, and the instructions in the flask @app.route functions respond accordingly.


(2) web integrations

modern people using a tablet to tweet with mini tweet bot

flask

flask provides excellent integrations for python and website browsers. There are other web frameworks that python experts prefer, but I chose this one because it is simple for what I wanted to do, I found the documentation helpful, IBM Bluemix Cloud Foundry python template uses flask, and it will be a part of the Holberton School curriculum when I get to web integrations. The major part of flask that I utilized was the functions to load webpages and take user input from HTML5 forms and input. Once flask is imported, there is no need for any other configuration except to establish the port on the device running the app. The mini tweet bot uses the below example configuration for the port, and I also included some of the variables that I used for the file upload functions.

configuration

from flask import Flask, render_template, request, jsonify
from werkzeug import secure_filename
app = Flask(__name__)
port = int(os.getenv('PORT', 8080))
UPLOAD_FOLDER = './uploads'
app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER
ALLOWED_EXTENSIONS = set(['txt', 'png', 'jpg', 'jpeg', 'gif'])

To help explain how flask interacts with user input, I will use the home page python code, which tweets user input or images directly to the linked twitter account. Below is the snippet of code, which has the route instructions for GET and POST method requests from the user in the decorator: @app.route.

@app.route

@app.route('/', methods=['GET', 'POST'])
def index():
if request.method == 'GET':
return render_template('index.html', newstring="none")
if request.method == 'POST':
tweetvar = request.form['tweet']
if not censor(tweetvar):
return render_template('confirmtweet.html', tweetvar="profanity")
if request.form['translate'] == "ascii":
tweetvar = translate(tweetvar, 'a')
elif request.form['translate'] == "leet":
tweetvar = translate(tweetvar, 'l')
if request.form['action'] == 'translate':
return render_template('index.html', newstring=tweetvar)
else:
if request.files['file']:
file = request.files['file']
if allowed_file(file.filename):
filename = secure_filename(file.filename)
filename = os.path.join(UPLOAD_FOLDER, filename)
file.save(filename)
if tweet_image(filename, tweetvar):
return render_template('confirmtweet.html',
tweetvar=tweetvar)
elif request.form['action'] == 'selfie from webcam':
filename = take_picture()
if tweet_image(filename, tweetvar):
return render_template('confirmtweet.html',
tweetvar=tweetvar)
elif tweet_text(tweetvar):
return render_template('confirmtweet.html', tweetvar=tweetvar)
return render_template('confirmtweet.html', tweetvar='failure')

In the above index class example, when a user requests the homepage (i.e. index.html or '/') the conditional if request.method == 'GET': deals with that request and returns the index.html template using return render_template. The instructions from the conditional if request.method == 'POST': deal with the user request to input data through the index.html form's submit input button. The request.form call returns the value linked with the input key if the user input data through the web form. The tweetvar = request.form['tweet'] section stores the user input string in the variable tweetvar, which needs to be 5 - 140 characters (that check is filtered in the HTML form). Next, is the profanity check, which is necessary because no other automation will occur if there is profanity (this is discussed in the above sections). If profanity is found, then the user receives a failure message with the reason being their use of profanity. That instruction is in the return statement after the profanity conditional check.

translations

If no profanity is found, next comes the translations. Since, I built a filter to check for users attempting to tweet vulgar words to the public account, I decided to use that feature to allow users to translate their input. If user selects to translate the text, then the same page is rendered with the user input translated into either ascii-art or L337. The translation process is much the same as the censoring functions. However, in the translation function, the instructions are to check each letter from user input, and the character is in the ascii-art or L337 dictionary (depending on what user chose), it is simply replace with its equivalent. Once, the full user input string is parsed, there is a conditional check if request.form['action'] == 'translate':, which checks if the user clicked the translate button. If this was the user choice, then the translated text is returned to the same page for the user to copy and paste into the tweet, so they have the option to use the translated string. If the translate button is not the submit button choice, this means that the tweet button was clicked, and so the following else statement deals with the instructions to check if there is also an image input.

input files

If there is an image input, then the file goes through a secure process to save the file and then calls the tweet image and status update function. The secure image process has these steps: (a) takes the file as a variable (b) checks if it is an allowed type, © changes the file name to a secure name (i.e. the file /home/vagrant/.bashrc becomes home_vagrant_.bashrc), (d) appends the path to save the file in ./uploads directory, and (c) saves the file. The code for these above steps looks like this:

if request.files['file']:
file = request.files['file']
if allowed_file(file.filename):
filename = secure_filename(file.filename)
filename = os.path.join(UPLOAD_FOLDER, filename)
file.save(filename)
if tweet_image(filename, tweetvar):
return render_template('confirmtweet.html',
tweetvar=tweetvar)

The last 3 lines check if the tweet_image() function successfully tweeted, similar to the tweet_text() function, which is discussed above. If the tweet is successful, then the confirmation page is returned with success. After this check, there is a check to see if the selfie button was clicked. The selfie button is not yet working in the cloud, but works on my local mac OSX machine, and it uses the camera to tweet a selfie. The last elif deals with the situation of no image input; so only if the user requests to update the status of the linked account with a text string. If the call to the function tweet_text() (described above) is successful, then the confirmation tweet occurs.

return statements

After the user tweet functions are processed, there is either a successful or unsuccessful page rendered in the return statement. Upon successful completion, that webpage will have a confirmation message displayed with the tweeted string. If the tweet was unsuccessful, for example, if there was profanity, or the tweet failed for another reason, then a failure message is displayed.


(3) Cloud

Without a server to host the mini tweet bot application, it becomes difficult to allow anybody else to access the application. Additionally, for long term running processes such as automating tweets from a file every N seconds, a Cloud becomes an absolute necessity, unless you have a computer in your home that you always keep running and always keep connected to the internet. Since I do not have my own server, and want to keep mini tweet bot running, the one of the most important elements of this application is the integrated with Cloud Foundry Apps on IBM Bluemix. IBM Bluemix offers a 30-day free trial of their cloud infrastructure; to begin that trial period, sign-up through the above link in the intro. After the 30-day free trial, there is a minimal pay per use plan that charges reasonable rates, which are based upon how much GB hours you use. This app is in the price range of $0.07 cents per GB hour. Since the app is less than 128 MB with one instance, it therefore costs less than $1.00 per month to operate, which is very reasonable. Once you have a Bluemix account, setting up your environment to host an application such as mini tweet bot, is as simple as clicking a button. In your IBM Bluemix Menu, navigate to Apps > Dashboard > Create App. In the Apps section, IBM Bluemix offers approximately 180 options in areas of: Boilerplates, Mobile, DevOps, Security, Storage, IOT, Watson, Data & Analytics, Containers, and Cloud Foundry Apps; Scroll to find the Cloud Foundry Apps section.

At the time of this blog, the CF section appeared like this example.

Since mini tweet bot is built in python, I used the python CF integration (image displayed above), which I installed after selecting the python app, choosing a name, host, and clicking the “create” button. Once your CF App is installed, IBM Bluemix has a great tutorial on how to get your python app running in the cloud. One simply needs to install the Cloud Foundry command line interface (from the github repo linked above in the intro), edit the Cloud Foundry application configuration files such as the: manifest.yml, Procfile, and requirements.txt, and store those files in the same directory that contains your python app. These file templates can be found in the Cloud Foundry python template github repo (linked above in the intro), and IBM Bluemix has helpful documentation on how to use those configuration files and many other topics. Some of the other useful features of IBM Bluemix are that: Cloud Foundry CLI, provides the ability to connect to the directory of your project via SSH. There is a user interface through the website that allows you to see all the output logs of your app. The CF Command Line Interface has many other commands that I have not mentioned that improve the integration with Cloud Foundry apps on a cloud.

If you like this twitter bot, but don’t want to host it on IBM Bluemix, you can run the app in your local command line interface, and it should be visible from a local port on your own machine. Or find an alternative hosting company.


(4) bugs & need for improvement

bugs:

I have not tested to see how many requests the mini tweet bot can handle. Currently, it does handle multiple requests per minute in the same browser, but I have not tested by increasing the number of requests nor using more than 1 browser. Twitter API has restrictions for the limits on tweets per hour and mini tweet bot has “try and except” processes to check for and deal with failed attempts to tweet or access information from twitter. IBM Bluemix services went down approximately 1–3 times since I began working with IBM Bluemix, and if this problem persists, it may interrupt some of the background processes which deal with automatic tweets in a time interval. I did not see any other issues with my running applications though, and despite not seeing any other instances where my app went down, there was another error message that I received periodically, which seemed to an issue with host server that did not effect my application. I did not know the cause of this message; the output text from my logs was: LGR/nullproxy: error connecting to [IPADDRESS]: dial tcp [IPADDRESS]: i/o timeout

need for improvement:

These are some of the features that I am planning on adding:

  1. An if conditional to correctly allow more characters for tweets that include URLs.
  2. Integrate IBM Watson to help improve voice recognition with tweets (i.e. a user simply speaks in order to automate a tweet.
  3. Like tweets of users that bot retweets.
  4. Add twitter rant: tweets lines from text file every 60 seconds with max N lines.

Other:

Please contact me for more information on vulnerabilities, to help with upgrading, or with any other questions.


(5) build your own bot

  • fork or clone the github repository.
  • get your own twitter app from twitter dev tools linked above.
  • change the mycredentials.py file name to credentials.py
  • change the strings from the credentials file to contain your personal twitter information.
  • change the twitter feed link in the <aside> HTML tag to instead link to your twitter feed.
  • change the link of the twitter handle in the nav HTML tag to link instead to your linked twitter account.
  • change the icon/ logos to your preference.
  • find the cloud to host the app. Mini tweet bot is already setup with Cloud Foundry for IBM Bluemix, but other cloud services will work as well.

demo screen shot

NOTE: The mini tweet bot functions most successfully when hosted on a cloud. However, if you would like to run the app on your own machine, you can run it, and it will be loaded on a local host port IP address such as: http://0.0.0.0:8080/. If you do not want the user interface, you should then use only the tweet functions, and run them on an as needed basis. Here is an example of how to run a single function from the singletweet.py file:

$ cat singletweet.py
import tweepy
from credentials import *
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)
def tweet_text(tweetvar):
""" tweets text from input variable """
try:
api.update_status(tweetvar)
except:
print("error")
pass
tweet_text("this tweet is an example of running a tweet function in python")
$ python singletweet.py