Create aTwitter Bot Using Dev-Ops Principles.
In this article series, I will go through the steps on how to create a CI/CD pipeline and code for a bot that will automatically publish tweets.
Part 1: Overview and Python Code
One of the biggest issues with software projects, and software development, in general, is the step from development to operations. People new to the industry are soon made privy to the classic “It works on my computer” conundrum. Dev-Ops is a mindset and a culture that is focused on integrating development with operations. To succeed at Dev-Ops, which has proven to be a driver of more efficient teams, you must be familiar with all the important steps of the process. Knowledge of the drives of successful software projects can empower us to make better decisions and avoid common pitfalls that haunt software projects.
In this article series, I will go through all the different parts of creating a project using Dev-Ops principles such as Continous Integration (CI) Continuous Deployment(CD), Infrastructure as Code (IaC), and testing.
Set-Up
The project was structured as follows.
- For source control and versioning — git and Github were the tools that were used. Github was also used for version control
- For Continous Integration, I used GitHub actions and YAML files. Github has a lot of boilerplate code and APIs that make it easy to set up workflows.
- I used Google Cloud Platform as a hosting platform. Other popular options are Amazon Web Services and Microsoft Azure.
- To make and deploy infrastructure I Used Terraform. Terraform is easy to test and deploy and is also cross-platform so it can easily be reconfigured to other platform services.
- For actually making the Twitter bot, I used the python module tweepy to get authenticated and post the tweet.
- To store the tweets in a spreadsheet format, I used Airtable. One of the advantages of Airtable is its easy-to-use API which in this case made it an easy choice.
The code repo can be found on Github :
Setting up the Twitter handler.
The Twitter bot needed the following functionality. It should be able to retrieve the unpublished quotes, which were added to an Airtable sheet, divide the quotes into multiple tweets if necessary, and then publish the tweet, either as a single tweet or as a collection of multiple tweets in a string (with numbering). Then it should add the source as a reply to the tweet. Finally, the code needs to update the Airtable sheet so that the quote is marked as tweeted.
To set up the Twitter handler, all Twitter-related python files were put into their folder in the project.
Handling environmental variables
In all cloud-based projects, secret management is of importance. In this project, a combination of pydatic, GitHub secrets and Terraform form the basis of handling the environmental variables.
The Settings class defines all variables and uses the .env file in the project to get the values of all the variables. In this settings file, all the necessary keys and variables are stored.
To import the different environmental variables, the settings class is imported into the different scripts and then environmental variables from the .env file can be stored locally, and in the cloud environmental variables.
Get the data from the Airtable API.
The Airtable was structured as a sheet with columns representing the author, book, and quote, as well as a column indicating that the tweet has been posted on the Twitter account.
To retrieve the Airtable base, we need to have an Airtable API key, as well as the base_id and table name.
The function above gets data from the Airtable, here it is based on the quotes, author and book columns, and returns the data from a single row.
Airtable API access can be enabled by visiting the Airtable API page.
Divide the quote into multiple tweets if too long.
As some of the quotes might be longer than 280 characters, which are the maximum allowed into a single tweet, it is necessary to check the length of the text to be tweeted and divide it into logical parts so that they can be posted in a coherent sequence. We do not want the individual tweets to be split in the middle of a sentence, and certainly not in the middle of a word.
The divide into tweets function takes in a text string and returns a list of tweets, which is divided at logical places (Periods, commas, semicolons, or double dashes). We can tweet each item in the list returned. Note that it divides the tweet into 276 character-sized items if the original text is longer than 280. This is because if we have a tweet string, we want to enumerate each tweet in the tweet string.
Authenticate Twitter
To create a Twitter bot, we need to create code that can authenticate the twitter object, and also utilize the Twitter API. To get access to Twitter API, you need to create a Twitter developer account and create your own API access keys. Just follow the instructions to create an account and save the credentials in the environment file.
When we have the twitter access and consumer credentials, we need to be able to create an API object using pythons twitter package tweepy, and authenticate.
The function above creates the Twitter API object, to be able to do this it needs to have correct consumer keys and tokens. These keys are found by creating a project in the Twitter developer portal and generating the keys. Remember to give your project full access to the Twitter account so that it can post the tweets.
Post the tweet
Now that we have created an API and have a list of tweets and a reference string that we want to add to the tweet as a response.
In the function send_tweets_to_twitter we take in the tweets we want to be posted, and loop through the list of tweets. If there only is a single item in the tweets list, we just post that and don't bother to add an enumerator (If we had, we might have caught an error in twitter as the tweet might have been longer than 280 characters). If there a multiple tweets, we add each tweet to the below one as a response to the first tweet using the first tweet`s id tag. Finally, we add a reference as a response.
Putting it all together
Now that we have created individual parts, we need to tie all the pieces together in order to post a quote to Twitter.
In this function, we first use the authenticate twitter function to get access to our Twitter account, then we get the quote, author, book and Airtable row id from Airtable. We divide the tweet and post it. After the tweets have been successfully posted, we update the Airtable so that the quote is marked as posted.
Next up
We now have a functional code for posting tweets, but in order to automate it, we need to create a CI/CD pipeline, set up the cloud infrastructure, and deploy fail-safes for deployment. The next section will discuss testing, security, and continuous integration.
Please feel free to comment on the article or share it if you find it insightful. I also welcome constructive criticism and also suggestions for improvement.
