Building ML-Based Real-Time Slack Bot

Hi there,

Nowadays most companies use slack as an internal communication platform. In big companies, slack channels are growing so fast, it is quite possible to see thousands of channels. In Trendyol, we manage many channels as a Data Management Team. Usually, In those channels, people who are in various teams write their demands, authorization requests, or ask questions about tables that are in the data warehouse.

In these conditions, we have started to think about how to understand the requester’s intention and take action by using a slack bot. Today, we are going to see how we implemented this idea and how we use it in production.

Our agenda will be the following:

1. Overall Architecture
2. Training Machine Learning Model
3. Slack Bot Implementation
4. Nginx Configuration
5. Dockerizing Everything

You can find the entire source code of on github

Overall Architecture

Let’s start with the overall architecture and mention some key points. Our story starts with getting a message from someone on a Slack channel. We gave our REST API endpoint to the slack. Slack send us the message content and metadata as a POST request for every message on channels that we listen to. The message is checked whether it is sent by the bot itself or our team members. If the message is required to be replied to, the bot.py applies to preprocess steps and sends the features to the model that was trained before. The machine learning model gives a prediction about message intent and according to the intent, the bot replies to the message and takes the action. We want to log all messages and predictions to be labeled later. Therefore bot.py writes message content, model prediction, and other information to a Google Sheet that is connected to a Google BigQuery table. We label the messages regularly. Our cron job triggers model.py python script. The script trains 12 different models with 4 different algorithms and 3 different preprocessing steps. Chooses the best model and saves Model, Vectorizers, and best model name. After the training bot is improved end runs better. That’s the overall workflow. It’s time to implement those concepts.

Training machine learning model

Firstly lets’s start with creating our Python 3. x environment with our dependencies. We can create a file named environment.yaml file with the following configurations.

conda env create -f=environment.yaml
conda activate babur

After running the commands above we are ready to develop. We need to create some folders model, logs, and data.

.
├── README.md
├── data
├── logs
├── model
└── environment.yml

We will create a file named model.py under the model folder and import the necessary packages and set the variable MODEL_TEST_MODE as True to use later.

As we mentioned the overall architecture, we will get training data from a BigQuery table.

Note: If you do not want to use BigQuery, you can use any other way to store training data like (CSV, JSON, etc.). We have also implemented load_data_from_csv_file function in model.py

BigQuery table schema

Note: To Initialize client GOOGLE_APPLICATION_CREDENTIALS environment variable must be set with the path of the google credentials file.

After getting the data we have to apply some preprocessing steps. We need another function for common preprocessing steps.

We use several feature extraction steps such as count vectorization, tf-idf vectorization, etc. So we need another function for tf-idf vectorization. We also have to save our TfidfVectorizer object to be used by bot.py’s preprocessing steps.

Now, we can implement our machine learning algorithm functions. We will use four machine learning algorithms to choose the one that gives the best accuracy. Our functions will be like the following.

The functions above serialize and save the model (to be used by slack bot) if it is not testing mode. Everything seems ready to run. This will be the last part of model.py.

Let me summarize what we do above. Firstly, we get the data from BigQuery into a pandas DataFrame. After that, we apply to preprocess steps, encode labels (store label encoding mapping in a file). We should split the data into train and test sets to measure our models’ performance. We train 12 different machine learning models by using these train and test sets and choose the best model (store the best model and its preprocessing info in data/best_model.txt). Finally, we set the MODEL_TEST_MODE variable to False and train that model again with the entire data without splitting. Logging is also a very important part. Each training step including models performances can be seen from logs.

Do not forget the fact that more training data usually increase the model performance usually.

Now, it is time to start building our slack bot.

Slack Bot Implementation

Create a python file named bot.py under the root of the project directory. As we did the model part, we will start with importing the necessary packages.

To avoid hard-coded configuration, let’s create a file named .env. With the following structure.

We want to listen to some channels just for logging purposes. On the other hand, we will take action for some channels. To map channels id’s to their friendly names we create a file named channels_info.json with the following structure.

In bot.py get our environment variables like this.

We need a class for handling coming messages from slack channels. Let’s implement SlackMessageHelper class. This class will have three main functions.

get_response: Takes slack message, calls analyze_message function to get a prediction, takes action, and returns response accordingly.

analyze_message: Takes slack message, applies to preprocess steps, and uses the best model to get prediction and returns it.

get_sheet_cover: Writes message information and the prediction on the given Google Sheet with the following structure.

Now let’s go to the slack apps page. Choose your bot and find your bot client secret in the Basic Information section, we will use it in our app.

Note: If you do not have any slack bot yet, you can follow the documentation to create one.

After that open the Event Subscription section and enable event subscriptions. Place your domain that with the “/slack/events” endpoint. (If you run on local, you can use ngrok.

Slack Apps | Event Subscription Page

Make sure that you have the necessary permissions.

Slack Apps | Event Subscription Page

We can come back to bot.py to develop our bot. Initialize our slack App by using our slack bot token and signing secret. We will create a function with @app.event(“message”) decorated. This function takes every message on channels that the bot is added. After parsing the payload rest is very simple. We just use our slack_message_helper object to take action. We do not want to give a response to our team members so we will create a list contains our team member names to be filtered.

Here is the implementation.

Our web application will be run on the 5000 port. Remember, we gave our endpoint to slack. Slacks send each message payload to us via a POST request.

We use gunicorn to serve our application. We can use a shell script to run the app. Create a file called run.sh with the following content.

run.sh

#!/bin/sh
exec gunicorn -b :5000 --access-logfile - --error-logfile - bot:app - --timeout=3000

Nginx Configuration

We build an Nginx server to manage our application requests. Start with creating a folder named nginx that contains two files nginx.conf and project.conf.

Configurations will be the following.

nginx.conf

project.conf

Nginx will listen on 80 ports and our application 5000.

We have completed the 90% of our application. Now it’s time to dockerizing.

Dockerizing Everything

The last step is to create Docker Images and containerized our application. We can simply start with Nginx. Under nginx folder, let’s create a file named Dockerfile.

nginx/Dockerfile

This file will create an image with our Nginx configurations. Now it is time to create a docker image for our bot. Under the project root directory, we create a Dockerfile as well.

./Dockerfile

Lastly, we will create a docker-compose.yml to stand up our container in an organized way.

Bonus: We can create a train.sh file to the schedule training phase.

#!/usr/bin/env bash
cd ~/babur
docker-compose down
/home/babur/miniconda3/envs/babur/bin/python /home/babur/model/model.py
docker-compose build --no-cache
docker-compose up -d

Finally, we have ready to the production run the following to build our images and stand up our containers.

docker-compose build --no-cache
docker-compose up -d

Summary

Let’s look at what we have done so far with an example scenario:

• A message comes to the slack channel that Babur listens to.
  “Hi, I need authorization about your system. Can you help me?”
• Babur gets the message via Slack API.
• Analyses message with the best model.
• Predicts the message as an “authorization” with %85 probability.
• Gives an answer.
• ”Hi, thanks for your message. You must request authorization through the system. Then we can define authorization for you. Good work.”
• Babur also saves the incoming message to excel. We can label the messages in excel and use them to train our model.

Special thanks to Erkan Ekser for his great dedication of this project and huge effort.