How to Make Your Own Private Virtual Assistant Powered by Machine Learning in Python

Introduction

These days, Artificial Intelligence is being used in so many different ways, from self-driving cars and intelligent robots, to things like generating art or designing microchips for computers. We are all familiar with assistants like Google Assistant, Amazon Alexa, Siri, Cortana, and more… These are all powered by Machine Learning and there’s a lot that goes behind the scenes to give a proper response. Using Speech Recognition, they first understand what you say and convert that into text, followed by interpretation and retrieving an answer and finally speaking it back to the user.

Even though there are so many assistants out there that we can use, I had always wondered how I could make my own, since I could share all my personal information with it without worrying about privacy, create my own replies, and add my own functionality, all customized for me.

In this article, I will show you how you get started to make your own Assistant powered by Machine Learning, in Python. We will create an intent classifier, then we will add functions that our assistant can execute based upon the user’s intent. We will then add speech recognition and text-to-speech capabilities to our Assistant.

So let’s get started!

Getting Started

Before we start, you will need Python 3 installed. We will also need some packages such as Scikit-Learn, but we will install those once we need them. You must also know basic Python concepts such as functions, classes, etc.

First, create a folder for the assistant, which will contain the project files and subfolders. Then, let’s create a file called main.py, in which we will create a class for our Assistant. You can add your own custom variables as well. You can also name it whatever you would like.

Intent Classification

In order for our assistant to understand what the user wants, we have to create an intent classifier. Intent classification is the automatic categorization of text data and an intent classifier automatically analyzes text (what the user said) and categorizes them into “intents”. For example, is it a greeting, or is the user asking for weather?

This is where Machine Learning, specifically a text classification algorithm, will be used. It will take the user input, and classify the user’s intent. You can use whatever text classification algorithm you would like, but for this article, I will use Naive Bayes Classifier using the Scikit learn library.

Install scikit-learn using the following command.

pip install scikit-learn

We will also use pandas to read the csv file with the training data. Install it using the following command:

pip install pandas

Next, create a subfolder called intent_classification, and create a file called data.csv. This data will be used to train our intent classifier. You can add any data you would like, just be sure to not include spaces after commas. The first line will contain the column names (text and intent). Here is an example -

This is just an example, feel free to add your own data to increase the accuracy of the model and customize it.

Now, we will create the intent classifier. First, make a new file called intent_classification.py, where we will create our intent classifier class. I will show you how to create a Naive Bayes Classifier to classify the topic of the text. Naive Bayes is a classifier (classifying text in this example) which uses the Bayes Theorem to determine the probability (in this case the probability of text being a certain class).

That’s it for intent classification! Now, let’s go to our main.py file and add the functionality.

Create a new function called reply in the Assistant class, which takes in an argument of text.

Adding a function

Based on that intent, we can now execute a function. You can add your own functions, but I will show a function that will get the current weather.

I will use OpenWeatherMap to get weather data. To get started with that, you will have to go to https://openweathermap.org/api, create an account, and get a key. Then, install the python package called PyOWM using the following command:

pip install pyowm

Make a new folder called assistant_functions inside the main project directory, and create a new file called weather.py.

There, create a function called get_weather

Then, you can import that function into main.py, by adding the following the beginning of main.py

from assistant_functions.weather import get_weather

In the reply function in the Assistant class in main.py, add a dictionary which contains the functions for different intents. Then, add code to call the function based on the intent The reply function should look similar to this -

You can test it out by calling the reply function and passing the text as a parameter. You can also create more functions and add them in a similar way.

To test the reply function, add the following to the bottom of main.py:

assistant.reply(”weather”)

Adding Text-to-speech and Speech Recognition

Next will come the text-to-speech and speech recognition functionality of our assistant.

We will be using pyttsx3 for text-speech, which you can install using

pip install pyttsx3

Now, we will import pyttsx3 and create a speaking function that takes an argument of text and says it using pyttsx3.

First, update your Assistant __init__ function in main.py, to avoid initializing the speech engine multiple times:

Then add a speaking function to the class, called say():

Go back to the reply function and instead of print(), make it self.say()

Next, we will add a function to listen using the speech recognition library, which we can install with the following command:

pip install SpeechRecognition

You may also need to install PyAudio — https://pypi.org/project/PyAudio/

In main.py, we will first import speech recognition, and then update the __init__ function:

import speech_recognition as sr

Now, we’ll create a new function called listen() in our Assistant class. The timeout parameter is the maximum number of seconds that it will wait for a phrase to start before giving up and throwing an exception. The phrase time limit is the maximum number of seconds that it will allow a phrase to continue before it stops listening.

Adding a loop to listen and reply

The final step is to create a main() function in the Assistant class. We will add a loop which listens for user input and then calls the reply function.

Now we can create an object of our Assistant class and call the main() function:

Wait until it prints listening before you start speaking. Try saying “weather” and it will tell you the temperature. You can add more functions that respond to other intents such as greetings.

Conclusion

That is how you can make your own Assistant in Python. Feel free to add your own custom functionality to your assistant and make it your own! If you want to see my Assistant project, here is the link to my GitHub repository — https://github.com/reybahl/Assistant.

Thank you for reading, and I hope you liked it!