How to create virtual assistant in Python 10 min 50 lines of code | No need to buy ALEXA, BUILD your JARVIS

Published in

Analytics Vidhya

6 min readJan 29, 2021

Introduction

A virtual assistant, also called an AI assistant or digital assistant, is an application program that understands natural language voice commands and completes tasks for the user.

A virtual assistant, also called an AI assistant or digital assistant, is an application program that understands natural language voice commands and completes tasks for the user.

Latest Demo and code walk-through

Previous Version Demo: https://youtu.be/LliTjuxDw_o

Content-

What we are building-
Code Explanation
Complete code
GitHub Repository
How you can contribute
References
How to stay updated about new builds and release

1. What we are building-

Our virtual assistant will able to do the followings things-

Voice Recognition
Text to Speech
Open Photos
Open Google Photos
Tells you joke
Shutdown your system
Weather forecasting
Launch Games
Launch Windows Applications
Open Websites
Tells you about almost everything you ask
Tells you date and time
Greetings
News
Face Recognition (need to train ML model)

You can interact with your laptop’s microphone/console. The response generated by the assistant will display on the console or as a speech via the speaker.

Future Updates: Selfie Click, Chat more like humans, etc. You tell me what do you want. Or contribute in https://pypi.org/project/JarvisAI

2. Code Explanation-

So let’s create our own virtual assistant.

Notes-

All the codes is available on my GitHub.
Demo YouTube video and code YouTube video is also available on my channel.
Required links and packages are mentioned below.
Sharing and Subscribe me on YouTube would be appreciated
The code and functions may be update on future so stay tuned with me on YouTube and this project official page.

Lets code-

2.1. Required packages and libraries-

https://pypi.org/project/JarvisAI/

pip install JarvisAI

This is the latest virtual assistant module, created by me. It provides the basic functionality of any virtual assistant. The prerequisite is only Python (> 3.6).

Usage and Features-

After installing the library you can import the module-

import JarvisAI
obj = JarvisAI.JarvisAssistant()
response = obj.mic_input()
print(response)

Check this script for more examples- https://github.com/Dipeshpal/Jarvis-Assisant/blob/master/scripts/main.py

Available Methods-

The functionality is cleared by methods name. You can check the code for example. These are the names of available functions you can use after creating JarvisAI’s object-

import JarvisAI
obj = JarvisAI.JarvisAssistant()
response = obj.mic_input()

res = obj.mic_input(lang=’en’)
res = obj.website_opener(“facebook”)
res = obj.send_mail(sender_email=None, sender_password=None, receiver_email=None, msg=”Hello”)
res = obj.launch_app(“edge”)
res = obj.weather(city=’Mumbai’)
res = obj.news()
res = obj.tell_me(topic=’India’, sentences=1)
res = obj.tell_me_time()
res = obj.tell_me_date()
res = obj.shutdown()
res = obj.text2speech(text=’Hello, how are you?’, lang=’en’)
res = obj.datasetcreate(dataset_path=’datasets’, class_name=’Demo’, haarcascade_path=’haarcascade/haarcascade_frontalface_default.xml’, eyecascade_path=’haarcascade/haarcascade_eye.xml’, eye_detect=False, save_face_only=True, no_of_samples=100, width=128, height=128, color_mode=False)
res = obj.face_recognition_train(data_dir=’datasets’, batch_size=32, img_height=128, img_width=128, epochs=10, model_path=’model’, pretrained=None, base_model_trainable=False)
res = obj.predict_faces(class_name=None, img_height=128, img_width=128, haarcascade_path=’haarcascade/haarcascade_frontalface_default.xml’, eyecascade_path=’haarcascade/haarcascade_eye.xml’, model_path=’model’, color_mode=False)
res = obj.setup()
res = obj.show_me_my_images()
res= show_google_photos()
res = tell_me_joke(language=’en’, category=’neutral’)
res = hot_word_detect(lang=’en’)

The above methods are very clear by their name itself. Read on GitHub or Pypi’s Jarvis_AI official page for more information.

2.2. Code-

Step 1 (Imports and Create Object of JarvisAI)-

import JarvisAI
import re
import pprint
import random

Object creation of JarvisAI as per as documentation–

obj = JarvisAI.JarvisAssistant()

We have created this ‘t2s(text)’ function. This will convert any text to speech. The entire program we will use (call) this function to produce speech from text.

Step 2 (Create text2speech or speak function)-

def t2s(text):
    obj.text2speech(text)

Step 3 (continuously listen on background and HOT WORD Detection)-

We want to continuously listen to input from the user, so we will use while loop here.

We will use ‘hot_word_detect()’ function. Why? What it is?

I guess no need to explain what it is.

This hot_word_detect will start listening your voice in the background. So this hot word detection will return 2 things ‘status’ and ‘command’. Status is boolean and command is the text whatever you speak. If your desire hot_word is detected then it will return status=True.

while True:
    status, command = obj.hot_word_detect()
    if status:
        # We will do something...

Okay, suppose your run this much of code. It will first ask to setup few things, if you are running the program for the first time.

Once done, it will create a “config/config.ini” file. It has configuration related to your virtual assistant.

Now run the script again, it will start listing your voice in background.

To wake up assistant you need to say your hot word. Hot word is your bot_name in config.ini file.

So, you can says- “Hello alexa”, “Hi alexa”, “blah blah alexa”. Only one saying hot word your assistant will we wake up. Otherwise it will keep listening in background.

Step 4 (After Hot Word Detection, Give Commands via microphone)-

So, if status is “TRUE” then we want to use mic_input() this will listen your voice again from your microphone. Now you can say anything. It will convert your voice into text.

while True:
    status, command = obj.hot_word_detect()
    if status:
        while True:
            res = obj.mic_input()
            print(res)

Step 5 (Perform action according to user command, Let’s implement Jokes)-

Suppose user says- “Tell me a joke”.

Now we will use regex to match desire word in user input. We want to match “jokes|joke|Jokes|Joke” any of these patterns in “res” below.

(Note: ‘res’ stands for ‘response’ here. I am super lazy so that’s why I wrote ‘res’)

while True:
    status, command = obj.hot_word_detect()
    if status:
        while True:
            res = obj.mic_input()
            print(res)

            if re.search("jokes|joke|Jokes|Joke", res):
                joke_ = obj.tell_me_joke('en', 'neutral')
                print(joke_)
                t2s(joke_)
                break

If pattern (“jokes|joke|Jokes|Joke”) found in “res” then we want to call tell_me_joke(‘en’, ‘neutral’). This will return single joke as ‘str’. We can print this and also use our “t2s” function to speak (convert this text into voice).

Step 6 (Now Implements other actions as well)-

One more example-

Implement Date and Time: It will tell you the current date and time of your system.

if re.search('date', res):
        date = obj.tell_me_date()
        print(date)
        print(t2s(date))
        breakif re.search('time', res):
        time = obj.tell_me_time()
        print(time)
        t2s(time)
        break

You can define your if-else based logic as above we used in “jokes” and call any of these methods accordingly-

website_opener_res = obj.website_opener(“facebook”)
mail_res = obj.send_mail(sender_email=None, sender_password=None, receiver_email=None, msg=”Hello”)
launch_app_res = obj.launch_app(“edge”)
weather_res = obj.weather(city=’Mumbai’)
news_res = obj.news()
topic_res = obj.tell_me(topic=’India’, sentences=1)
time_res = obj.tell_me_time()
date_res = obj.tell_me_date()
shutdown_res = obj.shutdown()
setup_res = obj.setup()
local_dir_images_res = obj.show_me_my_images()
google_photos_res = show_google_photos()
joke_res = tell_me_joke(language=’en’, category=’neutral’)
wake_word_res = hot_word_detect(lang=’en’)
datasetcreate_res = obj.datasetcreate(dataset_path=’datasets’, class_name=’Demo’, haarcascade_path=’haarcascade/haarcascade_frontalface_default.xml’, eyecascade_path=’haarcascade/haarcascade_eye.xml’, eye_detect=False, save_face_only=True, no_of_samples=100, width=128, height=128, color_mode=False)
face_recognition_train_res = obj.face_recognition_train(data_dir=’datasets’, batch_size=32, img_height=128, img_width=128, epochs=10, model_path=’model’, pretrained=None, base_model_trainable=False)
predict_faces_res = obj.predict_faces(class_name=None, img_height=128, img_width=128, haarcascade_path=’haarcascade/haarcascade_frontalface_default.xml’, eyecascade_path=’haarcascade/haarcascade_eye.xml’, model_path=’model’, color_mode=False)

3. Complete code-

Demo Script-

Dipeshpal/Jarvis-Assisant

Jarvis Assisant is Virtual Assistant to do basic task - Dipeshpal/Jarvis-Assisant

github.com

4. Github Repository

You can feel free to use my code. Star it if you like my work, subscribe on YouTube if you love.

Just clone the repository- https://github.com/Dipeshpal/Jarvis-Assisant.git

Then run pip install -r requirements.txt

It will automatically install everything.

5. How you can contribute-

By Suggesting Ideas: https://github.com/Dipeshpal/Jarvis_AI/discussions

By finding Bugs: https://github.com/Dipeshpal/Jarvis_AI/issues

By Contributing on code: Just open this GitHub repository, read it you will understand how you can contribute.

Your contribution will reflect on this project.

6. References-

GitHub repository and Code Demo: https://github.com/Dipeshpal/Jarvis-Assisant
GitHub Pypi repository to contribute: https://github.com/Dipeshpal/Jarvis_AI
JarvisAI Library: https://pypi.org/project/JarvisAI/
YouTube Channel: https://www.youtube.com/DIPESHPAL17
Demo and Code (YouTube) version 1: https://youtu.be/LliTjuxDw_o
Demo and Code (YouTube) version 1: https://youtu.be/C-sKlSULLrI

7. How to stay updated about new builds and release?

JarvisAI Library: https://pypi.org/project/JarvisAI/
YouTube Channel: https://www.youtube.com/DIPESHPAL17

Buy me a coffee: https://www.buymeacoffee.com/dipeshpal
Don’t have money?
Subscribe me on YouTube: https://www.youtube.com/dipeshpal17