Hands-on AI: Learning How Computers Learn
Artificial intelligence, also known as AI, can take many forms. Whether somebody views AI as a sentient being with emotions and feelings as portrayed in science fiction or simply as an algorithm to boost sales, it has launched to the forefront of the conversation surrounding consumption and optimization in our modern society. Like most, I hold opinions surrounding the topic, but without ever having hands-on experience working with AI, I felt as if I couldn’t properly defend my stance on the subject without first learning how it works.
Our brains are incredibly powerful supercomputers. We perceive the world and calculate split-second responses to millions of stimuli without a second thought. So, how do we model the same complex process in a computer?
The answer lies in machine learning, an underlying component of artificial intelligence. Neural networks enable a computer to take a unique input from the user and generate a human interpretable output as a response. Before I started coding, I took time to understand how neural networks function in their simplest form. The following video from 3Blue1Brown gives a comprehensible overview of simple neural networks.
To summarize, neurons — each of which contains some value corresponding to an element of an input — comprise the makeup of a layer within a neural network. The programmer determines the number of layers within the network to optimize the computer’s completion of a predetermined task. The first layer, known as the input layer, forms the foundation of a neural network. Based on the values of the input layer, the neurons of each following layer obtain new values determined by mathematical equations. The neural network continues to run until it reaches the output layer, comprised of a group of predetermined neurons representing possible outcomes. In the context of the previous layers, the computer returns a single value from the output layer equal to an understandable answer.
Unlike humans, a computer does not perceive the outside world. As mentioned in the video, humans find the simple task of determining a handwritten number effortless, whereas an AI needs more context before deciding what a symbol means. Neural networks are highly effective because they enable a computer to identify patterns within data. For example, programmers construct networks to recognize handwritten numbers so output layers will return the predicted value based on analyzing a user-provided image.
After researching the underlying functionality of neural networks, I worked on applying these ideas to create an artificial intelligence chatbot. To construct my bot, I followed an online tutorial, writing each line by hand and pausing as I went along to take notes so that I understood the concepts.
I learned about the three main components of training a simple AI: pattern matching, training, and execution. In pattern matching, a programmer groups input patterns together so the computer can comprehend the data it receives. In the context of my chatbot, I provided the computer with phrases to look for within significant categories of conversational intention. The computer stores each of these intents within a single file, intentions.json. The AI will reference this file during training to influence its neural network. The computer knows nothing more about conversation than what the programmer includes in the intents.json file. Below is the intents.json file that I developed for my chatbot. In creating patterns for my bot, I did my best to include many possibilities within each intent category. Hence, it was easier for my AI to identify varied responses. I’ve shortened the length of the following code segment to improve readability.
{
"intents": [
{
"tag": "identification",
"patterns": [
"Who are you?",
"What is your name?",
"whats your name",
"what do you go by",
"what are you",
"what do I call you"
],
"responses": [
"My name is Chatterbox, your personal chatbot assistant! What can I do for you today?"
]
},
{
"tag": "greeting",
"patterns": [
"Hi",
"Hi there",
"Hello",
"What's up?",
"Hey",
"hey man",
"how do you do?",
"morning!",
"Good morning",
"EVening!",
"Good evening",
"How's it going?",
"what's popping",
"whattup",
"hey bro",
"what's going on?",
"what's up with you?"
],
"responses": [
"Hello there!",
"Hi there. How can I be of assistance?",
"Hey! What would you like to talk about?",
"Hello! How are you doing today?"
],
"context": [
""
]
},
{
"tag": "user_feeling_good",
"patterns": [
"I'm doing well",
"I'm feeling good",
"I'm doing great!",
"I'm awesome!",
"I'm great, thank you for asking",
"im good",
"im doing well",
"im great",
"great thanks",
"im well"
],
"responses": [
"I'm glad to hear you're doing well! What was your favorite part of your day?"
],
"context": [
""
]
},
{
"tag": "user_feeling_neutral",
"patterns": [
"I'm alright",
"im okay",
"im just alright",
"just alright",
"could be better",
"im doing so so",
"Doing alright",
"im doing okay",
"I've been alright"
],
"responses": [
"I'm sorry to hear that you're not feeling great. Is there anything that I can do to improve your day?"
],
"context": [
""
]
},
{
"tag": "user_feeling_bad",
"patterns": [
"I'm not good",
"im not good",
"bad",
"im doing bad",
"I'm doing bad",
"man this sucks"
],
"responses": [
"I'm so sorry to hear that you're feeling bad. What can I do to make your day better?"
],
"context": [
""
]
},
{
"tag": "goodbye",
"patterns": [
"Bye",
"See you later",
"Goodbye",
"Get lost",
"Till next time",
"bbye",
"bye bye",
"cya!",
"Have a good day",
"Good bye",
"Gotta go",
"I have to go",
"Gotta run",
"I've gotta head out",
"I'm going to go now",
"I am going to go now",
"See you next time",
"See you"
],
"responses": [
"See you later! Thank you for chatting with me :)",
"Have a nice day! I can't wait to talk again soon!",
"Goodbye! Make sure to come visit me again soon!"
],
"context": [
""
]
},
{
"tag": "thanks",
"patterns": [
"Thanks",
"Thank you",
"That's helpful",
"Awesome, thanks",
"Thanks for helping me",
"You're the best",
"You are the best",
"Thank you so much",
"You really helped me with that",
"Gee, thanks",
"I couldn't do it without you",
"I could not have done it without you",
"You're the best",
"You are the best",
"Great, thank you",
"Thank you for all of your help",
"Thx"
],
"responses": [
"No worries, I'm happy to help!",
"Of course! Any time!",
"It's my pleasure. Let me know if I can do anything else to help."
],
"context": [
""
]
}
...
}Next, we need to train our AI. Below is training.py, the script used to develop a neural network to train an AI to identify patterns in chat messages and respond accordingly. It uses the information provided by intents.json to build a neural network and group together text patterns into intention categories. This file is the driver of the AI, the collection of commands that comprises the “brain” that the computer will eventually use to interpret the text provided by a user.
# training.py
# Comments (lines not considered during runtime by python) are
# denoted by '#')
# For clarity, I've done my best to comment each line and describe
# its function in plain english
# Comments may not follow python coding conventions. This is to
# improve readability for the user.
# Import libraries
import random
import json
import pickle
import numpy as np
import tensorflow as tf
import nltk
from nltk.stem import WordNetLemmatizer
lemmatizer = WordNetLemmatizer() # initialize lemmatizer
# read intents file into python script
# where intents -> dict{} structure
intents = json.loads(open('intents.json').read())
# initialize list variables for words, classes and documents
words = []
classes = []
documents = []
ignore_letters = ['?', '!', '.', ','] # letters to ignore in training
for intent in intents['intents']: # loop through every intent
for pattern in intent['patterns']: # loop through every pattern
# tokenize the pattern into list[] word_list
# -> where tokenizing refers to splitting off each individual word
# into separate indicides of list
word_list = nltk.word_tokenize(pattern)
words.extend(word_list) # place word_list within words[] list
# append tuple containing the tokenized word list of each tag
# to documents[]
documents.append((word_list, intent['tag']))
# if the unique tag is not already within the classes[] list
if intent['tag'] not in classes:
# append that unique tag to classes[]
classes.append(intent['tag'])
# use list comprehension to group inflected forms of words together into one
words = [lemmatizer.lemmatize(word) for word in words if word not in ignoreLetters]
# sort together the lemmatized gorup of words
words = sorted(set(words))
# sort classes list
classes = sorted(set(classes))
# dump words and classes into .pkl files (used for training neural network)
pickle.dump(words, open('words.pkl', 'wb'))
pickle.dump(classes, open('classes.pkl', 'wb'))
# initialize training list
training = []
# initialize output layer based on the number of possible output scenarios
output_empty = [0] * len(classes)
# loop through documents list
for document in documents:
bag = [] # intialize holder list
# place tokenized words into word_patterns[] variable
word_patterns = document[0]
# within word_patterns, lemmatize each tokenized word
# -> where lemmatization involves grouping together words
# despite form (past, present, future, plural, etc)
word_patterns = [lemmatizer.lemmatize(word.lower()) for word in wordPatterns]
# for every word within words[] list
for word in words:
# append True if word is in our lemmatized word patterns
# otherwise append False
bag.append(1) if word in word_patterns else bag.append(0)
# create a list of all of the output possibilities
# this will be our output layer
output_row = list(output_empty)
# for the index of every class of response, mark true
output_row[classes.index(document[1])] = 1
# append the bag of boolean statements regarding word patterns
# and output layer
training.append(bag + output_row)
# shuffle the training data
random.shuffle(training)
# turn it into a numpy array (different data structure that works in this context)
training = np.array(training)
# neural network training using tensorflow keras
# optimal arguments suggested by video.
train_x = training[:, :len(words)]
train_y = training[:, len(words):]
model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(128, input_shape=(len(train_x[0]),), activation = 'relu'))
model.add(tf.keras.layers.Dropout(0.5))
model.add(tf.keras.layers.Dense(64, activation = 'relu'))
model.add(tf.keras.layers.Dropout(0.5))
model.add(tf.keras.layers.Dense(len(train_y[0]), activation='softmax'))
sgd = tf.keras.optimizers.SGD(learning_rate=0.01, momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
# fit the model with training data
model.fit(train_x, train_y, epochs=200, batch_size=5, verbose=1)
model.save('chatbot_model.h5') # save chatbot model to be used by user
print('Done')Finally, chatbot.py plugs user input into the AI’s neural network, breaking down a statement into intention patterns so the computer can determine how to respond accordingly. Our AI will start to deal with input that it has never seen before, meaning that its output can potentially be flawed if it misinterprets what a user is trying to say. Based on the settings in our training script, however, we’ve done what’s possible to optimize our bot to handle confusing sentences to the best of its ability.
# training.py
# Comments (lines not considered during runtime by python) are
# denoted by '#')
# import libraries
import random
import json
import pickle
import numpy as np
import nltk
from nltk.stem import WordNetLemmatizer
from tensorflow.keras.models import load_model
lemmatizer = WordNetLemmatizer() # initialize our lemmatizer
intents = json.loads(open('intents.json').read()) # read intents file
words = pickle.load(open('words.pkl', 'rb')) # load .pkl files
classes = pickle.load(open('classes.pkl', 'rb'))
model = load_model('chatbot_model.h5') # load model
def clean_up_sentence(sentence): # function to simplify user input for computer
sentence_words = nltk.word_tokenize(sentence) # tokenize sentence
# lemmatize every word in sentence
sentence_words = [lemmatizer.lemmatize(word) for word in sentence_words]
# return cleaned words
return sentence_words
def bag_of_words(sentence): # returns group of words within user input
sentence_words = clean_up_sentence(sentence) # store the cleaned words
bag = [0] * len(words) # make a bag the length of words
for w in sentence_words: # for every word in cleaned words
for i, word in enumerate(words): # for the index and value of words
if word == w: # if the word is equal to current word
bag[i] = 1 # set the value of that word in bag to True
return np.array(bag) # return a numpy array of our bag
def predict_class(sentence): # method to predict the class intention from user
bow = bag_of_words(sentence)
res = model.predict(np.array([bow]))[0]
ERROR_THRESHOLD = 0.25
results = [[i, r] for i, r in enumerate(res) if r > ERROR_THRESHOLD]
results.sort(key=lambda x: x[1], reverse=True)
return_list = []
for r in results:
return_list.append({'intent': classes[r[0]], 'probability': str(r[1])})
return return_list
def get_response(intents_list, intents_json): # method to return a response
# return response based on intents.json
tag = intents_list[0]['intent']
list_of_intents = intents_json['intents']
for i in list_of_intents:
if i['tag'] == tag:
result = random.choice(i['responses'])
break
return result
print("BOT RUNNING\n") # bot activation
while True: # interact with user
message = input("")
ints = predict_class(message)
res = get_response(ints, intents)
print(res)When we finally run our model, it can correctly identify and respond to what a user says. Based on the intents.json file, the chatbot has multiple responses for each situation. The chatbot would have more capabilities and more varied responses with a larger intent file.
# User input denoted by ->
BOT RUNNING
-> hello
1/1 [==============================] - 0s 32ms/step
Hello there!
-> what's up
1/1 [==============================] - 0s 12ms/step
Hello! How are you doing today?
-> I'm doing okay
1/1 [==============================] - 0s 11ms/step
I'm sorry to hear that you're not feeling great.
Is there anything that I can do to improve your day?
-> tell me a joke
1/1 [==============================] - 0s 12ms/step
As I get older and I remember all the people I’ve lost along the way,
I think to myself, maybe a career as a tour guide wasn’t for me.
-> lol
1/1 [==============================] - 0s 12ms/step
I'm glad you thought that was funny!
COMPLETEDAlthough the bot does well in most situations, there are times when it misinterprets a user’s intention.
# User input denoted by ->
BOT RUNNING
-> What's up
1/1 [==============================] - 0s 32ms/step
Hi there. How can I be of assistance?
-> Tell me the weather
1/1 [==============================] - 0s 12ms/step
How 'bout that weather!
-> What's on the news
1/1 [==============================] - 0s 12ms/step
Hey! What would you like to talk about? (?)
-> show me the news
1/1 [==============================] - 0s 12ms/step
This just in: Charlie Brunold programmed me to respond this way.
-> I wanted to tell you that I'm doing great.
-> I have had an awesome day and later I'm thinking I'm running
-> down to the store to do some shopping.
1/1 [==============================] - 0s 12ms/step
I'm glad to hear you're doing well! What was your favorite part of your day?
-> Probably going shopping down at the store.
1/1 [==============================] - 0s 12ms/step
Hello there! (?)
COMPLETEDWith a larger model including trillions of data points, a programmer could iron out the inconsistencies of the chatbot. Through creating my bot, I’ve realized that the key to good artificial intelligence lies in the data. Although talking with my bot is the equivalent of talking to a ten-year-old, I know that if I had every piece of human information at my fingertips, it would perform incomprehensibly well.
After building out my chatbot, I’ve discovered that we’ve figured out how to imitate our humanity mathematically. The key to AI is that it’s simply synthesizing everything we’ve done in the past into something new and digestible. As a result, we spend so much time worrying that AI has it out for us when, in reality, we are the ones who have it out for ourselves. AI is humanity’s greatest mirror. It serves as a tool for reflection. I gave my bot a list of responses, but those don’t at all reflect the responses of somebody else.
I’m curious when we will start to understand that, fundamentally, we are the ones in control of AI. Anything “bad” that comes from the computer has at one point been generated and considered by a human being. That said, I wonder when we will take the time to regulate training data and create universally agreed-upon standards for data validation and omission. More people would benefit from the understanding that AI is an amazing tool. It would be a shame if we were to divide ourselves over it before we could even reap the rewards from its application.
With that being said, I wonder how far artificial intelligence will take us. We’ve developed systems to hold conversations, generate art and images, transcribe audio, and make calculations, but the one thing missing from AI is still human ingenuity. All of a computer’s capabilities lie on the back of human creativity. Will we ever reach a point where AI can synthesize new, unique ideas for the collective good? And in what form will that present itself? Will we have a bot telling us what policies to put into place to stop world hunger, or will we have an AI that tells each individual the unique steps they need to take to secure a well-paying job?
Below is a clip from the Lex Fridman podcast, where Sam Altman, the CEO of OpenAI, touches on the future of AI and his view on how he would react if artificial intelligence suddenly went beyond its current capabilities.
How different would our lives be, really, if suddenly AI generated an unknown theory explaining the secrets of life? Would we stop living the way that we have been living?
One day, I hope to find the answers to these pivotal questions through further exploration and discussion of artificial intelligence and how it relates to our existence as human beings.
Works Cited
3Blue1Brown. (2017, October 5). But what is a neural network? | Chapter 1, Deep Learning. YouTube. https://www.youtube.com/watch?v=aircAruvnKk&t=293s&ab_channel=3Blue1Brown
Fridman, L. (2023, March 25). Sam Altman: Openai CEO on GPT-4, chatgpt, and the future of AI | Lex Fridman Podcast #367. YouTube. https://www.youtube.com/watch?v=L_Guz73e6fw&ab_channel=LexFridman
NeuralNine. (2020, December 4). Intelligent AI chatbot in Python. YouTube. https://www.youtube.com/watch?v=1lwddP0KUEg&ab_channel=NeuralNine
