AI & Toys: How to explain Artificial Intelligence to children with autonomous toy car
A fun and educational project combining Lego, Arduino, Raspberry Pi and Artificial Intelligence for the event “Friends & Family day” at TotalEnergies Digital Factory.
Artificial intelligence (AI) is a fascinating and constantly evolving field. It’s important to introduce children to data literacy and AI from an early age, so they can understand the technologies that are shaping our world. In this article, I’m going to introduce you to a fun and educational project I set up to introduce children aged 7 and over to AI using a Lego autonomous car demonstrator.
Equipment
· Lego: to build the chassis of the car.
· Raspberry Pi: to run the Computer Vision model and control the servo motor to turn the wheels and the DC motor.
· Webcam: to capture images of the road and send them to the Raspberry Pi.
· Arduino servo motor: to turn physically the wheels.
· DC Motor & L298N : to move the car.
Assembly
1. The chassis of the car
The car chassis is the structure that supports the vehicle’s mechanical and electrical components. It must be solid, light and adapted to the size of the wheels and engine. The chassis of the toy car we are going to build has its own technical constraints, in particular the maximum angle of rotation of the front wheels. This angle determines the car’s ability to turn and change direction. It depends on the type of servo motor used, the way it is attached to the chassis and the geometry of the wheels. We’ll look at how to choose the right servo motor and how to connect it to the Arduino to control the wheels.
2. Servo motor connection
The servo motor is a device used to control the angular position of a shaft. It consists of an electric motor, a speed reducer and a position sensor. The servo motor receives electrical impulses from the Arduino, which tells it the angle it needs to reach. Depending on this angle, the servo motor will turn the car’s front wheels, changing its direction of travel. The maximum angle that the wheels can reach is limited by the car’s chassis, which can come into contact with them if the angle is too great. This maximum angle must therefore be measured and transmitted to the Python code that will control the servo motor. This prevents the servo motor from being forced against the chassis and being damaged.
Connect the servomotor to the GPIO port on the Raspberry Pi, observing the correct polarity:
Red wire: 5V Power #2,
Black wire: Ground #6
Yellow wire: GPIO 12
3. DC motor connection
Connect the DC motor to the L298N module, then the L298N to the GPIO port on the Raspberry Pi, observing the correct polarity:
Motor 1
Enable A (green wire): GPIO 21
Input 1 (blue wire): GPIO 20
Input 2(orange wire):GPIO 16
Motor 2
Enable B (green wire): GPIO 25
Input 3 (blue wire): GPIO 23
Input 4 (orange wire):GPIO 24
4. Powering up the Raspberry Pi
The Raspberry Pi will be the brain of the mechanism. It will receive the images of the line captured by the webcam, process them with the Tensorflow Lite computer vision model, and send instructions to the Arduino servo motor to orientate the wheels.
The Raspberry Pi uses the Python programming language, and to use the Tensorflow Lite model, you need to install two libraries:
- tflite-runtime, which enables the model to be loaded and run on mobile and embedded devices. It enables machine learning inference on the device with low latency and small bit size.
- python3-opencv which is an open source library that includes several hundred computer vision algorithms. It allows you to use the webcam, process the images and transmit them to the model.
In order to use these libraries, you will need to run the following command to install the Python library:
sudo python3 -m pip install tflite-runtime --break-system-packages
sudo apt-get install python3-opencvArtificial Intelligence: Computer Vision model
To recognize the line, I used a Computer Vision model based on Tensorflow Lite, a lightweight version of Tensorflow that allows models to be deployed on low-power devices such as the Raspberry Pi. There are two ways to train a Tensorflow Lite Computer Vision model: using Python code, or using a No-code platform by Google, Teachable Machine.
Computer vision?
Computer Vision is a family of artificial intelligences based on image recognition. Image recognition in computer vision is:
- The task of identifying and classifying specific objects, people, text and actions within digital images and videos.
- An application of artificial intelligence and machine learning that uses algorithms and models to process visual data accurately and efficiently.
The Approach
To orientate the car in relation to the black line representing the circuit, I used an approach based on colour recognition. I installed a webcam on the front of the car, which sends images to a computer for processing.
From each image, I keep only the top part, which gives me an indication of the overall direction to follow. I then divide this part into n images of equal width, corresponding to different possible angles for the servo motor that controls the car’s direction.
For each image, I check whether there is any black, which means that the line is present. If so, I calculate the degree associated with the image and send the command to the servomotor to position itself at that angle. So the car follows the black line based on the colour. In the next section, I’ll explain how to detect black in cut-out images.
The Python Script: main.py
Option 1: Line recognition by colour
OpenCV is an open source library for image processing and computer vision. It offers functions for detecting contours, colours, shapes, faces, etc. In this first approach, we’re going to use OpenCV to identify the black colour on a white background, which corresponds to the line we want to follow.
[sample of code, the complete one is on the option3]
######
#
## We test for all pictures if there is a black color for the line
#
#####
# Convert the imageFrame in
# BGR(RGB color space) to
# HSV(hue-saturation-value)
# color space
hsvFrame = cv2.cvtColor(crop_image, cv2.COLOR_BGR2HSV)
# Set range for Black color and
# ret will return a true value if the frame exists otherwise False
into_hsv =cv2.cvtColor(crop_image,cv2.COLOR_BGR2HSV)
# changing the color format from BGr to HSV
# This will be used to create the mask
L_limit=np.array([0,0,0]) # setting the black lower limit
U_limit=np.array([211,36,56]) # setting the black upper limit
# creating the mask using inRange() function
# this will produce an image where the color of the objects
# falling in the range will turn white and rest will be black
b_mask=cv2.inRange(into_hsv,L_limit,U_limit)
dark=cv2.bitwise_and(crop_image,crop_image,mask=b_mask)
# Creating contour to track red color
contours, hierarchy = cv2.findContours(b_mask,
cv2.RETR_TREE,
cv2.CHAIN_APPROX_SIMPLE)
if len(contours) >1:
#There is a black lineThis approach is very effective IF, AND ONLY IF, the background is uniformly white. For example, in my case, as I don’t want to cover my entire floor with white sheets, I want the algorithm to be able to recognize when it’s a line on a white sheet from another line, for example the lines on my parquet floor.
Option 2: 100% Computer Vision
Another possibility for carrying out this task is to use an embedded TensorFlow Lite model in the Raspberry Pi. TensorFlow Lite is a lightweight version of TensorFlow that enables deep learning models to be deployed on low-power devices such as the Raspberry Pi.
There are two possible approaches to using TensorFlow Lite:
- A “low code” approach with Teachable Machine: Teachable Machine is an online tool that makes it easy to create custom machine learning models from visual, audio or text data. Simply upload your own examples, label them and start training. The model can then be exported in TensorFlow Lite format and copied to the Raspberry Pi. All that remains is to write a few lines of Python code to load the model and use it to make inferences about the images captured by the camera.
- A Python code approach: This approach requires more knowledge of programming and machine learning, but offers greater flexibility and control. You can use the TensorFlow framework to create and train your own deep learning model on a more powerful computer. The model can then be converted to TensorFlow Lite format using the converter provided by TensorFlow. The model can then be transferred to the Raspberry Pi and loaded with the TensorFlow Lite interpreter. Python code can then be written to make inferences about the images captured by the camera.
Once the model has been exported and put into your Raspberry Pi, here’s how to run it:
[sample of code, the complete one is on the option3]
model_path = "/home/florianbergamasco_RaspPi10/Desktop/AutonomousCar/model/model.tflite"
label_path = "/home/florianbergamasco_RaspPi10/Desktop/AutonomousCar/model/labels.txt"
############################################
#
# Using the tflite model
#
############################################
# Cerate interpreter for the specified model
interpreter = Interpreter(model_path=model_path)
# Read the label file and load all the values in an array
with open(label_path, 'r') as f:
labels = list(map(str.strip, f.readlines()))
def nb_classe(label_path, label_to_find):
# Read the label file and load all the values in an array
with open(label_path, 'r') as f:
labels = list(map(str.strip, f.readlines()))
id_choixclasse=int(labels.index(label_to_find))+1
return len(labels), id_choixclasse
def function_tflite_interpreter(model_path, label_path, image_label):
#Obtain input and output details of the model.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
# Obtain input size of image from input details of the model
input_shape = input_details[0]['shape']
size = input_shape[1:3]
# Fetch image & preprocess it to match the input requirements of the model
img = Image.open(image_label).convert('RGB')
img = img.resize(size)
img = np.array(img)
processed_image = np.expand_dims(img, axis=0)# Add a batch dimension
# Now allocate tensors so that we can use the set_tensor() method to feed the processed_image
interpreter.allocate_tensors()
interpreter.set_tensor(input_details[0]['index'], processed_image)
t1=time.time()
interpreter.invoke()
t2=time.time()
time_taken=(t2-t1)*1000 #milliseconds
############ --> Obtain results
predictions = interpreter.get_tensor(output_details[0]['index'])[0]
s=""
# for i in range(len(predictions)):
# if(predictions[i]>0):
# print("predictions["+str(i)+"]: ",predictions[i])
top_k = len(labels)
top_k_indices = np.argsort(predictions)[::-1][0:top_k]
tlabel_sorted=[]
tscore_sorted=[]
for i in range(top_k):
score=predictions[top_k_indices[i]]/255.0
lbl=labels[top_k_indices[i]]
tlabel_sorted.append(lbl)
tscore_sorted.append(score)
index_max_score=top_k_indices[0]
max_score=score=predictions[index_max_score]/255.0
max_label=labels[index_max_score]
return max_score, max_label, tscore_sorted,tlabel_sorted
###########################################
#
#
#
# MAIN
#
#
#
##########################################
Angle_voiture_direction_droit=100
Angle_Max_direction_gauche = 60
Angle_Max_direction_droite=140
#Initialise the servo motor at 90°.
Reinitialise_moteur_centre(Angle_voiture_direction_droit)
# initialize the camera
# If you have multiple camera connected with
# current device, assign a value in cam_port
# variable according to that
cam_port = 0
cam = cv2.VideoCapture(cam_port)
time.sleep(3)#lancement de la camera
previous_value=0
while 1:
# reading the input using the camera
result, image = cam.read()
# If image will detected without any error,
# show result
if result:
height, width, channels = image.shape
#####
# Cut of the pictures
####
nbPixelperDegre = width/(Angle_Max_direction_droite-Angle_Max_direction_gauche)
# we cut the picture to have 1 picture = 3°
nb_degre_par_picture = 3 #°
largeur_cut =int(nbPixelperDegre*nb_degre_par_picture)
height_pictureCut = int(height/8)
i=0
while i<int(width/largeur_cut):
y=i*largeur_cut
x=0
h=(i+1)*largeur_cut
w=height_pictureCut
crop_image = image[x:w, y:h]
if len(contours) >1:
cv2.imwrite('picture/frame_coped.jpg', crop_image)
image_label = 'picture/frame_coped.jpg'
max_score, max_label, tscore_sorted,tlabel_sorted = function_tflite_interpreter(model_path, label_path, image_label)
if float(max_score)>0.80 and len(max_label.split("line"))>1:
###
# There is a black line
##
i=i+1For more details, please refer to this article where everything has been explained step by step.
This approach works well but won’t fully meet the needs of the car because the execution of the model is SLOW and will have problems following the line.
Option 3: a hybrid approach
The hybrid approach combines the advantages of computer vision and colour detection. It enables the line to be tracked more accurately and quickly than either of the two approaches separately. The hybrid approach uses detection to detect a black line and then submits it to the computer vision model to determine whether it is a line defined for the car or another line (parquet floor, tiles, etc.).
This approach is the most suitable for this use case, providing better performance in a limited execution time.
Hereis the full script:
import cv2
import os
import shutil
from tflite_runtime.interpreter import Interpreter
import numpy as np
from PIL import Image
import time
import subprocess
import RPi.GPIO as GPIO
from datetime import datetime
import numpy as np
model_path = "/home/florianbergamasco_RaspPi10/Desktop/AutonomousCar/model/model.tflite"
label_path = "/home/florianbergamasco_RaspPi10/Desktop/AutonomousCar/model/labels.txt"
############################################
#
# Rotation of Servo Motor
#
############################################
#Set function to calculate percent from angle
def angle_to_percent (angle) :
if angle > 180 or angle < 0 :
return False
start = 4
end = 12.5
ratio = (end - start)/180 #Calcul ratio from angle to percent
angle_as_percent = angle * ratio
return start + angle_as_percent
def tourne_servoMotor(AngleTourne):
GPIO.setmode(GPIO.BOARD) #Use Board numerotation mode
GPIO.setwarnings(False) #Disable warnings
#Use pin 12 for PWM signal
pwm_gpio = 12
frequence = 50
GPIO.setup(pwm_gpio, GPIO.OUT)
pwm = GPIO.PWM(pwm_gpio, frequence)
#Init at 0°
pwm.start(angle_to_percent(AngleTourne))
time.sleep(1)
#Close GPIO & cleanup
pwm.stop()
GPIO.cleanup()
def angle_par_classe (nbclasse, choixclasse):# on fait un angle maximum de 180°
valeurangle = (180/(nbclasse-1))*(choixclasse-1)
return valeurangle
def Reinitialise_moteur_centre(val):
tourne_servoMotor(val)
############################################
#
# Using the tflite model
#
############################################
# Cerate interpreter for the specified model
interpreter = Interpreter(model_path=model_path)
# Read the label file and load all the values in an array
with open(label_path, 'r') as f:
labels = list(map(str.strip, f.readlines()))
def nb_classe(label_path, label_to_find):
# Read the label file and load all the values in an array
with open(label_path, 'r') as f:
labels = list(map(str.strip, f.readlines()))
id_choixclasse=int(labels.index(label_to_find))+1
return len(labels), id_choixclasse
def function_tflite_interpreter(model_path, label_path, image_label):
# Cerate interpreter for the specified model
#Obtain input and output details of the model.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
# Obtain input size of image from input details of the model
input_shape = input_details[0]['shape']
size = input_shape[1:3]
# Fetch image & preprocess it to match the input requirements of the model
img = Image.open(image_label).convert('RGB')
img = img.resize(size)
img = np.array(img)
processed_image = np.expand_dims(img, axis=0)# Add a batch dimension
# Now allocate tensors so that we can use the set_tensor() method to feed the processed_image
interpreter.allocate_tensors()
interpreter.set_tensor(input_details[0]['index'], processed_image)
t1=time.time()
interpreter.invoke()
t2=time.time()
time_taken=(t2-t1)*1000 #milliseconds
############ --> Obtain results
predictions = interpreter.get_tensor(output_details[0]['index'])[0]
s=""
# for i in range(len(predictions)):
# if(predictions[i]>0):
# print("predictions["+str(i)+"]: ",predictions[i])
top_k = len(labels)
top_k_indices = np.argsort(predictions)[::-1][0:top_k]
tlabel_sorted=[]
tscore_sorted=[]
for i in range(top_k):
score=predictions[top_k_indices[i]]/255.0
lbl=labels[top_k_indices[i]]
tlabel_sorted.append(lbl)
tscore_sorted.append(score)
index_max_score=top_k_indices[0]
max_score=score=predictions[index_max_score]/255.0
max_label=labels[index_max_score]
return max_score, max_label, tscore_sorted,tlabel_sorted
###########################################
#
#
#
# MAIN
#
#
#
##########################################
Angle_voiture_direction_droit=100
Angle_Max_direction_gauche = 60
Angle_Max_direction_droite=140
#Initialise the servo motor at 90°.
Reinitialise_moteur_centre(Angle_voiture_direction_droit)
# initialize the camera
# If you have multiple camera connected with
# current device, assign a value in cam_port
# variable according to that
cam_port = 0
cam = cv2.VideoCapture(cam_port)
time.sleep(3)#lancement de la camera
previous_value=0
while 1:
# reading the input using the camera
result, image = cam.read()
# If image will detected without any error,
# show result
if result:
height, width, channels = image.shape
#####
# Cut of the pictures
####
nbPixelperDegre = width/(Angle_Max_direction_droite-Angle_Max_direction_gauche)
# we cut the picture to have 1 picture = 3°
nb_degre_par_picture = 3 #°
largeur_cut =int(nbPixelperDegre*nb_degre_par_picture)
height_pictureCut = int(height/8)
## time.sleep(0.5)
table_presence_line=[]
i=0
while i<int(width/largeur_cut):
y=i*largeur_cut
x=0
h=(i+1)*largeur_cut
w=height_pictureCut
crop_image = image[x:w, y:h]
######
#
## We test for all pictures if there is a black color for the line
#
#####
# Convert the imageFrame in
# BGR(RGB color space) to
# HSV(hue-saturation-value)
# color space
hsvFrame = cv2.cvtColor(crop_image, cv2.COLOR_BGR2HSV)
# Set range for Black color and
# ret will return a true value if the frame exists otherwise False
into_hsv =cv2.cvtColor(crop_image,cv2.COLOR_BGR2HSV)
# changing the color format from BGr to HSV
# This will be used to create the mask
L_limit=np.array([0,0,0]) # setting the black lower limit
U_limit=np.array([211,36,56]) # setting the black upper limit
# creating the mask using inRange() function
# this will produce an image where the color of the objects
# falling in the range will turn white and rest will be black
b_mask=cv2.inRange(into_hsv,L_limit,U_limit)
dark=cv2.bitwise_and(crop_image,crop_image,mask=b_mask)
# Creating contour to track red color
contours, hierarchy = cv2.findContours(b_mask,
cv2.RETR_TREE,
cv2.CHAIN_APPROX_SIMPLE)
if len(contours) >1:
#There is a black line
now = datetime.now()
now=str(now)
now="".join(now.split(':'))
now="".join(now.split(' '))
now="".join(now.split('.'))
cv2.imwrite('picture/frame_coped'+str(now)+'.jpg', crop_image)
image_label = 'picture/frame_coped'+str(now)+'.jpg'
max_score, max_label, tscore_sorted,tlabel_sorted = function_tflite_interpreter(model_path, label_path, image_label)
if float(max_score)>0.80 and len(max_label.split("line"))>1:
nb=0
while nb<len(contours):
table_presence_line.append(i)
nb=nb+1
i=i+1
##########
#
###. TURN THE CAR
#
#
### --> Move the servomotor with the right Angle
#
#########
Orientation_a_donner_voiture=Angle_voiture_direction_droit
if len(table_presence_line)==0 :
Orientation_a_donner_voiture=previous_value
if len(table_presence_line)>0 :
#on cherche le milieu du trait
Centre_line = sum(table_presence_line)/len(table_presence_line)
Orientation_a_donner_voiture =Angle_Max_direction_gauche+(Centre_line * nb_degre_par_picture)
if previous_value!= Orientation_a_donner_voiture :
tourne_servoMotor(Orientation_a_donner_voiture)
previous_value=Orientation_a_donner_voiture
# If captured image is corrupted, moving to else part
else:
print("No image detected. Please! try again")
The DC Motors: turn.py
The DC motor and the L298N module play a crucial role in robotics projects with the Raspberry Pi. On one hand, the DC motor is responsible for converting electrical energy into mechanical motion, enabling devices to move or perform actions. The L298N, on the other hand, is a motor driver that acts as an H-bridge, allowing the direction and speed of the motor to be controlled by reversing the current or adjusting the applied voltage.
Voltage and pulse width modulation (PWM) are essential for controlling motor speed. By adjusting the voltage applied to the motor, its speed can be controlled. PWM is a technique that modulates the width of current pulses to finely regulate motor speed. Increasing the pulse width (i.e. increasing the duty cycle) increases the motor speed, while decreasing it reduces the speed. This precise control capability is crucial for robotics and automation applications where precise, repeatable movements are required.
A very good tutorial is accessible right here.
import RPi.GPIO as GPIO
from time import sleep
import os
# Define pins
M1_En = 21
M1_In1 = 20
M1_In2 = 16
M2_En = 25
M2_In1 = 23
M2_In2 = 24
# Setup
GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
GPIO.setup(M1_En, GPIO.OUT)
GPIO.setup(M1_In1, GPIO.OUT)
GPIO.setup(M1_In2, GPIO.OUT)
GPIO.setup(M2_En, GPIO.OUT)
GPIO.setup(M2_In1, GPIO.OUT)
GPIO.setup(M2_In2, GPIO.OUT)
# Voir aide dans le tuto
M1_Vitesse = GPIO.PWM(M1_En, 100)
M2_Vitesse = GPIO.PWM(M2_En, 100)
M1_Vitesse.start(50)
M2_Vitesse.start(50)
def sens1() :
GPIO.output(M1_In1, GPIO.LOW)
GPIO.output(M1_In2, GPIO.HIGH)
GPIO.output(M1_En, GPIO.HIGH)
GPIO.output(M2_In1, GPIO.LOW)
GPIO.output(M2_In2, GPIO.HIGH)
GPIO.output(M2_En, GPIO.HIGH)
def arretComplet() :
GPIO.output(Pins[0][1], GPIO.LOW)
GPIO.output(Pins[0][2], GPIO.LOW)
GPIO.output(Pins[1][1], GPIO.LOW)
GPIO.output(Pins[1][2], GPIO.LOW)
print("Motors stoped.")
nb=True
while nb==True:
sens1()
sleep(1)
arretComplet()
sleep(0.2)
M1_Vitesse.stop()
M2_Vitesse.stop()
GPIO.cleanup()Launching the scripts
To launch the two scripts without using an interface, either connect to your car using SSH, or instruct the two scripts to start when the Raspberry Pi is switched on with the command “ crontab -e ”:
You can use a console mode editor like nano.
Simply add the following line with the @reboot option:
@reboot python3 /home/florianbergamasco_RaspPi10/Desktop/AutonomousCar/main.py &
@reboot python3 /home/florianbergamasco_RaspPi10/Desktop/AutonomousCar/turn.py &It is important to note 2 points:
1. Use absolute paths
2. Add the & at the end of the line to launch your command in a separate process. This avoids blocking the Raspberry Pi’s start-up if your software doesn’t give up quickly.
When you talk about an autonomous car, you also talk about electricity…
To power the Raspberry Pi, I used a USB battery that plugs directly into the micro-USB port on the board. This battery gives me enough autonomy to drive the car without cables. The USB battery I chose has a capacity of 20,000 mAh, which means it can supply 20 A for one hour, or 1 A for 20 hours.
Of course, you have to take into account the power consumption of the Raspberry Pi and the motors, which varies according to use. On average, I can run the car for about 14 hours on a full battery charge.
Here’s the battery I use: Adeqwat 20000 mAh (around 50€)
Conclusion
I hope this article has been useful and inspiring for you to use computer vision in your own projects. Of course, this is just a demonstrator, so I’ve also used a teaching aid adapted to my audience, to explain and acculturate young people to AI. Please don’t hesitate to contact me if you have any comments, questions or suggestions for improvement. Have fun with your autonomous Lego car!
