Self Driving E-Rikshaw
INTRODUCTION:
In the modern era, the vehicles are focused to be automated to give human driver relaxed driving. In the coming years, we will essentially be able to do whatever we want on a car, be it having a nap, reading the news or having a video chat with friends. The car will do all the driving for us, allowing us to save time on the road. Autonomous vehicles would allow the transport to be precise and safe, thereby decreasing those unnecessary deaths. In the field of automobiles, various aspects have been considered which makes a vehicle automated. Since the inception of the commercial auto industry in the late 1890s, cars have become increasingly safe and convenient. Recently, carmakers have begun to introduce advanced driver-assistance systems such as adaptive cruise control (which automates accelerating and braking) and active lane assist (which automates steering). These systems have become capable enough that new luxury vehicles can drive themselves in slow-moving highway traffic. Research into autonomous cars has progressed remarkably since the first demonstrations in the 1980s. There are a lot of technologies which make self-driving cars possible. A lot of sensors like RADAR, LIDAR, GPS etc. are used in the self-driving cars along with advanced deep learning techniques involving object detection, computer vision and training of neural networks on different conditions that a car can go into. An autonomous vehicle needs sensory input devices like cameras, to allow the car to perceive the world around it, creating a digital map. We are focusing on imaging where cars perform object detection. The detection and tracking of objects around an autonomous vehicle are essential to operate safely. Object detection is a two-part process, image classification and then image localization. Image classification is determining what the objects in the image are, like a car or a person, while image localization is providing the specific location of these objects. We are training a convolutional neural network which will help a vehicle get a better understanding of its environment and get an artificial visionary power.
BACKGROUND AND OBSERVATION:
For generations, the automobile industry has been a source of innovation and economic growth. The ability to drive is a symbol of mobility and independence that spans generations. Automobiles play a significant role in our lives and afford many benefits to society.
Yet for all the benefits conferred on society, no other invention in the history of civilian technology has caused as much harm as the automobile. Every 30 seconds, someone dies in a traffic accident, adding up to well over 1 million deaths each year1. In the U.S., automobile accidents are the leading cause of death for people between the ages of 3 and 34. Moreover, human error is the cause of over 90% of automobile accidents.
Also, the inefficiencies related to automobile usage is staggering. Most automobiles sit unused more than 95% of their lifespan, and a freeway operating at maximum efficiency has automobiles on only 5% of its surface. In congested urban areas, 40% of all gasoline used is spent when cars circle to look for parking spaces4. Furthermore, in some U.S. cities, parking lots comprise more than a third of the land, becoming the single salient landscape feature of our built environment.
Autonomous vehicles could alleviate or completely solve these serious problems. The technology behind autonomous driving is typically divided into two categories: sensor-based implementation or a “Connected Vehicle” implementation. A sensor-based implementation is often an extension of current advanced safety features, while connected-vehicle technology (V2X) involves cars communicating with each other (V2V) and with infrastructure (V2I).
OBJECTIVE AND NEED
The self-driving vehicle is a technological advancement that is causing widespread wonder. We may not have arrived at a Jetsons-like future of flying automated vehicles just yet, but the self-driving vehicle is getting closer and closer to becoming a widespread, viable method of transportation. And, while many are sceptical about the future of driverless and self-driving vehicles on the road, there’s a good deal of data and research to support the idea that self-driving vehicles may be better for our future.
With that in mind, here are some reasons to embrace self-driving cars:
Our roads will be safer:
Self-driving vehicles, are purely analytical, relying on cameras, radar, and other sensors to navigate. There’s no emotion involved, and certainly no distractions like cell phones or impairing factors like alcohol to affect driving performance. The computers in a smart car simply react quicker than our minds can and aren’t susceptible to the many potential mistakes we can make on the road. As a result, a future full of self-driving vehicles will be a safer one.
We’ll be more productive:
While many already commute on buses or trains, a whole lot of people still travel to work in their cars. A self-driving vehicle would allow them to get some work done, knock a few emails out, or even get a little extra sleep if they have to wake up early to get to work. Many people find themselves tempted to look at their phones while driving anyway, so why not at least do it safely in a self-driving car.
We can save money
Self-driving vehicles are safer, they’ll cut down on any accident-induced costs, which means insurance premiums won’t skyrocket either. Additionally, self-driving cars don’t sustain the same kind of wear and tear as human-piloted vehicles. They don’t floor the gas pedal or slam the brakes unless an emergency is detected, making for better upkeep, slower depreciation, and even better fuel efficiency.
DEEP NEURAL NETWORKS BASED APPROACH
DATA PREPROCESSING
Before we get into building the various components of the object detection model, we will perform some preprocessing steps. The preprocessing steps involve resizing the images (according to the input shape accepted by the model) and converting the box coordinates into the appropriate form. Since we will be building an object detection for a self-driving car, we will be detecting and localizing eight different classes.
INTERSECTION OVER UNION:
It is used in object detection because it compares the ground truth bounding box with the predicted bounding box. By dividing the area of overlap with the area of the union, we have a function that rewards heavy overlapping and penalizes inaccurate bounding box predictions. The bounding box’s goal is to ultimately confine the object within the image as accurately as possible and IoU is a great metric for determining this.
DEFINING THE MODEL
You only look once (YOLO) is a state-of-the-art, real-time object detection system, which has an mAP on VOC 2007 of 78.6% and an mAP of 48.1% on the COCO test-dev. YOLO applies a single neural network to the full image. This network divides the image into regions and predicts the bounding boxes and probabilities for each region. These bounding boxes are weighted by the predicted probabilities.
One of the advantages of YOLO is that it looks at the whole image during the test time, so its predictions are informed by the global context in the image. Unlike R-CNN, which requires thousands of networks for a single image, YOLO makes predictions with a single network. This makes this algorithm extremely fast, over 1000x faster than R-CNN and 100x faster than Fast R-CNN.
MODEL ARCHITECTURE
The YOLO model has the following architecture (see Fig 3). The network has 24 convolutional layers followed by two fully connected layers. Alternating 1 × 1 convolutional layers reduce the features space from preceding layers. The convolutional layers are pre-trained on the ImageNet classification task at half the resolution (224 × 224 input image) and then double the resolution for detection.
IMPLEMENTING THE MODEL ON REAL DATA
When we need to perform object detection on real data, we use the feed from the video camera to do so. Video is a sequence of images at different time frames, so we will predict the class probabilities and bounding boxes for the image captured at each time frame. We will use OpenCV video capture function to read the video and convert it into image/ frames at different time steps. We then load the pretrained weights of the YOLOv2 model and freeze all the weights except for the last layer during the training of the model. We remove the last convolutional layer of the YOLOv2 model and replace it with a new convolutional layer indicating the number of classes to be predicted.
JETSON NANO
Jetson Nano is a small, powerful computer for embedded applications and AI IoT that delivers the power of modern AI in a $129 module. Get started fast with the comprehensive JetPack SDK with accelerated libraries for deep learning, computer vision, graphics, multimedia, and more.
To setup Jetson Nano, you can go to this site for a complete guide.
INTERFACING SOFTWARE AND HARDWARE
In computing, an interface is a shared boundary across which two or more separate components of a system exchange information. Software interfaces (programming interfaces) are the languages, codes and messages that programs use to communicate with each other and to the hardware. In our project, there must be communication between the hardware that is GPU (Graphics Processing Unit) and different subsystems of the E-auto. We know that for any automobile to operate properly we must have three main parts of the automobile working that are
- Acceleration System.
- Braking System
- Steering System.
These three subsystems of the auto are connected to motor drivers and connectors, which will be controlled by the GPU.
On JETSON NANO we have various GPIO pins to help interface various hardware components to it.
ESC (ELECTRONIC SPEED CONTROLLER SYSTEM)
In E-Auto we have a BLDC (Brushless DC) motor that controls the acceleration of the auto.
A BLDC motor consists of two main parts, a stator and a rotor. For this illustration, the rotor is a permanent magnet with two poles, while the stator consists of coils arranged as shown in the picture below.
We control the Electronic Speed Controller with the GPU that is Jetson Nano. We first connect a SERVO Motor Kit with the ESC to help generate PWM signals for the ESC. As to control the ESC we need to generate PWM signals that are PULSE WIDTH MODULATION signals. Pulse Width Modulation (PWM) Signal is a method for generating an analog signal using a digital source. A PWM signal consists of two main components that define its behaviour: a duty cycle and a frequency.
EBS (ELECTRONIC BRAKING SYSTEM)
In the E-Auto we have a Drum Brake that controls the Braking System of the auto. A drum brake is a brake that uses friction caused by a set of shoes or pads that press outward against a rotating cylinder-shaped part called a brake drum. The term drum brake usually means a braking system in which shoes press on the inner surface of the drum. When shoes press on the outside of the drum, it is usually called a clasp brake. Where the drum is pinched between two shoes, similar to a conventional disc brake, it is sometimes called a pinch drum brake, though such brakes are relatively rare. Drum brakes have been around for almost as long as the automobile itself and are still fitted today in a modified and more sophisticated form in modern cars. The term drum brake describes the design principle: namely, an enclosed cylindrical structure.
WORKING PRINCIPLE:
When the driver steps on the brake pedal, the power is amplified by the brake booster (servo system) and changed into hydraulic pressure (oil-pressure) by the master cylinder. The pressure reaches the brakes on the wheels via tubing filled with brake oil (brake fluid). The delivered pressure pushes the pistons on the brakes of the four wheels. The pistons press the brake linings, which are friction materials, against the inside surfaces of the brake drums which rotate with the wheels. The linings are pressed on the rotating drums, which in turn decelerate the wheels, thereby slowing down and stopping the vehicle.
We designed a piston system to control the braking system of the auto. The Piston system is controlled by a motor that is connected to the GPIO Pins of the Jetson Nano.
A piston is a moving disk enclosed in a cylinder which is made gas-tight by piston rings. The disk moves inside the cylinder as a liquid or gas inside the cylinder expands and contracts. A piston aids in the transformation of heat energy into mechanical work and vice versa. Such a system is known as a reciprocating engine.
The engine consists of a fixed cylinder and a moving piston. The expanding combustion gases push the piston, which in turn rotates the crankshaft. After the piston compresses the fuel-air mixture, the spark ignites it, causing combustion. The expansion of the combustion gases pushes the piston during the power stroke.
We control the piston system with a servo motor that takes signals from the GPU and help control the Piston system that in turn controls the braking system of the auto.
STEERING SYSTEM
Steering is the collection of components, linkages, etc. which allows any vehicle (car, motorcycle, bicycle) to follow the desired course. The primary purpose of the steering system is to allow the driver to guide the vehicle. there are a rack and pinion steering system. This converts the rotational motion of the steering wheel into the linear motion that turns the wheels and guides your path. The system involves a circular gear (the steering pinion) which locks teeth on a bar (the rack).
A stepper motor is an electromechanical device it converts electrical power into mechanical power. To control the Steering of the auto, we have used a Stepper Motor. Stepper motors are DC motors that move in discrete steps. They have multiple coils that are organized in groups called “phases”. By energizing each phase in sequence, the motor will rotate, one step at a time. With a computer-controlled stepping, you can achieve very precise positioning and/or speed control. When the stator’s vector magnetic field is rotated by an angle, the rotor also rotates with the magnetic field at an angle. Each time an electrical pulse is an input, the motor rotates one degree further.
ARCHITECTURE OF THE SYSTEM
When we interface all the components of the system that we require to convert the e-vehicle to a self –driving vehicle, we get the underlining architecture of the system as follows:
INTERACTION MODEL
The user will enter the details of the destination it wants to reach and the system will return the map of the path it will follow to reach the destination. Upon the confirmation by the user, the system will start its self-driving mode and reach the destination.
DATA PROCESSING MODEL
SOME IMAGES :
CONCLUSION:
The three common tasks, namely object tracking, safe lane detection and pedestrian detection were implemented in the project. As we have implemented the prototype of Self Driving vehicle with a single camera, we have found this promising in the use cases where we have marked lanes and light traffic. This fits best in campus-like scenario of colleges, universities and even multinational companies. Contemporary developments in Self Driving vehicles reflect the vivid future that these vehicles behold. According to official predictions, most cars are expected to be fully autonomous by 2030s. Our project is a step forward in developments and advancements taking place in this area of research.
