Published in



Making Drones Smarter

The Science Behind AI Drone Technology

By Valkyrie Holmes

Intel “Smart” Drones

The last few months have been filled with questions surrounding artificial intelligence. Oracle is using AI to automate digital marketing, Cellino presented a way to scale production of stem cell therapies at TechCrunch Disrupt using AI, and Codex is generating programs in 12 coding languages using artificial intelligence. But I’ve been interested in the science behind drone technology, specifically, and how we can use AI to revolutionize the firefighting industry.

For those who want to know more about Project Firefly, a plan that utilizes autonomous drone technology in conjunction with fluid dynamics to contain wildfires, you can view this article here. But I wanted to dive deeper into the autonomous side of things and figure out how drones learn to “see”.

What is AI?

Artificial Intelligence is basically the science of “making machines smart” using algorithms trained over time. It’s being used to revolutionize radiation therapy for cancers, help astronomers locate galaxies, and prevent fraud among other things. A major part of AI is the process of taking in vast amounts of data and training to perform simple tasks from the given information.

The goal is to create systems that can perform tasks independently. There are many different types of artificial intelligence:

  • Computer Vision is a symbolic learning pattern that takes image processing and uses it to train an algorithm to notice patterns. This is a form of classification.
  • Statistical Learning is a type of machine learning that takes into account past trends and uses them to match up new data, like speech recognition.
  • Machine Learning is one of the biggest subcategories of AI and it’s a form of pattern recognition that the algorithm uses to put certain pieces of information together to form the final piece.

There’s another part of machine learning called deep learning which is essentially when the computer forms a neural network that takes many different notions of what an image or pattern is and puts them all together to form a picture or final product. A convolutionary neural network can be used to recognize certain objects in a scene, which also connects to computer vision and object recognition.

Example of neural network layering

So we can break AI into two major categories: symbolic (image-based) and machine learning (data-based). You can use them to either classify things or predict trends.

Another important distinction to make is that when you have data embedded in the learning program itself, it’s called supervised learning. If you want the machine to figure out the pattern on its own, it’s considered unsupervised. A lot of the time, researchers will have a specific goal in mind and the machine has to go through a series of trials and errors to figure it out on its own, which is called reinforcement learning.

While machine learning is truly remarkable, there are some limitations on what it can accomplish. If you train one algorithm to accomplish a task, you can’t transfer that skill onto another algorithm without training it all over again. You also can’t train it to do complex or abstract tasks like feel emotions.

One of the biggest things we have to understand is that, unlike humans, robots don’t have the ability to learn in one shot. Think about a time where you’ve burned yourself on a hot pan or hit yourself on the side of the table. You felt that pain and realized it’s probably not a good idea to do that again, so you learned from one experience. Robots don’t have that capability so every small task has to be trained individually.

There’s also the problem of unintentional bias. When deploying AI algorithms into the workforce or even in justice systems, many have seen a racial bias being shown in these algorithms, which experts are already trying to fix.

Human Vision vs Computer Vision

Computer Vision: Piecing the Puzzle Together

Drones are unmanned, aerial devices that are usually remotely controlled by a human being on the ground. They’ve been continuously used in eCommerce for delivery purposes, construction for monitoring dangerous areas, agriculture for planting and maintaining crops, search and rescue, and most notably, the military.

By incorporating AI, drones can use data from sensors that measure visual and environmental factors to assist flight, which increases accessibility. Most rely heavily on computer vision, which can record information on the ground and detect moving objects, allowing for more high-performance, onboard image processing through a neural network. Information from object detection algorithms, classification, and tracking all combine to stop drones from crashing into things and track targets.

Types of annotation services (bounding box)

Let’s give a quick overview of how computer vision works.

Computer Vision is a subfield of deep learning and AI where computers are taught how to see and interpret the world. Machine vision was originally derived from an experiment done in 1999 measuring animal vision. The researchers placed an electrode in a cat’s visual cortex and showed it a series of images, hoping that it would fire off a signal in the animal’s brain. When it didn’t work, they took out the projector slide leaving a single line of light on the wall and suddenly, the visual cortex responded. Simple shapes like lines and curves could be interpreted by the cat because it had been trained to see bits and pieces of things and put them together, a lot like a deep neural network.

Soon after, we were training computers to process pixels as a certain set of values, pertaining to the presence and intensity of the colors red, green, and blue. A digital image is technically a matrix to a computer so therefore, computer vision is a study of matrices, using algorithms and linear algebra to manipulate them into a final product.

Think about it like this: when you and I see a cat, that’s just what we see: a furry animal called a cat. But when a computer sees a cat, it has to first learn all of the individual features that make up a cat, not exactly what a cat is. Neural networks work as though you have tons of puzzle pieces and you have to figure out which are the edges, what color goes where, and how to filter the image to come up with a final product.

1999 Experiment Illustration

There are many different types of computer vision like image segmentation (which splits images into multiple regions to be examined separately), object detection, and pattern detection but each one follows the same basic steps:

  1. Acquire an image (video, photo, 3D model, etc)
  2. Process the image through deep learning models (you have to train the models with thousands of pre-identified images first)
  3. Interpret the image and try to classify it into a learned category

With all that in place, we can train computers to do remarkable things!

The Power of Autonomous Drones

Drone technology takes advantage of propulsion and navigation systems, sensors, cameras, GPS tech, and computer vision to target destinations. Information from object detection algorithms, classification, and tracking also helps train the drone to respond to environmental conditions and analyze where to fly. This gives scientists real-time data, which is useful for assessing areas unsafe to humans or making high-speed calculations.

Object tracking and identification

For example, Above is using UAV drones to collect environmental statistics to assess damage from disasters or climate change. Shield AI assists ground forces and first responders in exploration and uses hivemind software to communicate with each other and identify survivors. Scale is using AI and machine learning to help train drones to better identify homes and cars on the map.

These companies and more are improving data annotation for aerial imaging and creating algorithms to track and recognize new objects. Self-navigation drones are being trained to find the most optimal way to move through obstacles without human control. There are tons of new developments happening every day and it’s honestly so remarkable! These advancements are going to be life-altering and more people should know about them.

How does AI impact your life? Artificial intelligence is everywhere and if you think you’re not exposed to it, think again! It’s on your computer, your phone, your Fitbit, your TV, and more! Think about how much of an impact this technology would make now and the results may shock you.

Smart drones are the future!

Read more:



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Valkyrie Holmes

Valkyrie Holmes

What’s up, my lovely people? I’m Valkyrie, an eighteen-year-old crypto-enthusiast and engineering student looking to educate the masses and disrupt industries.