DataSeries
Published in

DataSeries

The Largest Hurdle with AI

Think back to a challenge in your life that once seemed impossible; perhaps, getting promoted to the position you’ve always wanted at work or sticking to a new exercise routine every day. What does it take to start exploring something new, even though it might be daunting at first? As I recently learned by starting to work with Artificial Intelligence, the most difficult part of overcoming challenges like these is convincing yourself to get started in the first place.

Artificial Intelligence isn’t just for Mad Scientists

Artificial Intelligence has had a lot of attention in the media in recent years. It seems that every day, highly-trained scientists are discovering a way to apply it in yet another industry. The thought that ordinary people could also discover the uses of this technology may seem laughable, especially given the technical knowledge required. Recently, however, I discovered this was more than possible by creating my first neural network to identify handwritten digits (if you’re not familiar with neural networks, check out my introduction on the topic on Medium).

What exactly does identifying handwritten digits entail? Basically, the neural network looks at [very blurry, thanks for the data U.S. government] images with numbers like this:

And then it decides what the number it is seeing is. This may seem like a straightforward and simple task. Why would anyone need something as complex as Artificial Intelligence to do this? Actually, it can be very hard to recognise handwritten digits. For instance, try identifying these:

You would probably agree that this is not as easy of a task, but neural networks can reach near-perfect accuracy in this (my basic neural network got up to about 94% accuracy, while professionally optimised versions have gotten up to 99.79% accuracy). Visual recognition such as this is used in many apps, like Google Translate and Search, but it also has more implicit uses; consider how self-driving cars process speed signs around them, for instance. Given these important and prevalent uses of classifier neural networks, I thought my first venture into the technology would be well placed in this field.

Learning Required? Yes. Learning Possible? Definitely

I had to learn new information about the technical aspects of Artificial Intelligence to do this, but it was quite manageable and only took a few days. The key lessons I had to learn were to actually understand why the algorithms behind neural networks worked, not just the technicalities of coding them. These details included:

  1. How data is used in neural networks;
  2. How neurons’ weights and biases (the settings) were adjusted;
  3. How weights and biases (the settings) affect the result;
  4. And how to minimise errors and make the network more accurate.

The first point, above all else, is sensitive to poor planning and execution. If your data isn’t adequate, the entire system doesn’t work. The U.S. government learned this expensive mistake when it attempted to make a neural network aid for tanks, but amusingly ended up with a multi-million dollar weather detector due to data issues (learn more here).

Of course, there were moments where the technical aspects of algorithms were quite complicated to grasp. For instance, neural networks have many algorithms that are based on calculus, with lovely equations and notations such as:

Understanding why the algorithms behind these equations worked, however, allowed me to get over this difficulty by still knowing what the network was doing at each step.

What if I Want to Try?

So you may now be wondering, “Well if it’s really so easy, can I just do it myself?” Yes! The main prerequisite you need is to know a common programming language like Python. These programming languages have many shortcuts, such as external libraries in Python, that allow you to overcome the difficulty of complicated equations like the ones above. Additionally, images of handwritten digits are freely available from the U.S. government’s MNIST database and there are many tutorials online that explain how and why neural networks work, instead of solely providing technical details.

In all, Artificial Intelligence may seem frustratingly out of reach to the average person, but the hardest part is actually just getting started. Indeed, some details look like they came from another planet, but understanding what the network is doing circumvents all that. After all, as Nelson Mandela said, “It always seems impossible until it’s done.”

If you want to see how my neural network works, you can find it on my Github Repository. If you’re interested in learning more about how neural networks work, check out these resources:

https://www.youtube.com/watch?v=aircAruvnKk&t=1000s

http://neuralnetworksanddeeplearning.com/chap1.html

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Madhav Malhotra

Cofounder at The Plastic Shift. Learning how to create a sustainable planet. Linkedin: linkedin.com/in/madhav-malhotra/