The Role Big Data Will Play in AI’s Future

By Sherry Tiao

AI has been hyped up for ages. Remember the 60s, and Herbert Simon predicting, “machines will be capable, within twenty years, of doing any work a man can do”?

Well, we know how that turned out.

In the years that followed, the promises of AI failed to materialize. There were some accomplishments, namely the neural networks of the 90s and in the past 10 years, predictive analytics.

But it didn’t live up to the glittering expectations. Humans were not replaced by robots. The world didn’t become massively more efficient because of AI. It didn’t launch us forward into a shining future.

Now the hype is back and this time, it’s different.

AI’s Recent Accomplishments

Suddenly, it feels like every day we’re hearing about new AI accomplishments. For example:

  • Google DeepMind’s AI software AlphaGo bested world Go champion Lee Sedol. More recently, AlphaGo won against Ke Jie, widely considered to be the number-one ranked player in the world.
  • At Tufts University, scientists and biologists reportedly programmed an AI that’s capable of developing a new scientific theory, and discovered how the regeneration process of the flatworm works.
  • IBM Watson defeated humans on Jeopardy

And AI is providing real business breakthroughs.

AI has been evolving throughout the years. But suddenly, it’s making waves and is the story behind many new advancements. So what’s different this time?

How AI Crossed the Chasm

Throughout the years some aspects of AI became steadily easier to do. Deep learning especially has given it a big leap, but over time, AI evolved with different techniques and algorithms to get better.

However, there are three recent trends that helped AI cross that chasm and go mainstream:

  • Cheap parallel computation, including leveraging GPUs
  • Big data
  • Better machine learning algorithms

These developments mean we can now teach the machines instead of programming them. Here’s why that’s important:

Before, when we wanted to automate a task by using artificial intelligence, we had to understand it completely. We had to write about its behaviors in complete detail and also all of the variables we could possibly think of, and then figure out how to represent it in a computer program.

The more complicated the task, the harder it was to automate it. And that’s why we don’t have robots whizzing around our home, fulfilling our every need — it was simply too difficult.

Big Data’s Relationship to AI

Now, with big data available, we can feed that data into a machine-learning algorithm that then learns how to reproduce the behavior. Collecting the data is far faster than trying to understand and provide for every single eventuality. Which means that we’re now speeding up progress on the AI front.

Think about the companies that are making waves with their AI discoveries. Google, Facebook and Amazon are just a few of the examples that spring to mind. Their accomplishments are possible precisely because of the massive, massive amounts of data they have on hand.

A revolution in automation is happening.

Of course, it’s not as easy as that.

The big data world has long been saying that machine learning will become the biggest disruptor for big data analytics. But here’s an issue, and it’s tied to the data itself.

Big data is essential to AI’s success. AI itself doesn’t reason and deduce the way human minds do. Instead, it learns through trial and error. That’s why having large amounts of data is more important than ever. The more data AI has, the more accurate it will become. They are true partners, and one would not be good without the other.

That’s why the key to further AI and machine learning accomplishments will be the ability to analyze that data.

So far, data scientists have been the heroes of AI. But just think. If so far, we’ve seen so many accomplishments when the ability to analyze the data is tied to a few, what can we accomplish wheneveryone, from everyday business analysts to data scientists, can take deep learning to another level and use it to solve real business problems?

Here’s what we’re looking forward to: taking the work of these data scientist heroes and bringing it into the real world. What can happen with the AI revolution once we broaden access to that data?

The Future of AI

Of course, the future we paint won’t happen right away.

Before it comes, we believe you’ll see demand and more breakthroughs in the world of data analytics, but there will be struggles with understanding, proper usage and talent. Analytics will struggle to keep up with the vision, as leaders envision what they want, and data scientists have to fit the reality to the vision.

But in the end, that new future will arrive. And big data, with easy access to data analytics will surely be a key part of it.

This article was originally published on the Datameer blog.