Cognitive Computing and Big Data Analytics

Maciej Duraj
Aug 12 · 5 min read
Image for post
Image for post
My own graphic design

Developing true artificial intelligence or a thinking computer has been mankind’s dream for decades. It is a notion that has been mystified through the use of science fiction literature and film over the years. However, it is still just a concept that many of us don’t believe is anywhere close to reality. The advancements in this regard, however, are closer to reality than you may think.


IBM wants to prove skeptics wrong and create a system capable of human-like thinking. It already has a computer capable of some sort of cognitive computing with the Watson.

The Watson has already proven itself by winning a game of Jeopardy.

According to IBM, “It analyzes natural language questions and content well enough and fast enough to compete and win against champion players at Jeopardy!”

Most computers rely on mathematics and algorithms. they generally cannot interpret or read natural language like humans can. However, the Watson has this ability and shows the potential for AI in big data.

Future Potential

According to IBM, true cognitive computing will involve AI thought processes such as learning:

“Using advanced algorithms and silicon circuitry, cognitive computers learn through experiences, find correlations, create hypotheses, and remember — and learn from — the outcomes.”

This will undoubtedly be done at a high level as the computers capable of such cognitive analysis will already have programming beyond mathematical algorithms and data retention. The learning may involve patters not seen in the programming — such as sensing changes in temperature and predicting future data collection from this analysis.

IBM mentioned an example how useful such computing systems may be:

“For example, a cognitive computing system monitoring the world’s water supply could contain a network of sensors and actuators that constantly record and report metrics such as temperature, pressure, wave height, acoustics and ocean tide, and issue tsunami warnings based on its decision making.”

The human brain is more complex and efficient than any human-made machine. So although matching such a thought process with a computer may be a pipe dream, at least for now, the potential of a machine being able to interpret things through the human-like thought process and learn from its own mistakes is really the first step in AI.

The idea is to combine the best of human qualities with the mathematical precision of a machine. A true AI-system may not necessarily be something we want or can comprehend (think Skynet).

Potential in Big Data Analytics

Using a cognitive system or a thinking computer for big data seems like the perfect fit. Think of something that is very much open to interpretations like predictive analytics. Big data today is read and interpreted mostly by data scientists and not computers. Computing systems and software allow data scientists to collect the data in data sets and later make sense of it all. A thinking computer could help this interpretation move along quicker and make sense of it.

Will there be a day where the data scientist is replaced by a thinking computer or a supercomputer with cognitive analysis abilities? IBM would like to think so, although that day may be a long way off.

Data scientists are humans and thus by nature come with a certain amount of bias or prejudice. Could a cognitive system eliminate this bias and judge the data independently of human fallacies?

I think that in theory yes, it can. However, we also have to keep in mind that cognitive computers are built by humans or companies that come with their own bias to begin with.

Robert Plant mentioned the idea of using systems being used to help search engine companies while the Roadrunner supercomputer was being decommissioned by the U.S. govt.

Search engines such as Google use Web crawlers for various tasks like finding less important websites and ranking them with others to see which will show up at the top of search results. A thinking system could be very useful in helping to rank or analyze sites that are based on data open to some interpretation, not just statistics and algorithms, which is what Web crawlers today seem to be based on.

Other ideas where such a system could be useful, and I mentioned earlier, is in predictive analysis. The thinking computer may be able to identify human patterns and read them before data sets are even gathered into structured data for further interpretation.

It can then draw on its huge database to predict outcomes or the best business practices from big data by analyzing companies that succeeded and ones that did not with various business strategies. This in theory can eliminate bias.

Current Understanding and Cognitive Power

According to the IBM Journal of research and Development, Watson actually defeated two highest ranked players in Jeopardy. If it was able to do this, why couldn’t it beat data scientists in their own game eventually as well. Obviously big data analytics and Jeopardy are not the same thing, but the concept remains the same of how Watson may prove as a worthy competitor in both endeavours.

According to technology columnist Steve Wildstrom’s blog, “Deep QA, Watson’s parent, is based on an open source project called Unstructured Information Management Architecture that aims to find answers to real-world question from the vast amounts of information that exists outside of structured databases. The Watson hardware is based on IBM’s Blue Gene/p series of commercial supercomputers.”

Prior to Watson, IBM had created a supercomputer called Deep Blue that is known as one of Watson’s predecessors. It became famous for defeating chess master Garry Kasparov in a series of games (although Kasparov won the first round and no side was overly decisive).

Both of these examples, show that IBM is creating advancements in AI at the hardware level that could find a home big data analytics. Even if such a system may not replace data scientists entirely, it can provide a great complimentary analysis to the unstructured data sets being looked at by companies from big data sets.

IBM wants students to compete on big data projects and spur new ideas or innovations based on its use. The company is spurring a push for Watson to make its way into universities. It will be interesting to see what results of this project, but it should be a great step forward for big data and cognitive systems merging in usage.

The Innovation

A place for variety of stories from different backgrounds

Sign up for The Innovation Digest

By The Innovation

Official newsletter of The Innovation Take a look

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Maciej Duraj

Written by

I am a tech journalist and an aspiring artist-graphic designer. My sites include and

The Innovation

A place for a variety of stories from different backgrounds

Maciej Duraj

Written by

I am a tech journalist and an aspiring artist-graphic designer. My sites include and

The Innovation

A place for a variety of stories from different backgrounds

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store