How We Are Riding The Wave of The Data Tsunami
Grab your umbrellas, it’s raining data. According to an article published by Forbes, we’re about to be hit by waves of data. It turns out, businesses are collecting and analyzing more data than ever before. The article, Are AI And ML The Answers To The Data Tsunami?, questions whether businesses can keep pace with the influx of data and rapid advancements in technology.
Can they use AI and ML to ride the wave? Yes. And also no. Well, partly.
The right answer in our estimation is complex. AI and ML can certainly help, but they are only part of the answer. Let’s take a step back and look at why we’re actually faced with this rise in data and the need to manage it at all.
It’s more akin to global warming than a tsunami
First, the cost of data storage has decreased tremendously since the dawn of computing, but especially since the 1980’s. What used to cost millions of dollars to manage and store gigabytes of data has plummeted to approximately two cents or less per gig. As a result, managing data is cheaper than ever. And, as a result, you have the data tsunami.
This has led to the mindset of ‘why not?’ We no longer have to make a decision about what to keep or what’s important, so why not just store it all? And now that we have all this data sitting there, is has created an expectation to eventually mine it. Because, I mean, why not?
On the other hand, over the last 10 years the idea that data is ambient has come to fruition thanks to things like the Quantified Self Movement that combines data collection and technology with everyday life. This data collection has really been made by possible by the IoT and this collision is evident in devices like fitness trackers.
The various types of data like this are only accelerating along with our interest in collecting and storing it — and it will only continue to rise.
But remember, machines aren’t people
The article calls out two barriers that AI and ML pose to being the solution to managing all this data. The first is transparency and the second is compliance.
When it comes to transparency, people have this innate desire to know what’s going on under the hood with Machine Learning. People, us included, have a lack of what we’d call an inability to be dispassionate about data. Yet to comprehend what the data is telling us, that’s exactly what we have to do. The more we try to make Machine Learning work more like us, the less useful it becomes.
The article also states that businesses should be mindful of compliance, specifically relating to customer privacy. Today, most organizations that are affected by this type of compliance are already fully aware of the boundaries. It’s all about balancing the cost and the benefit of using the data while adhering to the restrictions on how to use it. This can make the data tsunami seem quite treacherous at times.
Addressing the real challenge
Of course there are challenges when it comes to AI and Ml, but there are bigger ones than transparency and compliance. The biggest challenge is the public appearance of these technologies. We’re still getting past that idea that ‘the machines are coming to get us..” They aren’t. At least, not yet.
Therefore, we need to help people understand the concept of data and how to be literate in statistics and data analysis versus the fear of losing something to the machines. On the bright side, every time we have this innovation boom, history shows that there are more and better paying jobs on the other side of it.
Now is the time to learn and skill up in AI and ML so things like compliance and transparency become curiosities instead of fears.
Originally published at https://introspectdata.com on June 3, 2019.
