Making sense of data
The dawn of a new age
We are standing on the edge of a new age, the age of big data. The world is full of data. It’s pretty obvious that gathering all the data available around us is impossible at the time. Even the data linked to a single human being is enormous. Making sense of data we gather is an even greater feat.
Data mining. Yes, “mining” and quite frankly it’s more valuable than gold, black gold etc… Companies have gone from nothing to multibillion dollar multinational gigantic companies thanks to the data they gather and use. Google, Facebook, Twitter… It’s even better than mining for natural resources which are scarce and finite. We have lots and lots and lots of data, it’s virtually infinite relative to what we can handle. The challenge is to make sense of it. Creating value out of nowhere, or so it would seem.
Revolutions in human history
We are witnessing more and more revolutions in our history. We are exponentially evolving towards a future that we only glimpsed through science fiction and imagination. We outsource tasks that we used to do on our own to machines. Technological breakthroughs always come with a way to extend the human capabilities making the once impossible or improbable possible.
The steam engine allowed us to make machines do the heavy lifting for us humans. Things that previously required an enormous amount of time and effort, now, became possible with little to no effort and at record times. That breakthrough marked a transition in human history. A transition to an industrial age that allowed humanity to take it’s production to a whole new level.
Computers also marked our history by allowing humans to automate a lot of their logic thus enabling them to do things that were simply impossible with the human resources that were available.
Computers coupled with the internet marked an even greater turning point in technological revolutions. Collaboration on a massive scale is now possible. The amount of humans leaving there fingerprints on the web is gigantic — Estimated at 2.769 billion people in 2013 at 39% penetration of the world’s total population and it’s evolving quite rapidly.
The internet gave birth to many companies that revolutionized our ways of doing things thanks to the software they provide. Software that helps us increase productivity, create even more and better software, entertain our selfs, keep in touch with friends and social circles… It touched our lives in more ways than we give it credit for.
These revolutions are tightly coupled with each other. Each revolution enabled the next one. It’s only fitting for the internet to make the forthcoming revolution possible.
The forthcoming revolution
Internet companies gather huge informations on us. We share our once privately kept data willingly. It’s becoming second nature for us humans to publicly share details of our lives disregarding our own privacy. Bottom line, the internet made a new type of resource available: personal data. Data that is currently being used in questionable manners to enforce national security in the USA and certainly in a lot of other countries in other more intrusive ways.
Data is a lot more than just the personal data type, but, only recently did we have the ability to take advantage of this type. For better or worse, it presents great opportunities for who ever figures a way to exploit them.
I am quite convinced that the forthcoming revolution is about engineering automated systems that make sense of data as we humans do or at least try to simulate the understanding that humans have for data. Just imagine the power that comes with such systems. “Predicting” future events, generating profiles on people, stock market predictions, generating insights on any set of data… The possibilities are endless and can apply to all sorts of domains.
The Net solved a part of the puzzle by making huge amounts of data widely available, the next part is up to advances in the field of artificial intelligence. We are already witnessing the first steps. The use of machine learning to exploit data is one example. But we are still on the edge of this new age. We are still dictating a certain logic for machines to execute but with a twist on how it adapts by “learning” from previously collected data.
Automating the process of understanding and generating meaning from data will push us to next level. Just as the process of autonomous logic execution by computers did in countless domains.
One pattern to rule them all
The challenge is finding a universal pattern in mining data. Algorithms that adapt to any kind of data. It would be artificial intelligence at its best. This would be a step forward towards replicating the human intelligence through machines. The moment we achieve such a thing, if ever, it will mark the turning point for humanity.
A transition to automatically created, viable and valid logic that machines generate and execute on their own. It’s the definition of a smart being. Maybe even a living one. Who knows what awaits us. Maybe machines will take over us. Maybe we self-destruct before reaching such a point. These are exciting times to live as a human being. I guess we’ll just have to wait and see.
