Information Engineering Beyond Data
A Follow-Up To A Popular Article And Perspective On The Intersection Of Systems Thinking, Deep Learning, and Feedback (Data Streams, IoT, etc)
This article is a follow-up to an article I published late last summer. Beyond Data was a synthesis of a ten part series on the challenges of feedback. It is a quick read and likely valuable for its context. I have given you my feedback, you are free to decide what you will.
The World Is Changing
It is growing. It is evolving. It is accelerating. And all of this leads to a feeling of overwhelming complexity. Just to be clear, I am only talking from the perspective of data. I know, the rest isn’t static either…
The amount of data produced each hour is staggering. This is even true if you only consider the number of articles informing you of that fact. Doubly so, if you include the technology built to “deal” with it. And triply so the sales literature hawking that technology. The world is awash in data.
Sorting Through All This Data Requires New Frameworks & Perspectives
I try to keep these articles short — so if you want my perspective and a framework for valuing data, read here:
If you want some perspective on how data becomes knowledge:
Feedback given, but let’s summarize. Not all data is equal. Not in type. Not in quality. Not in power. Data <> Information. In order to create information and knowledge from data — it needs to be connected and structured.
Data in the form of feedback is best distinguished by the same qualities we began with — growth, acceleration, and evolution. To further complicate things, it is transmitted and needs to be judged relative to both sender and receiver. It requires layers of perspective. I have often compared it to energy. Or perhaps you prefer — a force. Or The Force…
It Requires New Technologies & Processes
Again — not just one. This is a complicated problem. It needs to be approached with tool sets, not tools. With techniques, not a recipe.
It requires IQ and CPU. Enter Deep Learning. While often an afterthought in conversations about Machine Learning and AI, it is the technique most aligned to the new challenges we face. First, Deep Learning offers a layered framework that suits our task. Second, it has come to represent a strong body of knowledge and experimental results when dealing with complex and evolving systems.
Enter Engineering
Analytics is science. It is also engineering. Nothing is well learned without application. This emerging challenge is going to require a great deal of engineering excellence if we seek to really leverage it. The information age is awash in data, data that is flowing in streams and torrents, packets and other new things that arrive near daily.
Systems have to be created that are capable of real-time calibration and control. They will likely need to be distributed and highly efficient to fit into our ever distributed and miniaturized world. They will need to assess and provide feedback. You can think of it as an array of distributed neurons (neural networks anyone?). Only early attempts at that analogy have only gotten us so far.
Like engineering, we need to look to physics. Data science has far too long dismissed the building complexity of the data itself. It has had to — the solutions don’t fit otherwise. And far too many DS&A professionals believe that is what the data management team is for.
This article can’t get into all the aspects of physics that apply here, but data science is moving from Statics to Dynamics… and beyond. The new environment requires more conceptualization and application in areas like synthesis, stability, structure, equilibrium, and more. And so far, I have only touched on the machine layers of our deep learning model.
We humans need to recognize that specialization serves only in well-understood systems where efficiency and optimization have become priorities. This realm of true data science… nix that — information science requires holistic approaches rooted in true discipline. That may arise from interdisciplinary efforts but it must transcend them.
Let’s end with that. Data has transcended us. It happens. Human nature is to create things that stretch our own capacity to handle them — although we like to label them engineering (genetic, nuclear, chemical, etc).
You can hear the fear that data is now producing in our media headlines. Data is dangerous… or at least scary (and powerful). Our AI overlords are arriving any day… likely riding on the backs of robotic dogs. Society has a sense. Some of us have a plan.
Thanks for reading. Stay tuned for the next installment… coming soon.
That base article once again:
Don’t fear our coming AI overlords… feel sorry for them
