On The Shoulders Of Big Data
The Feedback Canon — Installment #11a
In our last installment, we got Beyond Data via a recursive review of our progress to-date. As we push forward into our next exploration of the power and analysis of feedback, we will look to conquer a few more of the hurdles that distinguish it from data. In the second half of this article, that will mean having fun with a few games. But to start, we need to build more context.
Context is important to feedback, data as well, but for feedback it is critical. For context in this discussion, we early on described feedback in these terms:
Feedback builds and suppresses. It flows and connects. It is highly dimensional. It might be fair to call most data scalar while feedback is a vector, but that is really an over simplification.
Since we’ve already oversimplified, pardon me if I go to the opposite extreme. Feedback is to data what Einstein’s gravity was to Newton’s. Extreme? Maybe… but this does a better job of encapsulating the higher dimensional complexity of feedback. In its simplest form, feedback is data. But if you want to advance your data science to the next level, you need to consider a more nuanced and complicated view.
If I have seen further, it is by standing on the shoulders of giants. — NOT Isaac Newton (actually Bernard of Chartres)
It is contextually convenient to begin this next section of the article with an example of poor attribution. Data is notoriously miss attributed (as is history). This tends to come from a lack of context. Or more specifically, capturing data (or incomplete feedback) rather than feedback (implying that a feedback packet is not complete without context) decreases the value of the information.
Put differently, the data model is a simplified version of the feedback model. In many states, that is perfectly acceptable and certainly easier to deal with. But just as general relativity complicated our understanding of gravity, it also advanced our understanding of space-time and the universe in general.
How deeply this analogy holds, remains to be seen. General relativity was both revolutionary and flawed. It is also far more difficult to understand, at least the mathematics. I will share a great book at the end of this article for those looking to understand the conceptual framework. But importantly, it created a framework to understand a vast array of new and observable data.
Our feedback canon aims to build that framework for big data. Not all of it. But much of big data came from the recognition that most data is actually feedback — that feedback requires the capture of additional data (& meta-data) — and that all of this offers some big opportunities (and problems!). Most of the problems with big data, if I might offer, come from the fact that recognition was very shallow (and mostly unstructured).
At this point, our exploration requires some examples to build a more tangible framework and better illustrate what I am proposing. Something along the lines of our other thought experiments to help us conquer any lingering doubt. Better still, we should play another game! But for that, we will need another addition to this article.
Exploration & Conquest — Read it here.
For a great explanation of many of the conceptual aspects of general relativity consider:
