The Closed World Perspective and its influence on Data Science
Have you even been puzzled by the phrase “outside the box” , and wondered what it means to think outside the box.
How do we even think outside the box, when you don’t know where the box is?
Over the next 5 mins, lets demystify the box, and see what is that one thing most successful data scientists have in common!!
“outside the box” is used to encourage thinking beyond the predictable or the most obvious. However, in most cases people tend to think of non-obvious, and come up with something that is not useful, or implementable, given the limitations around the problem.
This is where, “inside the closed world” perspective comes handy. This encourages people to think under the constraints surrounding the problem. In most cases, people carry pre-conceived notions, also called “Fixedness”.
We have been doing time and again the same thing, without challenging ourselves, this is fixedness!!
Lets take an example:
Take a look, and guess what it could be?
If you are from the 90s you might be thinking it is a Television.
Lets take another one.
You might be thinking in the lines of a website , microwave, etc..
Little did we realize its just a 180 degree rotation of the first image.
Since we are trained to identify that the television is the one with the buttons on the bottom. And if it has the button on the top it’s not a television. This is what fixedness is!!
For a television, the reason that buttons are at the bottom is because when televisions were originally developed, the technology used was a picture tube. This Picture tube would get very hot upon usage. It is very likely that the heat could rise to the buttons, and lead to quality issues. To overcome these constraints, the buttons have been placed at the bottom.
In the 20th century, the technology advanced and the picture tube no longed existed. However, control buttons are still at the bottom. Why?
Because we really never thought about it until now to move the buttons to the top. We never thought about the advantages of the buttons on the top. This is Fixedness.
There are multiple experiments proving how fixedness limits us from thinking beyond the obvious. Another classic example is the Candle experiment by Karl Duncker, the cooling unit stayed at the top of refrigerator, and the list goes on.
Fixedness is our first reaction.
Jugaad — The Indian version of creativity.
Jugaad as quoted by Prof. Jaideep Prabhu, author of Jugaad Innovation , is a frugal, flexible, and inclusive approach to problem solving and innovation.
A deeper look at what it means, makes it evident that, we are trying to solve a problem, within the constraints.
Constraints can be in any form, financial, time, resource, etc., and are usually found around the problem. The Tata Nano, the 500$ ECG Machine by GE, are all innovations which have come out, within the constraints. Many companies in the West, have used this approach to change the way they build and market products.
That brings me back to “Inside the closed world” perspective. Where do you think constraints are available? Within the problem itself. And where do pre-conceived notions evolve? From within ourselves.
So know , its evident that inside the box there are constraints, there is fixedness!
Innovation is all about breaking the fixedness, and discovering the constraints!
Shifting the focus towards data science , what are the standard steps we follow!!
Similar to the data science Venn diagram, this has also been evolving over time, and there have been various versions created. I don’t want to get into details of this, but trust me if you google for “data science life cycle” you get thousands of versions.
Source : https://www.saviantconsulting.com/blog/cio-enterprise-machine-learning-implementation.aspx
However, most data scientists would agree with me that this is not a sequential process!
It’s more of an iterative process, where we look back and forth across each step. And, there is one thing that is common in all these iterations. Yes, you guess it right. DATA! Along with that comes the Business Problem and Domain understanding.
Pivoting the discussion a little bit towards the “box” we started off, we learnt that constraints lead to innovative solutions, and that constraints are set by the problem itself.
Consider you are building a Classification model, and find that the model has high accuracy, but low recall. What your your next steps be?
- Look for the False negatives, and analyze which features contributed?
- Talk to the domain experts, and understand what is more important for them?
Isn’t this the closed world perspective we spoke of?
With that in mind, let’s shuffle the data science life cycle a little!
Aren't we tuned to implicitly look inside the closed world?
- Don't we look at data to find answers for a bad model, before we look for another algorithm?
- Wouldn't we look at the data to decide what algorithm might work?
- Aren't we trying to build a solution, suiting the business constraints?
- Aren't we focused on deriving data insights aligned with the use case?
This provides enough evidence that successful data scientists are tuned to thinking inside the box.
On the contrary, there are few data science problems which require you to focus only on building algorithms. In such cases, the closed world is different.
The closed world is defined by the problem itself.
The closed for a problem could be perceived differently by people, and this varies based on experience, domain knowledge, etc.)
Example : The closed world for a Kaggle problem, is completely different from that of a business problem.
Thinking beyond the obvious, isn't thinking outside the box!
Looking forward to hearing your perspective on the “Closed World”. Feel free to share in the comments section below!
Stay tuned on this publication, for more on the Closed World Perspective.
