Why Intuition Matters?
Intuition is about understanding the why and how behind things, not what and when.
Intuition is the most talked about, but less implemented. In the mechanical world, intuition is often confused with extensive practice, but developing intuition requires the right perspective during practice.
Have you ever thought, how an F1 racer knows when to use a brake, and how intense that should be? Or, how do you know how much salt to add while preparing a dish?
Is that attributed to Practice or intuition?
When was the last time you have understood something intuitively? Or, you realized you understood the intuition behind something?
The dictionary defines Intuition as the ability to understand or know something without needing to think about it or use reason to discover it, or a feeling that shows this ability.
After all, we aren’t superhumans and need to learn to know things better. We try to shorten the gap between knowledge and understanding every day. Given the short attention span, it is difficult to remember everything we read or come across.
Intuitive understanding can help everything snap into place. Learning becomes difficult when we emphasize definitions over understanding. The modern definition is the most advanced step of thought, not necessarily the starting point.
Intuition matters in everything, and it matters the most!
Consider the theoretical definition of a neural network:
A neural network is a network or circuit of neurons, or in a modern sense, an artificial neural network, composed of artificial neurons or nodes. Thus a neural network is either a biological neural network, made up of real biological neurons, or an artificial neural network, for solving artificial intelligence (AI) problems. The connections of the biological neuron are modeled as weights. A positive weight reflects an excitatory connection, while negative values mean inhibitory connections. All inputs are modified by a weight and summed. This activity is referred to as a linear combination. Finally, an activation function controls the amplitude of the output. For example, an acceptable range of output is usually between 0 and 1 (or -1 to +1).
Isn’t that confusing to understand, and remember? Try the one below.
Neural networks help in identifying hidden patterns/features in the data, which is analogous to the way a human brain learns.
A human brain, tries to detect & associate patterns in their environment [Feedforward] all the time, and learns based on corrections[Back propagation]. If the human brain sees a “Kindle” for the first time ,the brain classifies it as a “Smart Phone” (based on its previous examples/patterns like a screen, color, etc). But if the brain is told, that’s not a Phone, it’s a Kindle ,it again learns patterns of a kindle, to be able to classify correctly the next time.
A neural network learns(in the context of neural network, learning implies assigning weights) each attribute through hidden layers, which breaks down the object into different attributes, in the same way the brain tries to associate patterns based on multiple senses. It starts with a random initialization of weights, and then learns the weights of the hidden layers through Feed forward and back propagation.
The weights are summed up to arrive at a final classification, which is on the range of 0 to 1 (or -1 to +1)
This is how we can intuitively understand what a neural network is, and can remember it for a longer time.
Now that we know why Intuition matters, let’s touch upon how and why intuitive understanding is closely related to the field of Data Science.
Data Science, the so-called sexiest skill in the 21st Century has a lot of doing with Intuitive thinking, given its multidisciplinary nature. Most data scientists are often confused, whether or not it is required to understand the Math behind the algorithms. If you are one of them, this publication is for you!!
Data science is a combination of multiple disciplines, and the scope of data science has been widening. The evolving data science Venn diagram from 3-set Venn diagram, to an n-circle Venn diagram, is a proof of its multidisciplinary nature. Take a look at this KDNuggets article on the multiple data science Venn diagrams out there.
However, the focus has always been on data-driven decision making, which is often being confused with data science. Data backed decision making as I understand is all about trying to understand the historic behavior and be able to make better business decisions. This is about identifying patterns in data to help answer questions like,
- Who are my next top customers?
- How are my sales going to be, in the next quarter?
- Which area should I put my delivery agents, to be able to serve customers better?
Most of these questions can be solved by basic intuition and the right perspective of looking at the data and might not require advanced modelling techniques. This is where Intuition is important.
Let’s consider an analogy to get an intuitive understanding.
Lets say, I am planning a Bike trip to Leh, the Mecca for Bikers(being a biker myself, I chose this example) . A Bike consists of complex components like engine, fork, tires, etc. and each of them is a complex structure. In this scenario, to be able to ride a bike to Leh, do I need to know all the mechanics of it? Or is it sufficient if I know how they work, what I should do to make it move and what should I do, in case of a breakdown?
You don’t need to be a mechanic to ride a Royal Enfield!
Applying the same analogy on machine learning algorithms, as a data scientist isn’t it sufficient if we know when and how to use a particular model, and not the Math behind it, or how is the algorithm derived?
A data scientist can apply these ML models, to solve business problems if he/she understands which models to be used, and how to tweak the parameters of the model, and not necessarily the entire math behind.
How feasible is it to understand the math behind all ML algorithms, which is a combination of Probability, Statistics, etc.
Each machine learning model implementation in itself, is an individual research subject.
This is where intuition plays a significant role, if we intuitively understand the concepts of Machine learning and the components that form it (stats, probability, etc), we will be in a position to apply the right model, with the right complexity, without jumping to advanced models (not knowing weather it is required or not).
Stay tuned on this publication, to understand more about intuition, and how it can be applied in solving real world business problems.
