Big Data is not self-reliant and here’s why

Hint- It’s getting emotional.

Everyone is familiar with Big Data, extracting, analysing, and dealing with large datasets. To be honest I don’t know anyone who isn’t familiar with the “Sexiest Job of the 21st Century”. But with all the glamour around the Big Data profession, people tend to neglect its counterpart, Small Data.

Small Data has been in the shadows of Big Data for a long time now, and it is now time for it to make its mark in today’s market.

What is Small Data?

Small Data is data. Data, small enough that the human brain can easily comprehend. Data, small enough to not require dedicated algorithms to analyse. It may come as a surprise, but Small Data is as important as Big Data if only not more important.

What is the fuss behind Small Data?

The Small Data concept is much more accessible than the Big Data concept. There is no need to collect gigabytes of data, clean it up, and write hundreds of lines of code. All you require is great observational skills.

Small data revolves around insights one can gain through the day to day life of consumers.

What sets Small Data apart from Big Data?

1) Big Data doesn't have that spark.
Big Data is born out of databases, and these databases are too narrow to create insights. It fails to spark perception.

Take the example of e-commerce websites. They are only limited to the data they get from their websites, and that too only includes their customers' purchase from the site. The company does not know what their customers purchase from shops, which in turn keep this data very guarded. But on the other hand, small data requires an intricate insight into the lives of the customers and focuses on their real needs.

2) Big Data is based on analytics.
Big Data neglects the emotional factor of customers. While increasing the battery life of a laptop will increase the respect of a particular brand, which big data can easily help us identify. But it'll fail to tell us how to increase the emotional factor of a brand, how to make the brand more lovable. Small Data makes up for this incompetence of Big Data. Small data helps in the design aspect of the works. It tells you the best way to get your brand more love. It helps in generating emotional excitement in the brand.

By the above two points, I in no way intend to critique Big Data. Summing it up, Big Data has problems and small data is essential to overcome these problems.

The Small Data Methodology

Any new project in small data starts with Subtext Research. Subtext Research is the equivalent of Data Collection in Big Data. It's the most time-consuming process.

Subtext Research, in turn, is a framework of seven steps.

• Collecting data- this step begins with establishing navigation points, getting perspective from cultural observers and local observers. This helps in creating a hypothesis, which gives a plan of action.
• Finding clues- This step is exactly how it sounds. Just as a detective, one must ask questions and look for subtle hints that will help us in our "investigation". The clue can be physical or emotional.
• Finding a connection- Now that we have Small Data collected in front of us, we must realize if there are any similarities between the clues? Do they tilt us in a certain direction? Do they begin to validate our initial hypothesis?
• Causation- It is now time to put ourselves in the customer’s feet and see if this is what they want and how they feel about it. We must find out what emotions are evoked.
• Correlation- Once we have generated emotion in the customer, it is time to figure out what generated the emotion.
• Compensation- Having found a shift in emotion, it is time to find the desire of the customer. What does the customer seek? How do we fulfil this desire? It is also necessary that the desire should be in line with the culture of the customer.
• Concept- This is where the idea, the method to fulfil the desire will be found. Ideas don’t germinate under pressure. Thinking about all the small data collected in peace will give the most effective solution.

Although the data explored in small data is small it has large implications on human behaviour.

Case Study-

To put emphasis on the importance of Small Data, we’ll investigate Martin Lindstrom’s work. Martin is a pioneer in Small Data.

I would pick up his case of how he managed to save Lego, the world-renowned toy company from potential bankruptcy. Lego's sales were dwindling in the 90s and early 2000s. Technical toys were taking over the market. Swayed by their Big Data analysis of strategies to get relevant and push sales back up, they realized that kids wanted instant gratification, and therefore they started making Lego sets quite easy. Despite this, sales didn't increase.

This is when they got Martin up for the case. Martin used Small Data and recognized that kids are also incredibly competitive and liked the idea of a challenge. Being the only person to have solved a particular puzzle gave them a feeling of pride. This emotional feeling could never have been picked up by Big Data. Subsequently, Martin suggested Lego make their toys much more complex so that completing them have the kids a sense of satisfaction. The Lego board implemented this and soon after Lego sales were rising again.

I’ll also recommend Martin’s book on Small Data to get more in-depth knowledge in the wondrous world of Small Data.

Conclusion

In today’s day and age people not only want an efficient and cheap product but they want a product they can relate to.
Roomba is a particularly good example of this. According to the Boston Globe, the Roomba has become a part of the families that purchase it going as far as to even name it.

The importance of the emotional value of a brand is increasing day by day. And this is where Small Data comes out of Big Data’s shadow and shines.
For the larger part of the time since Data has become an integral part of our lives, we’ve neglected the one aspect of data.

And I think it is finally time for it to be given the importance it has in the success of any company, for companies to recognise that Big Data is not the solution to everything and that to truly succeed both Small Data and Big Data should be implemented.

At the end of the day, Big Data alone wouldn't increase business. We need rich, deep data. No matter what form.