Why Big Data Is the Sexiest Tech Trend
Recently I’ve attended several lessons, conferences, job-fairs and hackathons that covered a wide range of tech topics: 5G networks, AI, IoT, hardware innovations, business & marketing and Big Data.
In particular, in a job-fair in Padua, I’ve been surprised by the number of students (like me) that are interested on working on Big Data. This is a sake shared by Computer and Management Engineers, Statistical students but also by Aerospace or Mechanical Engineers and Designers that, at a first glance, have nothing to do with the topic.
So, I’ve started thinking about why Big Data are so attractive, so “sexy” to undergraduates of engineering and economics. Calling them “sexy” is a defiance, but there’re some reasons why I can state so.
The obvious feature that makes Big Data so interesting is the fact that they are becoming more and more relevant in the business, driving companies’ strategies.
This happens because:
a. Business transactions, info shared through social media (e.g. Facebook, Twitter, …), data gathered up by sensors (e.g. Smart meters) and then exploited in machine-to-machine systems create a huge volume of information to be analysed.
b. This quantity of data streams through informative systems at high speed and require (almost always) a real-time analysis.
c. Being a huge amount of information, it will be easy to face a problem of variety of data formats. Some are stored in structured databases and they are usually numeric like financial transactions and stock tickers, others are unstructured such as documents, media file and e-mails.
d. The flow of these data, despite its velocity, can be highly inconsistent due, for example, to a periodic detection (e.g. daily, seasonal or event-triggered). This creates the opposition between computational peaks and “stand-by” periods.
e. A huge quantity of various data means also that exists a complexity problem. The provenience of information from different sources (see point a.) make their linking and matching harder.
So, the better a company can face these tasks, the better it can take the right decision at the end of the data analysis.
The margin between a company that can navigate in the world of Big Data and a competitor that is less powerful is considerable. According to Qlik Data Literacy Index (October 2018), companies that have a strong knowledge on data driven decision making have between 320 and 534 million $ in higher enterprise value. In this scenario the leading role has been taken by EU, with UK as best country of the region (still considering it member of the Union), followed by Germany, France, Italy and Spain.
Moreover, Big Data is a tech trend that pervades almost every business area: from banking to retail, covering also government, health-care and factories.
For example, Adidas exploits Big Data algorithms to optimise the emptying of their warehouse by setting dynamically discounts on their products analysing own sales and market demand. In this way, up to the 93% of products is sold (with discounts between 35 and 45%) increasing by 23% profit.
This is similar to the strategy implemented by Leroy Merlin Italia using an app (Abacus) for warehouse management that led to an increment by 200% of orders in 8 years.
As mentioned before, also governments are exploiting Big Data as in Austria, where the fight against tax evasion is carried on no more with sample checks, but analysing an enormous amount of data of each company outgoings.
But the sexiest thing of Big Data is that:
every experience can be enriched with new features thanks to their analysis
One interesting story comes from The Netherlands where SciSports (SAS and Google partner), a sport analytics company is re-inventing the way we watch and comment a football game. Their aim is to automate the collection and the analysis of a football game data: from the number of fouls, passes, dribbling completed, distance covered per player it’s possible to define the strengths and the weaknesses of a single player. This data, and their analysis are pure gold for football insiders to discover young talents and analyze what went wrong in a game for example. Napoli’s former-coach, Maurizio Sarri, used to monitor trainings with drones to later look what to improve or Wout Weghorst, one the best scorers of last Eredivisie (Dutch highest football league), was discovered in the minor championship thanks to the analysis of data collected from his previous games.
Data scientist roles have grown over 650% since 2012, but currently, 35,000 people in the US have data science skills, while hundreds of companies are hiring for those roles. (source: Forbes)
Big Data raises some challenging issues, but at the same time invades the market and provide more enterprise value. For this reason, they enliven the labor market: there’re several working positions related to big data. There’s need of experts on statistics as Data Scientists, Data Architects and Insight Analysts in order to identify complex business issues, design the infrastructure to storage unstructured data and apply ML and data mining know-how to extract information for CRM systems.
Campaign Analysts are the ones that exploits these results to categorise consumers. Then, engineers (typically Big Data Engineers) with knowledge on data collection, storage and analysis work on optimising database management and data flow. Developers are involved in designing web-based applications for client-side and in visualising in the best way analysis’ results (=creating dashboards) using tools like Tableau and Qlik. Other important and wanted professionals are Business Intelligence Developers: they design agglomerated (and, so, complex) data structures in order to obtain the most intelligible reports or dashboards. Previously their division was incorporated by the company’s Finance Division; nowadays they’ve been divided, and finance managers control and evaluate their actions thanks to the results given by Business Intelligence algorithms. In the end, comparing the Finance Division to the “recent” Business Intelligence/Data Management Division, there must be a “data manager”, better called Chief Data Officer: several studies claim that before the end of 2019 the 90% of companies will have one in their organisation chart.
Data Scientist has been named the best job in America for three years running, with a median base salary of $110,000 and 4,524 job openings. (Forbes)
In conclusion, Big Data is a tech-trend that is deeply changing the market: according to some previsions, we’re going to reach 30 billion devices connected to internet by 2020, 75 billion by 2025. This means an indeterminable number of TB of data to be analysed: new algorithms are going to be designed, new horizons are going to be discovered and new work opportunities will be given to future professionals and students: these challenges sound so exciting.
That’s why Big Data can be considered the “sexiest” tech trend.
