Velocity: The #1 V in Big Data
Big Data. It seems to be the all-knowing, all-seeing. It takes vast amounts of data and finds correlations that previously were obscured behind mirrors and smoke, it can flip through the pages and chapters of the present and read the future. How is this possible? With Volume and Variety. Volume and Variety are part of a collection of words known as The V’s of Big Data. The V’s of Big Data describes aspects that are key to big data’s functioning. However, the most important V to Big Data is Velocity.
Taking Volume and Variety as an example of the V’s of Big Data. A key aspect of Big Data is analyzing extremely large datasets, instead of settling for a sample of a dataset due to computational limitations. Thus, a large volume of data is key to Big Data. Similarly, pulling data from a large pool of data sources is key to amassing a large volume of data. Additionally, having such a large volume of data means there will be inevitable differences in the type of data collected, such as structured and nonstructured data. Thus, having variety and possessing the capability to navigate such variety in data is another key aspect of Big Data.
These two tenets of Big Data are essential to uphold. Without either of these, Big Data cannot do what it sets out to do, find hidden connections. However, it is not just these two tenets that shape what Big Data is. According to various sources, there are anywhere from 3 to more than 15 different V’s of Big Data that each play a pivotal role in shaping Big Data and driving it to where it is today, be it Veracity, uncertainty of data, or Variability, evolving behavior in a data source.
However, while Big Data in its current iteration is all-knowing (Variety) and all-seeing (Volume), it has not yet gained “divine” status as all-knowing, all-seeing, and all-doing. That is where Velocity comes in. Currently, Big Data is more of an afterthought of sorts. We experience events before we are enlightened on the hidden correlations within and can only look forward from the present. Take the example of Big Data matchplay analysis in tennis, for example. During the ATP Tour Finals tennis great Djokovic harnessed the power of Big Data to strategically review points in the match and improve performance after each match. These insights were not only backed up with concrete data in the form of video but demonstrated how big data can find unseen correlations. Coaches no longer had to comb through hours of footage to find key points as they could now be generated within 20 minutes of a match. But what has happened in the match has already happened, and there was a delay between events and correlation because current methods cannot process this data fast enough. Djokovic and the team can only improve in the next match. Big Data will undeniably raise their chances of success, but they might need to have been defeated in the last match, for example, for the correlation to be found between factors such as how Djokovic stands compared to how well he can receive.
Now let’s imagine the power of big data with the capability of analyzing high-velocity data as it comes in. One of the few sports in the world that sees athletes move at high velocity, more than 300km/h, is Formula 1, the pinnacle of motorsport. In Formula 1, every detail matters. More than 300 sensors are affixed on a Formula 1 car, and these sensors combined create more than 1.5 terabytes of data over the course of a race. Details are not only limited to the car. Teams take into account weather, track conditions, and countless other factors in optimizing performance over a race weekend and back at the factory. However, gaining the maximum advantage through these data points requires human strategists to carefully examine this data. While human strategists can make decisions at the moment, they are not foolproof. But what if strategists could be augmented by Big Data working at the moment? Big data is all-knowing, it is all-seeing. All it is missing is the all-doing.
So the capacity of Big Data to analyze high-velocity data is increased by integrating the newest advances in computer technology, quantum computing, and distributed computing to name a few, and Big Data goes from an afterthought to working side by side with humans, in the moment. Now after every point, Djokovic and the team can make adjustments in the moment. A Formula 1 team can now find hidden correlations and adjust their strategy based on something as minute as the angle of the sun on the track. Big Data might even surpass human coaches, who can only dream of capturing every minute detail down to how much dust is on the court. It is only with Big Data’s next evolution to real-time analytics that we will see the true power of Big Data, that it has evolved beyond us, and takes on the qualities of “thinking” and “acting.” In fact, that transformation has already happened. Take weather forecasting for example. We humans can step outside and observe that the weather is sunny, but are effectively blind to how the weather might change an hour from now. Big Data, on the other hand, seems to know the future. But it does no good if all the weather forecast said 12 hours later the next day that it made sense for it to rain at 3 p.m. the previous day because a myriad of factors contributed to rain. We expect the weather forecast to tell us what the weather will be now, and what the weather will be in the next hour. That is where real-time big data analytics comes in. When Big Data is able to handle the velocity of data, it then truly transforms into something that is all-seeing, all-knowing, and all-doing, that can foresee in the moment, not foresee after the moment.
Citations:
The 17 V’s of big data — IRJET. (n.d.). https://www.irjet.net/archives/V4/i9/IRJET-V4I957.pdf
Big Data is serving top tennis players a match-winning advantage. ZDNET. (n.d.). https://www.zdnet.com/article/big-data-is-serving-top-tennis-players-a-match-winning-advantage/
Shapiro, J. (2023, January 27). Data driven at 200 mph: How analytics transforms formula one racing. Forbes. https://www.forbes.com/sites/joelshapiro/2023/01/26/data-driven-at-200-mph-how-analytics-transforms-formula-one-racing/?sh=250961e739db
