What is big data?

Let’s consider a scenario where we are supposed to store some data gained from students like email id and name, for this purpose we will create a table in the database with these fields and store it there. Now consider application like Facebook where we have unstructured data like a post itself can contain text, image, video files, etc. This unstructured data is not just hard to handle due to being unstructured but at the same time is huge in size. This data which is mostly unstructured and huge in size is called “Big data”.

Characteristics of Big data:

Now as we know what big data is, let’s just go deep and try to further understand it by looking into its various traits

1. Size- As the name suggest big data is huge in volume. Usually it’s in orders of petabyte(1024*terabyte). For instance Facebook usually has data intake of around 4 petabytes.

2. Variety- As we know traditionally the date is stored in structured from creating a table with fields of certain data types. In case of big data, the data can be structured or unstructured which make it very diverse and huge to be sorted and then handled. Thus, normal traditional methods don’t work while dealing with Big data.

3. Speed of data generation- Consider current day social media which has a huge number of users posting content every day, this data is basically Big data and as can be predicted this data increases exponentially with time as the database is accessed by more and more users.

4. Tools- As we know by now that big data is complex and huge to handle, thus traditional database management tools like sql server, Oracle, etc. can’t be used to manage this data. It is because these tools have computational boundaries in their working. For instance, when you search something on Instagram the computation is fast and diverse enough to give you results to it instantaneously in various fields like accounts, audio, tags etc. this wouldn’t be achieved if Instagram would have used traditional methods for managing its data. Thus for big data we have separate database managing tools like hadoop and sparts which are included in apache.

5. Storage- Traditional data (small data) is mostly stored in localized single server whereas while dealing with big data we need many servers at various locations for efficient management of data flow. For instance, google has millions of servers spread all across the globe.

6. Veracity- The truthfulness or reliability of the data, which refers to the data quality and the data value. Big data must not only be large in size, but also must be reliable in order to achieve value in the analysis of it. The data quality of captured data can vary greatly, affecting an accurate analysis.

7. Value- The worth in information that can be achieved by the processing and analysis of large datasets. Value also can be measured by an assessment of the other qualities of big data. Value may also represent the profitability of information that is retrieved from the analysis of big data.

Application of big data:

As big data has more users linked to it analysis and usage of big data thus lead to its various applications -

1. E-learning- Big data as mentioned is linked to multiple users can be used to analyse students behaviour to various topics and different teaching method and thus can be used to suggest best method as per the performance of particular student. This method of learning is called adaptive learning, this helps both student and teachers to find the best way of learning.

2. Business and marketing- Brands are always in search of their consumers of interest, as big data can be used to gain millions of user feedback, analysing it can help brands get their targeted audience and thus help them market their products and services more efficiently.

3. Dynamic pricing-As big data holds multiple records of user interests it also holds records related to user payments; thus, a company can find the price which earns the company most profit by analysing big data.

4. Healthcare-Its said usually that the more the number of patients dealt with the better the doctor is, imagine having information regarding millions of patients across the globe. This information can be used by doctors to treat their patients more efficiently.

Limitations of Big Data:

Big data has many advantages as discussed above, now let’s look into some of its limitations-

1. Big data is a relatively new concept thus it contains issues like low digital literacy and lack of proper technology to manage data.

2. As already mentioned, big data is unstructured and huge thus is prone to data manipulations and data leaking.

3. Big data can cause social stratification that causes social disparity and problems.

4. Big data makes no sense for short term usage. Big data is meant for long term usage.

Conclusion:

Big data, if handled and managed cautiously with suitable authorities can be used for various purposes. The problem related to big data can be resolved with proper education and technological advancement and thus Big data is our future.

Author,

Gaurav Bomble, BOKIL SIDDHANT, Chanchal Budhwani, Rishita Bura, Shruti Borude