What is Big Data Analytics and Why it is so Important?

6 min readMay 23, 2018

1. What is Big Data Analytics?

Data is information in raw format. With increasing data size, it has become need for inspecting, cleaning, transforming, and modeling data with the goal of finding useful information, making conclusions, and supporting decision making. This process is known as Big Data data analysis.

Data mining is a particular data analysis technique where modeling and knowledge discovery for predictive rather than purely descriptive purposes is focused. Business intelligence covers data analysis that relies heavily on aggregation, focusing on business information. In statistical applications, some people divide business analytics into descriptive statistics, exploratory data analysis (EDA), and confirmatory data analysis (CDA). EDA focuses on discovering new features in the data and CDA focuses on confirming or falsifying existing hypotheses. Predictive analytics does forecasting or classification by focusing on statistical or structural models while in text analytics, statistical, linguistic and structural techniques are applied to extract and classify information from textual sources, a species of unstructured data. All are varieties of data analysis.

The Big Data wave has changed ways in which industries function. With Big Data has emerged the requirement to implement advanced analytics to it. Now experts can make more accurate and profitable decisions.

In this session of Big Data Analytics tutorial for beginners, we are going to see characteristics and need of data analysis.

2. Analysis versus Reporting

An analysis is an interactive process of a person tackling a problem, finding the data required to get an answer, analyzing that data, and interpreting the results in order to provide a recommendation for action.

A reporting environment or a business intelligence (BI) environment involves calling and execution of reports. The outputs are then printed in the desired form. Reporting refers to the process of organizing and summarizing data in an easily readable format to communicate important information. Reports help organizations in monitoring different areas of a performance and improving customer satisfaction. In other words, you can consider reporting as the process of converting raw data into useful information, while analysis transforms information into insights.

Let us understand difference between data analysis and data reporting in this Big Data Analytics Tutorial:

Reporting provides data. A report will show the user what had happened in the past, to avoid inferences and help to get a feel of the data while analysis provides answers for any question or issue.An analysis process takes any steps needed to get the answers to those questions.
Reporting just provides the data that is asked for while analysis provides the information or the answer that is actually needed.
Reporting is done in standardized manner while analysis can be customized. There are fixed standard formats for reporting while analysis is done as per the requirement and it is customizable as needed.
Reporting can be done using a tool and it generally does not involve any person while in analysis, person is required who is doing analysis and who will lead the process. He guides the complete analysis process.
Reporting is inflexible while analysis is flexible. Reporting provides no or limited context about what’s happening in the data and hence is inflexible while analysis emphasizes data points that are significant, unique, or special, and it explains why they are important to the business.

Any doubt yet in the Big Data Analytics tutorial for beginners? Please Comment.

3. Data Analytics Process

Now in Big Data Analytics Tutorial we are going to see the analytic process or how analyzing data can be done?

Big Data Analytics Tutorial for beginners — Process

a. Business Understanding

The very first step consists of business understanding. Whenever any requirement occurs, firstly we need to determine business objective, assess the situation, determine data mining goals and then produce the project plan as per the requirement. Business objectives are defined in this phase.

b. Data Exploration

Second step consists of Data understanding. For further process, we need to gather initial data, describe and explore the data and verify data quality to ensure it contains the data we require. Data collected from the various sources is described in terms of its application and need for the project in this phase. This is also known as data exploration. This is necessary to verify the quality of data collected.

c. Data Preparation

Next come Data preparation. From the data collected in last step, we need to select data as per the need, clean it, construct it to get useful information and then integrate it all. Finally we need to format the data to get appropriate data. Data is selected, cleaned, and integrated in the format finalized for the analysis in this phase.

d. Data Modeling

Once data is gathered, we need to do data modeling. For this, we need to select modeling technique, generate test design, build model and assess the model built. Data model is build to analyze relationships between various selected objects in the data, test cases are built for assessing the model and model is tested and implemented on the data in this phase.

e. Data Evaluation

Next come data evaluation where we evaluate the results generated in last step, review the scope of error and determine next steps that need to be performed. Results of the test cases are evaluated and reviewed for the scope of error in this phase.

f. Deployment

Final step in analytic process is deployment. Here we need to plan the deployment and monitoring and maintenance, we need to produce final report and review the project. Results of the analysis are deployed in this phase. This is also known as reviewing of the project.

The complete above process is known as business analytics process.

4. Introduction to Data Mining

Data mining, also called as data or knowledge discovery, means analyzing data from different perspectives and summarizing it into useful information — information that can be used to take important decisions. And so we are discussing it in this Big Data Analytics tutorial. It is the technique of exploring, analyzing, and detecting patterns in large amounts of data. Goal of data mining is either data classification or data prediction. In classification, data is sorted into groups while in prediction, value of a continuous variable is predicted.

In today’s world, data mining is been used in several sectors like Retail, sales analytics, Financial, Communication, Marketing Organizations etc. For example, a marketer may want to find who did and did not respond to a promotion. In prediction, the idea is to predict the value of a continuous (ie non-discrete) variable; for example, a marketer may be interested in finding who will respond to a promotion.

Some examples of Data Mining are:

a. Classification of trees

These are Tree-shaped structures that represent sets of decisions.

b. Logistic regression

It predicts the probability of an outcome that can only have two values.

c. Neural networks

These are non-linear predictive models that resemble biological neural networks in structure and learn through training.

d. Clustering techniques like the K-nearest neighbors

This is the technique that classifies each record in a dataset based on a combination of the classes of the k record(s) most similar to it in a historical dataset (where k 1). Sometimes it is called the k-nearest neighbor technique.

e. Anomaly detection

It is the identification of items, events or observations which do not conform to an expected pattern or other items in a dataset.

After this Big Data Analytics tutorial, you can read our detailed tutorial on Data Mining.

5. Characteristics of Big Data Analysis

We have already seen characteristics of Big Data like volume, velocity and variety. Let us now see in this Big Data Analytics Tutorial, characteristics of Big Data Analytics which make it different from traditional kind of analysis.