Basics: The Business Intelligence Process
“All you need is the plan, the road map, and the courage to press on to your destination.” — Earl Nightingale
All businesses in the world are in a race to generate more revenue and for that they operate on data. Data, which is generated from many internal or external sources. In order to take full advantage of data-based decision making, it is mandatory to have a full view of the data at hand. This full understanding only helps a business to convert raw numbers and chunks of texts into meaningful insights. The solution, which enables businesses to achieve data driven decision making using Descriptive Analytics is called Business Intelligence (BI). I have an article, which gives a detailed brief of Business Intelligence.
Just to recap, let us have a formal look at the definition of BI again. BI involves a process, by which a business can collect and stage data to analyze or visualize it so that the business can convert this raw data into actionable insights. These insights can be drawn to improve future decisions or measure pre-defined KPIs.
In this article, we will take a dive into the BI process, i.e., we will explore what are the various stages across the BI process, what are the requirements to setup a BI environment along with required tools and technologies and finally, we will explore how Descriptive Analytics, which is enabled by BI, differs from Predictive and Prescriptive Analytics.
Stages across the Business Intelligence process
The entire BI process can be broadly divided across 4 stages:
1. Identifying various Data sources
2. Collecting data from various sources
3. Data Staging and Cleaning
4. Data Visualizing/Dashboarding/Reporting
In order to get briefs of these various stages, let us understand what it takes to setup an environment to implement the BI process for a business.
Business Intelligence implementation phase
The initial stages of planning to setup a BI process mostly involves the business identifying various Key Performance Indicators (KPIs). Once, these KPIs are identified along with how the business plans to measure the changes in these KPIs, it is required to identify what data pointers are relevant for the said measurements. These KPIs can involve Measuring Sales performance in multiple regions, measuring operational overhead etc. With data and KPIs at hand, the business can start designing an architecture involving what data sources to use and what tools to use for Extracting Loading and Transforming the data. We will look at these stages now along with what tools can be used at these stages.
Identifying Data Sources
In order to generate any report using any technique and make data driven decisions, the very first architectural requirement is hosting the data on one or multiple data sources. These data sources can include raw flat files, SQL database, Web hosting, SharePoint lists and what not. Listing all possible data sources is not possible since there are numerous of these. The decision of which data source should be used is solely a question of requirement, performance and integration, i.e., the selected data source should be highly performant in terms of Data supply and integration of the data source with the existing architecture should be easily possible.
Extraction Transformation and Loading (ETL)
This might be one of the most crucial and critical step of the entire BI process and covers both Data Collection from selected sources, cleaning this collected data and Staging it for future use. Let us understand this ETL process with a simple example, which will also demonstrate why ETL is required in the first place. Suppose, once the Data and KPIs are identified, in terms of data sources a business decides to leave the decision to independent divisions (say the head office of a Global Retail store asks individual stores to host data on a data source they can easily integrate with their current system). In this case, various divisions will select a different data source for hosting. ETL process involves capturing data from these distinct data sources and bring it together (load) into common Data Warehouse by apply some Transformations for data harmony and cleaning. Hence the name Extraction (Extracting and staging data from Data sources), Transformation (Transforming staged to have clean data) and Loading (Loading of data in a common data warehouse). Thus, the ETL process covers 2 major stages of the BI process, i.e., “Collecting data from various sources” and “Data Staging and Cleaning”. Tools such as Alteryx, Talend, Azure Data Factory are known industry wide for their ETL abilities.
Reporting and Dashboarding
Now comes the most interesting part of the entire BI process (at least it is for me). We have our data cleaned and loaded on our Data Warehouse, now the only thing required is to fetch this data in a BI Viz tool such as Power BI, Tableau etc. and start experimenting to obtain desired results. At this stage, the initial knowledge of identified KPIs and measurements to be performed on these KPIs is highly important. The user may need to create modified fields and additional data pointers or even add conditional columns in existing data to match the desired results. Playing with data for KPI reporting also gives an individual the experience that, which visualizations can best represent a form of measurement. Basically, these visuals in a report or dashboard finally helps the business to track the KPI measurements, i.e. enables the business to see KPI measurements in past and KPI measurements in present to plan for future. This entire enablement provided by BI is coined as Descriptive Analytics. So, Descriptive Analytics helps a business to study internal improvements or market performance depending upon the requirements.
Predictive and Prescriptive Analytics
Predictive Analytics deals with the next stage of Data based decision making process. Once the KPI measurements are in hand with past and present performance visible, Predictive Analytics enables businesses to forecast and predict future trend of KPIs basis the data and measurements at hand. We can safely say that both BI and Predictive Analytics use almost same techniques of data processing and in some manner, Predictive Analytics can be the Next Stage of after BI.
Lastly, Prescriptive Analytics, aims at suggesting rectification measures for a business problem. For example, a company whose revenue in a specific geography is falling may take advantage of Prescriptive Analytics or Prescriptive Consulting to identify the root cause of decline in revenue. The process, again, would take advantage of similar data techniques. However, Prescriptive Analytics using software-based tools hasn’t yet reached high levels of reliability.
As a reader, if you have any suggestions, feedback or want to read about something specific, feel free to comment below or DM me on LinkedIn.
