Big Data and Retail Industry (Part II)

In Part I of this blog post, we discussed what big data is and how Big Data is changing industries. We start looking at the retail industry as an example. In Part II, we will take a closer look at the effect of big data on the retail industry.

How Retailers take advantage of Big Data

Data explosion and rapid change in technology challenged retailers and made the already tough market more competitive. The following Porter’s Five Force Model outlines the level of competition in the retail industry. (Sarbah, 2014; Investopedia Staff, 2004).

The rise of e-commerce and smartphones, for example, enabled customers to compare products more easily, leading to increased competition among vendors. For instance, social media made consumers unforgiving. Primarily, unhappy customers, who previously could tell 10 people on average about their negative experience, can now post negative reviews or videos on social media, and the whole world will know about them. By examining the challenges of retailers in the last decade, one will find out that gathering data, processing data from various sources, such as the local ERP database and a Facebook fan page, and coming to a conclusion for a specific problem is the main challenge of today’s retailers (Bickle, 2013). Based on the definitions provided in the first section, this is exactly where Big Data fits.

Big Data analytics help retailers find out what the most popular products will be by predicting trends (Marr, 2015), manage inventory levels by predicting trends and controlling the flow of goods (Lewis, 2010; Marr, 2015), optimize pricing to compete with competitors, and provide a customized experience for each customer (Lewis, 2010; Marr, 2015; Baumgartner, 2012).

Predicting Trends

For retailers, finding the most in-demand products in each category helps them be prepared before customers come to their shop. From a competitive standpoint, the sooner a retailer finds a trend, the better it can capture the necessary resources. Big Data helps retailers in this matter, so they can predict trends in more intuitive ways. By using techniques such as “sentiment analysis”, business owners can leverage artificial intelligence methods to analyze data from various sources, such as a fan page from social media, and eventually, they are able to predict trends in real-time based on geographical locations (Lewis, 2010; Baumgartner, 2012).

Forecasting Demand

Predicting trends alone is not enough to forecast the demand for specific products because other factors such as demography, economy, and weather play important roles in this matter (Lewis, 2010). To find out the necessary inventory level and have a smooth flow of goods, for example, a retailer needs to have a clear picture of spending habits across their targeted market (Lewis, 2010; Marr, 2015). Such predictions, as stated above, require data input from various sources, from governmental data to social media data. Hopefully, Big Data analytics techniques, as mentioned in the first section, are specialized in gathering and analyzing data from various sources.

Optimizing Pricing

After forecasting demand, it is necessary to set the right price to maximize profit. Determining how customers react to different prices is not as easy as it looks. Retailers need to use mathematical analysis in real time to find the appropriate price for each selling channel. As expected, the required data comes from various sources, such as demand forecasts, inventory levels, and competitors’ activity (Lewis, 2010; Phillips, 2010). “Markdown optimization” is one of the book's examples of using data analytics in pricing. A traditional retailer usually reduces prices at the end of each season, when the demand is at the lowest possible point. But data analytics algorithms show that gradual price reduction, starting from the moment demand starts to fall, increases revenue by 90 percent (Lewis, 2010).

Market Micro-Segmentation

Offering customized selling experiments is one of the hot topics in the retail industry (Lewis, 2010; Marr, 2015; Baumgartner, 2012). Retailers use several structured and unstructured data sources, such as transactional databases and customer behavior data, as an input for Big Data analytics to achieve real-time micro-segmentation, or, in other words, deciding which customers are likely to buy which products (Lewis, 2010; Baumgartner, 2012). However, reaching these conclusions requires accurate data. Williams-Sonoma [1], for example, uses Big Data analytics to process data from its local database, including customer income, housing value, and the number of children in the household, to tailor direct advertising based on each customer. A study shows that this practice increases the response rate by 10 to 18 times (Baumgartner, 2012).

In this section, some of the most important applications of Big Data in the retail industry were mentioned. By using quality data, the veracity part of Big Data definition, and leveraging advanced algorithms such as machine learning to analyze those data, a knowledgeable manager can achieve their goals. According to the McKinsey Global Institute, using Big Data can boost operating margins by 8 to 25 percent “for those pioneers that fully embrace Big Data” (Phillips, 2010).

OAM Model of Retail Industry

Industry Competition

It seems that Big Data makes the retail industry less competitive and leads the market towards bigger fish. The only way to benefit from Big Data is to have higher-quality data and the processing power to analyze it. Based on Porter’s five force model demonstrated in Figure 3, Big Data capability moderates the threat of substitution and competitive rivalry, the only two forces with a strong force, due to the fact that fewer companies have the competency to use Big Data.

External Environment

Using Big Data can help an organization spot external factors faster. Big Data is mostly about gathering and processing a massive amount of data from external sources, such as social networks and external events. Having access to this information raises the possibility of anticipating a change before it actually happens. For example, regarding the external environment, Big Data tools intensively monitor customer interests and demographic changes so that companies will be aware of any significant changes. (Center for Media Justice, 2013; Baumgartner, 2012)

Big Data gathering could be endangered by government regulation. Currently, there is no specific law regarding Big Data, but more concerns are rising about the surveillance nature of Big Data, especially the parts related to customer behavior monitoring (Center for Media Justice, 2013).

Organization

Using Big Data in an organization affects several aspects of the organization, such as leadership, structure, and system (Galbraith, 2014; Berner, 2014). Interestingly, the effect is mostly caused by one attribute of Big Data: velocity.

A genuine Big Data system performs and reacts in real time, so when an organization wants to benefit from Big Data, as part of its leadership principle, the organization needs to speed up the decision-making process (Galbraith, 2014). This concept is also known as “internal clocks” or “clock speed” (Galbraith, 2014), which “[…] affect how long it takes [an] organization to move from [an] idea to definition, to action” (Jick, 2003). Consequently, an organization, in order to adopt Big Data, needs to increase its clock speed to eventually be able to operate in real-time (Galbraith, 2014).

Some researchers believe that as a result of Big Data, an organization may experience “power shifts” (Galbraith, 2014; Berner, 2014). “Before an organization can make real-time decisions, it must get data scientists and analytics experts embedded into the decision process.” (Galbraith, 2014). This change is necessary, as in an organization that has adopted Big Data, decisions will be made based on data rather than experience and intuition (Berner, 2014). These changes also affect the structure of an organization significantly.

In order to benefit from Big Data, an organization needs to make decisions in real time based on real-time data. This requirement leads organizations toward cross-functional teams [2] (Galbraith, 2014). “Companies can use cross-functional teams that are in constant contact..." to respond to real-time inputs.” (Galbraith, 2014). Furthermore, some experts suggest that an organization, to adopt Big Data, is required to “transform from command and control hierarchies to post-bureaucratic organizational structures[3]. (Berner, 2014), so, as a result, “employees at all levels can be empowered while simultaneously being controlled.” Figure 4 illustrates the change in the organizational structure.

Other aspects

In the case of Big Data, changes in other aspects of an organization, such as people and systems, mostly derive from alterations in leadership and structure. The real-time decision-making process requires an organization to hire and keep people with data science and analytics proficiency. Staff also need to be capable of performing in a team-based structure. The table below summarizes all the changes that the Big Data adaptation may cause for the organization.

[1] Williams-Sonoma, Inc. is an American publicly traded consumer retail company that sells kitchenwares and home furnishings. [San Francisco Chronicle]

[2] A cross-functional team is a group of people with different functional expertise working toward a common goal. It may include people from finance, marketing, operations, and human resources departments. [wikipedia.org/wiki/cross-functional_team]

[3] The source refers to the Charles Heckscher idea of post-bureaucratic organizational structures; for more information, please see “Heckscher C., Donnellon A., 1994, The Post-Bureaucratic Organization: New Perspectives on Organizational Change, Sage Publications.”

Originally published at http://aminrashidi.com.