Harnessing Big Data is Life or Death for Your Business

By Alon Gamzu

Successful companies in the 21st century revolve around superior customer service. But it’s not as human-oriented as it sounds. Consumer behavior is shifting to online shopping because of technological innovations. Entrepreneurial Insights estimates the expanse of the digital universe will be over 40 trillion gigabytes by 2020. Many companies gather personal and web data, but they do not know how to leverage the information in their disparate formats. Benchmark spreadsheets are business intelligence tools, and involve no underlying algorithm to process unstructured data or reveal insights into customer satisfaction.

The age of big data involves automatic digitizing and tagging of all content into data silos, and data mining with big data algorithms to discover new knowledge from this content. If customer service is not integrated into the core of a company’s data structure, the business will fall behind companies that harness big data. Forget manual data entry or optical character recognition, or any number of expensive and time-consuming IT transformation projects. Cloud technology and algorithms not only allow for a quick solution to data access in new and old formats across different media, but they facilitate a company’s entire organization to tap into big data.

The largest roadblock companies face is the inability to harness productivity across organizational departments that use different core systems. Cloud computing allows all departments — from technology and development to sales and marketing — as well as virtual workspaces, to leverage big data analytics. It does not matter what your company does or how un-technological its products or services may be. Grocery stores and retail shops use inventory control technology in their labeling and at the cash registers. If a company does not adopt a data-first approach, it has little chance of lasting into the next century.

What is a data-first approach, and what areas can it influence? How can companies utilize big data to remain competitive in the future?

A Data-First Approach

Third-party cloud software, platforms, storage, or virtual and web algorithms allow a core transformation of the technological processes, analytic capabilities, and operational execution to work as a synchronized unit. When setting up a data-first approach from third-party services or when designing the big data algorithms oneself, the business side must engage and communicate in the same room as the technology side in order for engineering, business intelligence, and product marketing to be aligned and automated around a common goal. Cloud-based technologies profoundly reduce capital costs, exploit economies of scale, and increase flexibility so that small businesses and startups may compete with and surpass (or be bought by) established conglomerates.

Areas of Influence

The goal of big data operations and analytics is to streamline business performance, deliver superior customer service, and continuously reach new audiences. Companies that most efficiently utilize big data will integrate processes from each department as one unit so that they need to spend less time and money translating data into smart decisions. Many companies are incorporating big data into their systems, but do not fully unify the systems. Efficient big data systems optimize and unify several areas of influence.

Marketing

Marketing with the power of big data can influence user behavior by customizing product and service ads to consumer needs. Smart marketing will monitor all areas of user’s digital identities that may influence purchases — from their individual and economic profiles — to their web behavior, media habits, social media involvement, and e-commerce behavior. A consumer who spends her time on gaming platforms and forums will see gaming-related advertisements from smart marketers that harness big data. Google leverages its wealth of user information to tailor advertisements to the consumer while Facebook advertisements are based on user likes.

Not only does big data allow companies to increase their ROI due to better targeted marketing plans, but users are not bothered by irrelevant advertisements and instead receive offers which are tailored to their needs and wants.

User experience

Big data user experience algorithms offer the user systematic personalization and adjustment options that enhance customer experience. Most online shopping stores include filter options to let the user refine their search. Many search engines add an advanced search option to further narrow their results. Some review and information sites include Add To Compare options for quick feature and price comparisons. Better interfaces dock the horizontal line of items being compared so that they are always in view as the user scrolls down the comparison features.

Social media platforms like Facebook, Quora, and Twitter personalize user feeds according to one’s likes and follows, follow settings, or recent interactions. Google tailors its services so that the same search will produce different web results according to the user. Amazon’s big data systems monitor people’s purchase history and patterns, and display similar and complementary products at the best opportunities to influence another purchase.

Open source software, such as Wikipedia and Waze, exponentiates growth by enabling the user, expert, and peer networks to continually create, modify and advertise the services. Collective intelligence platforms allow for a better user experience by facilitating a larger quantity and variation of data collection in real time.

Most e-shopping platforms generate location-based shipping costs per a selection of carriers for the user to easily choose from. Users are presented with all shipping options and prices as soon as they enter their shipping address.

When companies use big data to streamline the user experience systems and leverage user identity data, customer service is personalized on a broad scale, benefiting both companies and customers alike.

Pricing

Pricing balances short-term profits and long-term market share. Higher prices raise profits but lower market share; lower prices raise market share but reduce profits. Firms consider market demand, production costs, and competitor responses to set prices that maximize long-term profits.

Though the theory is simple, the practical execution is complex. The economic algorithms for market demand and the game theoretic predictions for competitor responses are uncertain, and humans cannot sift through millions of transactions to select the best model. Innovative internet retailers now use big data to determine optimal prices, which they may change weekly or even daily. Amazon collects consumer purchase decisions for each price and feeds the extensive data into computer algorithms to estimate market demand.

Manual pricing is complex, so firms used to change prices only once or twice a year. Amazon’s algorithms are so efficient that optimal prices for millions of products are re-estimated every few days. Amazon was once criticized for its frequent price changes, as though it sought to take excessive profits from unwary consumers. Now Amazon explicitly shows the price changes. If you place several items in your Amazon cart but don’t buy them, you can see some prices rise or fall each day. Amazon understands the concerns of many consumers and their desires to make informed decisions.

Operations

Big data optimizes operations throughout the supply chain, and predicts demand to improve delivery and inventory management for the long and short term. Big data is not just more efficient; by reducing human work, it reduces production costs.

Alibaba receives 12.7 billion orders a year while employing only 26.8 thousand workers. To appreciate its scale, divide the figures by 13 thousand: imagine a firm with two workers but a million orders a year. Alibaba’s success is based on connecting consumers and businesses to manufacturers through cloud technologies. Alibaba uses data cloud computing, a shopping search engine with a background-checked rating system, and escrow (Alipay) and direct electronic payment services.

GeekWire shared a video of Amazon’s new high tech distribution center that shows how Amazon incorporates big data at each level of its supply chain to effectively distribute 1.1 billion orders a year with as few workers as possible. While Amazon employs 150,000 workers now, the electronic commerce and cloud computing company plans to reduce the number to 55,000 employees by 2021.

Shopping behavior

Consumer’s purchasing decisions vary according to three main influences. Traditionally, a consumer’s prior beliefs and experiences with a brand or item (we’ll call it “prior knowledge”) would be the initial consideration before making a purchase. This can often be influenced by company marketing efforts such as packaging, branding, and pricing (“marketing”). More recently, however, these two mechanisms only go so far in creating the optimal user experience. More and more consumers are turning to the powerful “review variable.”

User and expert reviews, as well as social media posts, increasingly determine shopping behavior, as they give users a third dimension of what it is like to own or use a product they’re considering. Purchasing decisions become a zero-sum game. The more a user relies on the review variable, the more her reliance on marketing and prior knowledge is reduced.

Reliance on the three influence sources may fluctuate depending on the type of products, the time of consideration, and the brand. For example, marketing influences may be stronger in person, when access to user reviews is limited. At times of urgency, such as Black Friday when sales are short, there is little time to reference reviews. Additionally, certain items will tap into emotions and garner high brand equity, such as luxury products. User and expert reviews have taken on increasing weight, however, as shopping has moved online.

Companies or peer-review hosting sites are remaining competitive by taking the “review variable” into consideration. They incorporate big data analysis of user opinion or host a complementary and trustworthy peer review network. Walmart, Drugstore, and Amazon e-commerce stores integrate the “review variable” into their architectures effectively. The platforms support user reviews, and their algorithms not only create suggestions based on prior purchases, but also suggest relevant items that other users have either looked at or purchased while considering similar goods. The Yelp and Trip Advisor peer-review platforms play a similar role in harnessing the influence of the consumer as a powerful factor in determining the quality of commodities and services.

The Harvard Business Review explains that companies whose market influence depends to a large degree on the user reviews must take this into consideration in order to remain relevant into the 22nd century. These companies must redirect their marketing resources toward big data analysis of review sites, user forums, and social media.

The Future

E-commerce global markets are fast growing, and huge global companies are putting all their efforts and resources into trying to dominate the established and emerging markets. These are already very efficient markets characterized by huge turnover, stiff competition, and small profit margins. Any small optimization advantage can make a huge impact for a company, and make the entire difference between a market leader surging past the competitors and a minor player. It’s a big data game with millions of products, brands, and categories, with dynamically changing prices, inventory, trends, and demands. The ability to gather all this information in real time, analyze it efficiently, and make smart decisions based on that information will determine the winner. Companies that want to stay ahead of the game are adopting an integrated approach in which all its optimization elements are fully synchronized.

About the Author

Alon Gamzu is a co-founder of Roundforest, the developer of the E-commerce and big data technology behind Comparaboo. With a Bachelor’s of Engineering and a background in Machine Learning, Alon worked for Intel’s analytics team, solving various prediction and optimization challenges. After working at Intel, he held a unique position at Google, consulting Israel’s global technology startups and helping them shape and execute their global growth strategy. With experience in both analytics and strategy consultation, the next logical step was to combine the two and launch a company that does just that. Since founding Roundforest in 2014, Alon has helped millions of users enjoy engaging products and technology, delivering big data in a clear and concise way.