How Digital Twins & AR Improve Safety and Minimize Inventory Shrinkage For Shopping Centers
Large shopping centers and malls have become like towns with large crowds and vehicles continuously moving in different directions. For instance, the Mall of America, located in Bloomington, Minnesota, United States, attracts ~40 million visitors annually, which is like a city in itself with a population of about 110,000 inhabitants (110,000 people visiting the mall every day). There is also a growing list of retail centers with a million visitors a year.
On the one hand, a large number of visitors to malls is a critical performance indicator reflecting higher sale at their stores and a positive impact on their space leasing rate. On the other hand, vast crowds in malls also mean more disorderly people, thieves, vagrants, panhandlers, criminals, and sometimes even terrorists making them vulnerable to violent incidents and shoplifting.
Safety and Inventory Shrinkage Issues at Retail Centers
Though not frequent occurrences, violent crimes result in enormous reputational and collateral damage. Recently, on November 7, 2018, a middle-aged man shot a 25-year-old man dead inside Target in The City Point Shopping Center, Brooklyn, New York. In the U.S., violent deaths at retail centers have increased by 12% in 2018 from the previous year. In 2017, violent incidents led to 424 deaths with ~94% deaths happening inside the retail premises (e.g., stores and parking lots). The retail/sales employees account for 26.3% of the total workplace homicide victims in the U.S. Most casualties were due to gunshots.
Loss of inventory from stores or retail shrinkage due to theft by employees, shoplifting and other errors cost the U.S. retail industry $46.8 billion in 2017. Shoplifting, the most significant factor, accounts for 36% of the retail shrinkage in 2018, followed by employee theft with 33%.
Thieves often target shopping center’s parking lots during holiday seasons. On October 8, 2018, thieves broke into around 100 cars at Georgia Mall. There were many car break-ins reported at several popular malls including Columbiana Centre, Columbia, and Gateway Center, New York.
How Do Digital Twins and Augmented Reality Help?
Digital twin models and augmented reality visualization, also referred to as immersive data visualization, can help retailers and mall operators curb violence and retail shrinkage.
Digital twin is a 3D digital representation of a physical shopping center. It uses advanced statistical decision methods, artificial intelligence (or machine learning), computer vision, high-power GPU computing, and virtual reality game engine technologies for data mining and visualization. The software model collects data from the internet of things (IoT) sensors, equipment, and business systems, and show them on this virtual replica in real time providing managers a complete situational awareness of the mall’s operation. Just like a video game, a user can immerse into a virtual shopping center to examine assets with a 360-degree view.
For many organizations, a digital twin acts as a centralized platform to simulate hypothetical situations to conduct a what-if or root-cause analysis with updated data for the enterprise to make timely decisions. Digital twin monitors and predicts impending events using historical data at shopping centers. For example, the software can provide the number of people currently in a mall and predict what could happen if 1,000 more visitors come to the mall in terms of availability of parking lots, air quality inside the mall or the difficulties in managing the crowd flow.
Augmented reality (AR) app uses the same technologies as digital twin, but instead of placing a user’s inside a virtual reality, it places digital content in physical reality. For instance, an AR app can superimpose a 3D digital version of a heating, ventilation, and air conditioning (HVAC) system on a table or the physical HVAC system itself allowing users to see the actual workings and real-time performance data of the system without opening it. An AR app becomes compelling information delivery system to manage field operations when integrated with a digital twin.
Users can access digital twins and AR apps on their smartphones, smart glass (e.g., Google Glass) and head-mounted devices (e.g., HTC Vive). Intuitive 3D visualization, portability, and ability to collaborate with remote experts in real-time makes immersive data visualization must-have tools for smart shopping centers.
Inventory Loss Prevention at Retail Stores
Surveillance cameras and sensors capture the movement of people, and machines in malls 24/7. They send streaming video and geospatial data to digital twin’s powerful analytical engine that has inbuilt computer vision technology. The data transmitted by the cameras and IoT sensors have metadata including time, location, size, and many other attributes used to locate and create behavioral characteristics related to objects.
The digital twin software analyzes facial expressions and body gestures of people inside the mall to detect any criminal intent. If the visitor’s postures match with those of a shoplifter, the software sends an alert with location and photo of the suspect to a nearby clerk to monitor his or her activities. The clerk goes to ask the suspect if he or she needs any help. The visit by the store’s personnel sends a message to the suspect that he or she is being watched and should not try to steal anything.
Digital twin can also detect checkout abnormalities when a cashier avoids or fakes scanning — also known “sweethearting” in the retail industry parlance — and informs his or her manager in real time.
Sometimes suppliers bill retailers even when they have not delivered goods at retailer’s stores. Digital twin can monitor and create a complete log of store deliveries using data from camera feeds, entries at gates, and delivery vehicle’s details; hence, the frauds related to bogus shipments can be checked.
Enhancing Safety and Security Measures at Retail Centers
Digital twin can store photographs of criminals, thieves, and blacklisted people and the data related to their vehicles. If a person of such background or the vehicles linked to criminals or bogus companies enters a mall premise, the software model can recognize them using face recognition and number plate scanning algorithms. Then, it sends alerts to security personnel to stop them at the gates and increase vigilance to see if they have any dangerous possessions such as knives or guns.
Digital twin’s algorithm can spot the suspicious behavior of a person, assign a threat tendency score and alert authorities. It can also detect an unattended bag and send a notification to security personnel along with its location for swift action. It can also alert authorities if an unauthorized person tries to enter a restricted area.
Digital twin can detect an area where ventilation is not proper or has an elevated level of CO2 concentration to inject fresh air. If a mall has fast food kiosks, paint, and home goods stores, many products in these stores produce volatile organic compounds (VOCs) that are associated with respiratory problems.
People often lose their belongings (e.g., watches, wallets and smartphone in restrooms or cash counters); sometimes their kids get lost in a crowd at a mall, or a sudden illness (e.g., epilepsy attack) strikes a person. Digital twin can locate left belongings, missing kids and people in an urgent medical help using the data from IoT sensors and camera feeds.
Digital twins can spot cracks in the shopping center’s building and stores using LIDAR — a detection system that uses lasers to find structural displacements. Facility managers can use this data to schedule maintenance work and avoid any accidents.
In an emergency, digital twins and AR apps can provide the location of and direction to trapped people inside a center for a speedy evacuation. They can also help in locating faulty equipment or fire-affected areas on a 3D indoor map along with instructions to repair and manage them, so that first responders or workers respond faster and make informed decisions. Digital twins using data from acoustic sensors in a mall area can detect if a sound is of a gunshot and find the place of occurrence to inform security personnel.
Deploying Digital Twins and AR Apps
Gartner predicts that 50% of major enterprises will use digital twins of their organizations in combination with digital business platforms by 2023. Shopping centers can deploy digital twin and AR visualization solutions by connecting information systems of their operations, sales and building management systems to the software. They can use predefined schema and existing digital assets of technology providers to accelerate the development of digital twins.
Digital twins have self-learning algorithms and are as good as the amount and quality of data we provide to them. Shopping centers have already installed several digital technologies and surveillance sensors generating enough data for state-of-the-art technology to make their spaces safe and secure for people visiting them. As and when they deploy new monitoring technology, they can keep integrating it to their existing digital twin making it an even more robust decision and monitoring system.
Author
Jan-Philipp Mohr — Founder & CEO of Hashplay
Apart from leading Hashplay, Jan-Philipp is a Mentor at Founders Space in San Francisco. He has also spoken at numerous Events about VR&AR, AI, and Visualisation in the US, Europe, and Asia. He holds a degree from the University of Kent in Business. In his spare time, he is in involved in multiple philanthropic organizations.
This story is published in The Startup, Medium’s largest entrepreneurship publication followed by +408,714 people.
Subscribe to receive our top stories here.
