3 Ways We’ll All Be Shopping with Robots

[Note: This is Part 4 in a five-part series on TickTock’s robots: P1, P2, P3, P4, P5]

The “death of retail” is greatly exaggerated and it’s actually an area ripe for disruption. There’s just something inherently human about shopping in person vs online: we like to touch and feel products, we share the experience with friends and family, and there’s an immediacy need that’s only served by walking out with a must-have item in your hands — now!

At TickTock, we dove deeply into retail logistics using our new point to point mobility platform, and found tons of opportunities for automation. There’s also a growing desire to experiment and try new ideas, from curbside pickup, in-store online order fulfillment, or augmented reality navigation.

Examples of new uses cases retailers are exploring with robots and augmented reality: Online order with curbside pickup; promotional displays that move; automatic floorplan generation; on-demand shopping services; and 3D maps for AR shopping.

A robotic solution would need to constantly update a 3D map of the store and be dynamically assigned new routes as associated asked it to perform new tasks. It would carry products and/or arrays of sensors as payloads.

So where’s the biggest opportunity?

For all the hype around warehouse automation, the supply chain doesn’t end when a truck leaves the distribution center. One retailer told us that 75% of their material handling costs actually happen after products arrive at the store. The costs are due to the massive amount of labor spent sorting and moving goods around from truck to shelf.

There are many stages of the supply chain between when something is produced in a factory and when it arrives at your home. The automation in manufacturing and distribution centers gets a lot of attention, as does delivery services. Automation of inventory within retail stores is an overlooked opportunity.

Beyond the obvious time waste for humans to push carts around, the business metrics are very clear. The U.S. Bureau of Labor Statistics lists Retail Trade as the 5th ranked job market after health care, professional services, government, and leisure. The BLS also shows that massive job openings exist and are going unfilled in the US thanks to record low unemployment.

Retail is growing, it’s powered by a massive labor force, and it can’t get enough people to get the work done.

TickTock’s mobility explorations with carts and tower designs showed promise but the value might not be the same across home goods, clothing, grocery, or other retail category. We again looked to the BLS, which confirmed the stock clerk role as a top three job function in retail, with a clear prominence in grocery.

Digging into the workflow

The entire TickTock team spent several months visiting dozens of local retailers to observe their inventory replenishment processes. The workflow primarily centers around products arriving via truck in boxes stacked high on pallets, before needing to be unpacked individually onto store shelves.

We assume that trucks will still deliver products stacked on pallets for the time being. The wasteful process of re-sorting them onto carts, pulling them out to shelves, and in-aisle merchandising needs improvement.

To our surprise, the careful placement of products onto shelves, known as “merchandising”, wasn’t the slow part. We found stock clerks and their amazingly capable human arms doing this at a rapid pace with an incredible precision that robot arms aren’t even close to matching yet.

There is however an immediate opportunity to eliminate the time spent walking carts around. These journeys take many minutes per trip in large stores, especially the mid-day restocking that happens when customers are actively shopping.

Stage 1: Eliminate all the walking time

With the TickTock robots transporting goods, there would still be a stock clerk in the back that unloads boxes from pallets as they do today (see position 1 below). This person typically has a back brace and isn’t customer facing.

Today’s grocery stores and big box retailers are huge. It requires a lot of walking time to get products from the back out to aisles.

There would also be someone in the aisle to remove products from boxes and place them onto shelves (see position 2 above). These merchandising associates waste multiple hours per day just running back and forth to return empty carts and fetch new ones to take out to the aisle.

It’s time for retailers to benefit from the virtual conveyors being used elsewhere in the supply chain.

You might not have noticed, but we found that a Walmart Supercenter might have as many as 30 active associates dedicated to inventory replenishment. Another retailer told us they spend 1/3 of all labor costs on inventory replenishment, adding up to hundreds of millions of dollars per year.

If retailers could get anywhere close to the 200% efficiency gains found by warehousing robots, this is a multi-billion dollar opportunity.

Step 2: Improve the ergonomics

Our observations in grocery stores led us to an unexpected revelation: The design of current transport systems in stores is killing the efficiency.

It can take anywhere from two to seven minutes to walk from the back of a store to any given aisle. This drives retailers to use the largest possible cart (e.g. U-Boat, flatbed, or pallet) to transport as much as possible in a single trip. Unfortunately, this unintentionally leads to a massive amount of wasted time doing resorts in the aisle. We noticed an abundance of localized walking back and forth between carts and shelves since the carts are too big to park adjacent to where the products need to go.

This stocking associate spends only 7 seconds using his arm (a uniquely human task for now) during a one minute process of getting juice into a cooler. The majority of time is wasted walking a box back and forth (something a robot can do) because the U-boat cart is too large to keep in the aisle.

An adjustable mobile shelving system seemed like the answer. Our proposal would be an autonomous transport system that ALSO aided in merchandising products in the aisle by pulling them right from boxes without resorting.

A tower cart design would allow products to be within arm’s reach of where they need to be placed on shelves. The mobile robots would not only eliminate travel time but also speed up the merchandising in the aisle.

We had a limited fleet of prototypes but put them all into action in a local Target to further visualize the process. We don’t show anyone loading products from the back, but you can see the robots rolling out to the aisles where our head of computer vision, Murad Al Haj, deftly plays the role of stock clerk.

Some B-roll footage of TickTock’s fleet of prototype robots simulating product transport in a Target. Dozens of robots would be involved in the inventory replenishment process, a major labor cost to retailers.

At one point the robot was cruising past the fitting rooms when an associate walked by with her arms full of clothes to restock. She smiled and absolutely loved the idea that this little robot would help her carry things so she could focus on customers and keeping the racks in order.

Step 3: Offer customer assistance

Retailers certainly want the efficiency gains but also aren’t sitting still when it comes to the customer experience. Some are exploring mobile kiosks or robotic order pickup towers. There’s also a desire to assist salespersons, especially within clothing, allowing associates to remain engaged with customers while robots transported products to and from the stock room.

This concept rendering shows a TickTock One robot with Nike shoes on it for a customer to try on. Multiple sales associates could stay with their customers to answer questions and help find the right fit and style, while the robot could return to the back to fetch more products from a single stock clerk associate.

The general trend was conservative though. Retailers first want robots to help their associates until the tech matures, and then explore robots interacting directly with customers. Eventually this will include everything from helping them find products, to autonomous shopping arts, to curbside pickup for online order delivery.

It’s hard to push a shopping cart, look at your mobile shopping list app, and grab products from the shelf at the same time. It’s even more frustrating when you can’t find something. A mobile robotic shopping cart could guide customers to products while allowing them to use both hands freely to shop.

An idea too soon

TickTock’s augmented reality controller (known as ARRViz) made it super easy to update maps, define these new behaviors, and see what the robots were thinking. The room lit up with smiles in a presentation to a major retailer when a slide said “If your associates can play Pokémon Go, they can train a robot”.

Most retail associates can be found with a smartphone in their pocket these days. Many are wrapped in an industrial case, often with a barcode reader on the back. Why not turn these into augmented reality navigation and robot training devices since they typically run Android already and could use ARCore?

The idea of empowering associates to re-task robots and focus on higher value work for themselves really resonated. So whether it’s cost savings, improving the customer experience, or trying new business models, there’s a big opportunity for robots and automation in retail. TickTock was too soon here and unable to bring these ideas to market, but I’m confident we’ll see a growing number of robots coming to a store near you.

[This is Part 4 in a five-part series on TickTock’s AR-powered robots. Be sure to check out Amazon’s Echo Show on Wheels, Consumer Robot Concepts, Low Cost Mobile Robots Using Android, Robots for Retailers, Augmented Reality and Robotics Overlap]