How Real-Time 3D is Changing the Way We Shop

The near future of retail will feature interactive real-time 3D models of products in search results, digital ads, mobile apps and AR/VR — changing how products are discovered, explored and purchased online.

IKEA Place: view furniture virtually and try it in your home with AR

In late April, the Khronos Group, an open industry consortium for 3D graphics technologies, issued a press release announcing the formation of an exploratory committee to create standards and guidelines for representing retail products in 3D. The announcement was accompanied by little fanfare, but it marked the beginning of a big change in how products will be sold online.

The Khronos 3D Commerce initiative is being driven by major players in technology, retail and manufacturing, including Facebook, Google, Microsoft, Adobe, Autodesk, Unity, Deloitte, Houzz, IKEA, Lowe’s, Target, Walmart, Wayfair and Samsung. The goal is to leverage the amazing capabilities of 3D, VR and AR to enable a global infrastructure for consumers to better discover, engage with and ultimately purchase products, and suppliers and retailers to scale up production of 3D content, making it as commonplace someday as text and images.

From virtual catalogs and showrooms to search results and ad units, the potential for displaying physical goods as 3D virtual models is enormous. Appliances, apparel, accessories and home furnishings are just a few of the product categories that lend themselves well to being represented in 3D, allowing a buyer to view a product from all angles, zoom in and out, and click/tap on parts of the 3D model to get more information while considering a purchase.

Having trouble picturing this? If you’re not up on what’s possible these days with real-time 3D, check out this virtual living room experience by clicking on the embed below and using your mouse or finger to explore. It uses the Sketchfab 3D model-sharing service to deliver and render the content into the Medium page that you’re reading. Sketchfab doesn’t use the new Khronos 3D Commerce standard (which doesn’t exist yet), but it does employ related technologies like glTF and WebGL, the Khronos standards for delivering and rendering 3D on the web.

3D living room via Sketchfab

Services like Sketchfab are made possible thanks to the convergence of several innovations in computing that, taken together, are driving the adoption of real-time 3D as the next major media type. Expect to see real-time 3D content everywhere in the coming months, with retail front and center.

The Benefits of Real-Time 3D for Shopping

Now that you’ve gotten a taste of what’s possible, let’s drill down into why this medium is so powerful for the shopping experience.

  1. It’s More Engaging. In a world where brands are increasingly fighting for attention to cut through the digital clutter, 3D provides higher production value and more interactivity. From a visual and interaction standpoint, it’s effectively “gamifying” the shopping experience. The last couple of generations of consumers have grown up on video games, CG movies and interactive devices; the shopping experience needs to match the digital world they inhabit.
  2. It’s More Informative. The ability to interact with any part of a 3D model means that the shopper can explore it to get more information about a product on the spot: for example, see the interior workings of a coffeemaker simply by pinching to get an “exploded” view (rather than scrolling through more images), or tap on a specific part of the machine to get more information (versus opening another page to see technical specifications).
  3. It’s More Intuitive. The world is 3D; people move, think and experience in three dimensions. As the technology to power 3D has become ubiquitous, computer interfaces are generally following suit. When it comes to a product catalog, why scroll through a series of images to see different views, when you could interact with a photorealistic model that you can see from any angle and at any distance?
  4. It’s More Useful. By displaying a 3D model in augmented reality, you can see how it would fit into your house, or in the case of clothing, how it would look on you or match your wardrobe.

On the consumer side, this is going to open up a world of product try-ons, try-outs, interactive product explorers and virtual showrooms. Additionally, AR allows allows buyers to measure their living space just by pointing the camera.

The Measure App for iOS

For retailers and manufacturers, an enhanced shopping experience promises to not only increase sell-through, but reduce returns and save shipping costs, as buyers will have a greater chance of getting things right the first time. The additional level of engagement also suggests new data opportunities, as these 3D services can learn the ideal viewing angles, the most-explored product features, and the best configurations and combinations within collections.

Early Experiments

With the advent of ARKit and ARCore, consumer augmented reality is accessible on a billion mobile phones and growing. This boom in immersive technology has encouraged retailers to experiment with representing their products in 3D for a variety of use cases.

Shopping apps. IKEA and Amazon have both released AR apps that allow consumers to see products in their homes before buying. Nike has introduced an AR feature into its app that allows customers to measure and try styles before buying. Online platforms like Shopify and Wayfair have demonstrated shopping in 3D using Apple’s AR QuickLook feature, where a 3D model shows up in the Safari browser and with a swipe, the user can drag the product into the real world, putting a blender on a kitchen counter or a motorcycle in their driveway.

Social networks. A year ago, Facebook demonstrated “3D posts”, allowing 3D objects to be placed in the newsfeed: in a similar fashion to Apple’s AR QuickLook, virtual models of for-sale items can be brought into the real world with a gesture. Leveraging its amazing computer vision technology, Snapchat allows its users to try on glasses, hats, makeup etc. on their own face. Both of these social platforms provide the ability to not only advertise within the experience, but also purchase directly as part of the flow.

Advertising platforms. My employer Unity powers an advertising platform that shows ads within mobile games, including AR ads. Brands have been piloting with us to engage consumers in new ways via AR; for example, smartwatch maker Fossil ran a campaign that lets you explore the various features of the watch, change styles and colors, and finally, try it on using AR. Verizon Media has run several AR ad campaigns in its owned-and-operated properties; for example, they ran an ad unit with The Home Depot during Christmas of 2017: click a link in the Yahoo! Mail interface, and a Home Depot-sponsored Christmas tree shows up in your living room, complete with blinking lights.

Unity Fossil Watch AR Ad Unit

The foregoing represent amazing innovations, but until recently, they have all been trapped in walled-garden ecosystems that run only on a specific platform, created with dedicated tools targeting only that platform. None of the 3D content is searchable online, shareable between apps, or able to be repurposed between tools and applications even within a single organization. This limits reach, creates undue friction throughout the value chain, and makes what could be an amazing return on investment cost-prohibitive to deploy at scale. But with the Khronos 3D Commerce initiative, things are about to change. The kindling is in place to start a wildfire.

Next Steps

The Khronos “exploratory group” was formed as the result of a series of ad-hoc discussions among the companies listed above. After identifying the problems and committing to a plan of action, these firms decided to bring the effort to an organization devoted to creating industry standards, and Khronos was the obvious choice. They’ve been at this for twenty years, developing standards for 3D graphics hardware, software and content creation.

Process-wise, the exploratory group is at the drawing-board stage, engaging in a rigorous and formal process to establish requirements and fully understand the scope of the problems to solve. But the overall goal is as we’ve outlined here: to make shopping in 3D an everyday thing. Once the exploratory group wraps up, work will begin via an official Khronos working group that will develop new specifications and/or enhance existing standards such as glTF to meet the specific needs of retail.

In the meantime, nobody is standing still. During the keynote at Google IO, the company’s annual developer conference, they teased a future of search, AR and shopping that is directionally aligned with the Khronos 3D commerce initiative. A Google search for sneakers displayed multiple results that included glTF models you could spin around to see at all angles, and then with a swipe bring into your world to see how it might look with the rest of your wardrobe. From search, to browser, to AR — seamlessly.

Google search result of a New Balance sneaker, displayed in AR

Get Involved!

If you’re a retailer, supplier, tech provider, content creator or anyone who services clients in these roles, we’d urge you to start preparing for this near future today.

Start working to digitize your products now. There are already several solutions for converting existing 3D models from CAD packages to glTF, or scanning existing real-world items. And expect to see more solutions come online as this initiative gains steam.

Experiment with ways you can use 3D and AR to tell your brand’s stories and show off your products in the real and virtual worlds. The scenarios described here are going to be fairly common, but there is a really rich palette to work with, and plenty of room to innovate on design, interaction and utility. And, there are already many examples to learn from, created by ambitious and adventurous brands.

Join the Khronos exploratory group. We’re looking for people and companies to contribute to the effort.

The future of retail is immersive. And it’s now. Come help us build it.

This is the first article in a series on the role of 3D graphics and XR in the near future of retail. In upcoming posts, we will explore the use of real-time 3D for in-store/brick-and-mortar experiences, and behind-the-scenes merchandising and store planning.

Near Future of Retail

Exploring the leading edge of retail as the digital-physical convergence unfolds: AI/ML, VR/AR, voice, IoT, digital spaces. Sharing what we see and why it matters, and channelling our collective wisdom into practical guidance for those with a vested interest in this near future.

Tony Parisi

Written by

VR OG. Entrepreneur. Investor. Co-Creator, VRML & glTF. Head of XR Ad Innovation, Unity Technologies. Pre-apocalyptic author. Music. @auradeluxe

Near Future of Retail

Exploring the leading edge of retail as the digital-physical convergence unfolds: AI/ML, VR/AR, voice, IoT, digital spaces. Sharing what we see and why it matters, and channelling our collective wisdom into practical guidance for those with a vested interest in this near future.