IPG Media Lab
Published in

IPG Media Lab

Visual Search: The Key to the Future of Instant Commerce

Are your products camera-ready?

Shopping often starts with an inspiration, either out of necessity or want. The best kind of shopping experience is usually spontaneous, imbued with a sense of playful discovery where our brains release doses of dopamine when we happen to come upon something that triggers that “got-to-have-it” impulse. But the bummer is, not everything in the real world is shoppable — nine times out of ten when we come across something we’d like to buy, we are not in a retail environment where we can simply pick out what we like and head to the checkout; instead, we have to make a note of it, mentally or written, and go to a store, physical or online, to actively search for it. No wonder marketers call this process from discovery to eventual purchase a shopper journey, for a long voyage it sure is.

But just as the advances in commercial air travel significantly shortened the time to journey across the oceans, the impending mass adoption of visual search tools is about to forever alter the path to purchase. When you can simply point your phone’s camera at any item that piqued your interest and be immediately directed to a shoppable page of the said product, suddenly the shopping funnel collapses and every manufactured object can become a piece of shoppable content. It’d be like traveling with a first-class VIP pass, but unconstrained by scarcity and therefore accessible to far more consumers.

Granted, we are still in the early days of this shopper journey revolution: our proprietary Futurecaster survey results indicate that, as of Dec. 2019, just north of 30% of U.S. mobile users have heard of visual search, and only about 10% have actually used a visual search tool. Our findings corroborate the data from an October 2019 survey by eMarketer and Bizrate Insights.

However, with visual search tools such as Google Lens and Pinterest Lens, as well as visual search features from Amazon, Facebook, and Snapchat now readily available to mobile users, the future of instant commerce is looming just over the horizon. What will finally push it over the edge into mainstream usage? How will it spark the instant commerce experience for various categories? And what are some of the things we as brand marketers can do to prepare for that future? Armed with the fresh learnings from our recent NY Retail Innovation Week event with Pinterest and PSFK, we are ready to tackle these questions one by one.

Camera as the New Keyboard

Last Wednesday, in collaboration with Pinterest, the Lab organized an event as part of PSFK’s NY Retail Innovation Week to talk about how visual search is about to change how we find inspiration in life and make purchase decisions. Led by Christina Adranly, our Director of Strategy, and Vikram Bhaskaran, the Global Head of Vertical Strategy and Marketing from Pinterest, we spent a lovely two-hour event taking a deep dive into the ongoing evolution of visual search and how it is set to change shopper behaviors.

Visual search is more than just a new way to search. It is a new express path to purchase that will unlock the future of instant commerce. Typically, it allows consumers to use specialized search tools powered by computer vision to identify physical objects and surface results of similar images and links to the web pages that contain them. If one used visual search on a manufactured product, then ideally the visual search engine would correctly identify the product, match it with the product photos of the item in question, and direct the prospective buyers to the product page. Instead of typing into Google an often inaccurate description of the said product, your phone’s camera becomes the new keyboard, and it will do the input job better than our words ever could. A picture is worth a thousand words, after all.

One of the key findings that Vikram highlighted in his presentation was that, according to Pinterest’s first-party data, visual search is a preferred tool during both the inspiration and purchase phrases of the shopper journey (64% and 53% of shoppers, respectively, picked it as their preferred search method), but not as much during the research stage, where text-based search still reigns by a large margin.

This distinction makes perfect sense in two ways: For one thing, visual search is very much a one-and-done kind of search by design. At the moment, it is difficult to modify a visual search query like one easily could with text-based search, given that it would require changing the appearance of the object being searched. Contextual cues are often needed to narrow down the search results and make comparisons, which is a common thing that many do during the research phase of shopping, and so far, there is plenty of room for visual search providers to solve this. For example, they can create easy ways to follow up a visual search query with modification on various criteria such as the size, shape, colors, and prices of the items being visual-searched via text or voice input, both of which offer more flexibility than visual search, or with pre-set drop-down menu as one would today to specify the recency or the size of the search results in Google Image.

For another, a good visual search experience, ideally, should be able to take us directly to the page where the searched item is available for purchase from our preferred retailer, be it Amazon or Target, therefore leapfrogging the entire research process. Of course, this has hardly been the case in reality so far. Most visual search tools have yet to integrate with the backend of ecommerce operations, typically powered by software made by companies like Shopify and Salesforce, and thus lack the kind of instant shoppability that an optimized visual search product would be. Even the supposed Snapchat-Amazon collaboration on visual search seems more like a unilateral move on Snap’s end rather than a deep integration between the two companies. So far, makers of visual search tools are mostly still focused on improving the accuracy of their results and have only just started to make a concerted effort to explore the commercial potential of visual search.

The Adoption Hurdles

Still, the consumer demand for a simplified shopping experience is obviously in place. 62% of millennials want visual search over any other new technology, a 2018 study by Visenze found. The latest data from Pinterest found that 55% of consumers say visual search is instrumental in developing their style and taste. And unlike voice search, people who hadn’t tried visual search tend to be more interested in it. According to the aforementioned eMarketer/Bizrate survey, 53% of U.S. internet users surveyed said they have not used but are interested in trying visual search, compared to the 28% of respondents who said the same to voice search on smart speakers.

Given the concrete consumer need in the market, it is expected that the global visual search market will surpass $14.7 billion by 2023, growing at CAGR of 9% from 2018 to 2023, according to a 2019 study by Report Consultant. However, for that estimation and all the previous talks of instant commerce to come true, visual search will have to increase penetration and reach the mainstream users. Given that for most smartphones today, visual search tools are only available via third-party apps, system-level integration of visual search into the default camera app of the phone would likely be the best bet to push for mainstream adoption, mirroring the path of QR codes to become a standard supported by built-in solutions.

And since visual search is a digital search process based on physical objects, it carries an unavoidable social aspect in which we’ll have to negotiate with others around us about the rules around consent and permission to conduct visual search on other people’s clothing and possessions in public or private settings. Not everyone wishes to become an influencer, but rampant visual search use may just turn every stylish pedestrian into one. What used to only happen on social media will soon be extended into the real world, and it will only be a matter of time before visual search becomes an integral part of social commerce, given most social platforms’ increasing tendency towards ecommerce integration as a way to diversify revenue streams.

The convergence of search and social will accelerate as visual search technology matures and improves in accuracy and speed along with advances in machine learning and computer vision research. Of course, there are other important factors, such as logistics and supply chain innovations, mobile payment integrations, and on-demand delivery services, that need to be in place for the future of instant commerce to come true. That being said, mass adoption of visual search is what will change shopper behaviors and jumpstart the whole shift, whose impact will reverberate across various industries.

The Cross-Industry Impact of Visual Search

Existing visual search tools tend to start their commerce integration with the usual suspects: fashion and home decor — two heavily visual-dependent product categories. According to the previously mentioned report by The Intent Lab, when shopping online for clothing or furniture, more than 85% of respondents respectively put more importance on visual information than text information.

Amazon, in particular, seems to be very invested in making inroads into the fashion sector through visual search tools. Starting with the discontinued Echo Look smart camera, which was designed to help users compare outfits and recommend similar apparel items, to their “Shazam-for-fashion” StyleSnap feature in development for the Alexa app, Amazon has been making repeated attempts at using visual search as a way to inspire shoppers to consider buying fashion items from Amazon by trusting its algorithmic recommendations for similar-looking items, many of which may be from Amazon’s own private labels.

Similarly, Google also recently added a new “style ideas” functionality to Google Lens, which enables users to point Lens at clothing, fashion items, and furniture to find the product or similar ones online. Other notable fashion and furniture retailers that have leveraged visual search to innovate their brand experience include Wayfair, IKEA, Walmart, and Target. It is worth noting that both IKEA and Target’s visual search features are powered by Pinterest Lens, while Walmart and Wayfair are both developing in-house visual search solutions. To reassert control over the discovery process, expect more retailers to develop their own visual search capabilities by teaming up with a third-party, white-label solution provider, rather than relying on search engines and social networks to act as the intermediary.

While fashion and home decor are a natural fit for exploring instant commerce by way of visual search, many other verticals can also benefit greatly by tapping into this emerging search tool to engage prospective customers and drive sales. Auto dealerships, for example, have long been looking for a mobile-first tool to spark consumer interests and boost test-drive booking. Visual search is a great fit for auto retail, because it allows car-lovers to easily identify the model and make of any random vehicle that caught their eyes and, when integrated with the proper backend, prompt them to book a test drive at a nearby dealership.

Food is another industry that visual search could bring some interesting changes to the shopper journey, especially in terms of grocery shopping and at-home cooking. One consistent pain point for people shopping for groceries is to figure out what ingredients are needed for a certain dish. By integrating with recipes sites, the visual search feature could not only help grocery shoppers and home chefs to differentiate similar-looking ingredients, but also suggest related recipes and, should they select one, the other ingredients needed.

Thinking outside the box of product discovery, visual search can even be leveraged for re-engagement. For example, pharmaceutical brands could develop a visual search tool that helps customers to figure out which of their subscriptions with confusingly long names need a refill without mixing things up and ordering the wrong thing. By making it easier to order refills, visual search can help pharma brands improve brand loyalty and develop a new touch point to engage customers.

Brand Takeaways

Consumer expectations travel across categories, and so does user behavior. Once visual search overcomes the adoption hurdles and hit mainstream usage, its impact likely won’t be contained in these obvious categories for long. What should brands do to prepare for a future of instant commerce powered by visual search? There are four high-level tactics to keep in mind.

For starters, brands need to be sure to make sure their branded assets, both in 2D and 3D, are correctly indexed and easily accessible. Product images — in and out of packages, in various types of lighting and from various angles — should be made available and properly labeled on each product page, both on websites and retail partner sites, to make sure that your products will show up correctly in visual search results and are probably linked to product pages where purchases can be made. Part of that may be working with retailers to ensure product pages are properly indexed and optimized for visual search.

Revamping your own website, either to add an online shop that sells directly to consumers or to ensure each of the product pages links out to the correct pages on retail sites, is another necessary tactic that brands need to take to future-proof online assets. According to research firm, Gartner’s estimation, early adopter brands that redesign their websites to support visual and voice search will increase digital commerce revenue by 30%. Evaluate the workflows and find data management partners to connect product data to visual search queries that occur both on- and off-site should be a priority.

In order to capture the kind of serendipitous instant commerce moments sparked by visual search, brands should also consider unique product or package design to make sure your products stand out visually. The largely homogenizing aesthetics plaguing the direct-to-consumer brands simply won’t work in a world where visual search becomes the primary mode of product discovery.

Lastly, it is also important to remember that visual search, like other forms of search, will develop its own set of SEO rules. How to make sure your product would show up when someone snapped and searched for a competitor’s product, or a similar product from your own product portfolio, is something that future-forward CMO should be thinking today. Visual search often indicates explicit intent to find out more about something, and it is up to brands to capture that intent and translate that into sales.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store