Machine Vision and Augmented Reality: Venture Capital Investment Themes
Researched and authored by Jad El Jamous in collaboration with Jonathan Pfitzner and Anshul Gupta from Aurelius Advisers / Truffle Hound
The following is a research report that aims to provide founders and early-stage investors a better understanding of the value generated by emerging computer vision and augmented reality technologies. Each theme attempts to explain a specific application of those technologies within one industry or across industries. Theme 1–6 are focused on computer vision, while theme 7–11 dive into AR.
Theme 1: Robotic navigation to enable the autonomous vehicle opportunity
Driving forces:
For machines to understand and therefore interact with the real world in the same way that people do, they need to see and recognize what they are seeing. The special power of computer vision technology is not only in the vision aspect like the name suggests, but in consequently giving them intelligent behaviours such as motion and navigation. The major commercial innovation that arises is autonomous vehicles such as drones and cars, as well as autonomous robots that roam houses and factories. Self-driving cars are for example learning to react to road environments using computer vision on top of LIDAR technology (Light detection and ranging). In fact, there appears to be in the self driving car ecosystem a competition between currently popular LIDAR solutions and computer vision solutions, with many believing that LIDAR is not enough, including Elon Musk founder of Tesla and Jianxiong Xiao (a.k.a. Professor X) founder of AutoX. In addition to pre-built mapping, the next frontier of autonomy is real-time SLAM technology (Simultaneous localization and mapping) that uses machine learning.
Probable scenarios:
Robots and AVs will soon be all around us, and we will interact with them as much as we interact with our screens today. The IEEE estimates that 75% of all vehicles will be fully autonomous by 2040. Goldman Sachs predicts that $100 Billion will be spent on drones up until 2020, from which 70B is military and 13B is for commercial and civil use. After finding different applications in military, drones are now moving more into the commercial market with use cases that range from delivery to inspection. In the car market, major manufacturers all aim to get fully autonomous cars on the road gradually starting from 2020 with goals of increasing revenues from connected services and Transportation-as-a-service. The expansion around the world however depends very much on both consumer reception and local regulations. The optimistic scenario is fast adoption led by lower transportation costs for both businesses and consumers, much lower car accident rates, freedom from the wheel to focus on other things, and major traffic reduction in cities.
Investments & Considerations:
The commercial and consumer robotics market has attracted in June 2017 investment from SoftBank and Qualcomm Ventures in a $114Million round in autonomous robotic navigation startup Brain Corp, after the acquisition by SoftBank of the robot maker Boston Dynamics (previously owned by Google).
As for the drone market, $450M dollars were invested in 2016 According to CBinsights. Funding went to hardware manufacturers, mapping and imagery, and commercial applications. The Economist magazine explains that DJI Innovations has 70% market share in hardware and new startups are focusing on creating the complementary services. To enhance drone navigation, startups Airmap which does airspace management and Echodyne which is working on radar technology both raised their B series in 2017.
But while safe and accurate navigation in drones is very important, it is more important with autonomous vehicles because of the people aspect and more complex ground environments. Fueling the whole AV opportunity is traditional car makers reinventing themselves through acquisitions and partnerships to make fully autonomous cars a reality (i.e. Ford investing $1 Billion in Argo AI and GM acquiring Cruise Automation for $581M), and tech giants like Alphabet and Apple getting in the game to make the car their next mobile platform (not to forget Intel’s acquisition of Mobileye for $15.3B in March 2017).
The space includes startups focused on computer vision solutions such as Cortica, Drive.ai, Deepscale, and Chronocam. In addition, it includes world mapping startups like Civilmaps which has the goal of generating map data that helps AV navigation, Mapillary which recently opened up an API of data points after “collecting 130 million images from places all around the globe” using crowdsourcing. Similarly, the VR startup Improbable which raised a $502M this year, had announced at some point that it was researching autonomous fleet navigation as a use case.
The main challenge that wants be solved today is that the car needs to operate under all conditions and without errors, in order to become a safe product that consumers trust as a transport option. Computer vision-enabled AV navigation is today a growth market for investors as the race to level 5 autonomy continues among major car manufacturers, yet with many opportunities for new startups to find more accurate real-time solutions such as that of Five.AI and Oxbotica — these startups are finding use cases in taxi fleets and delivery fleets.
Theme 2: Cameras everywhere for real world visual data processing and analytics
Driving factors:
Billions of cameras are already embedded today in electronic devices and are capturing pictures and videos, while millions of them are also on the streets and inside buildings monitoring every corner (the UK has 5–6Million public CCTVs). Machine vision will turn physical world challenges into Big Data problems, explains the founder of Sight Machine on WIRED Magazine. A simple example is a store CCTV counting how many people come into a retail store every day, understanding their interactions with products and analyzing their path (Deepglint). By working with visual data, businesses can start experimenting and optimizing towards certain goals (sales growth, customer success, employee satisfaction, asset management) — this can lead to less risk, better performance and better economic productivity.
Probable scenarios:
Due to the benefits of continuous capture and sense-making, we can start to imagine how much more “pixel data” from the real world we will want to collect in the hopes to find problems and solutions that could not be seen before by the regular human eye. We will add more cameras everywhere around us in our homes and on our cars, and drones will complement ground cameras from above. Drones can monitor public spaces and deliver products fast to whoever needs them. LDV capital estimates that there will be 44 Billion cameras watching the world by 2022, up from 14 Billion today. Kevin Kelly, Founder of WIRED, talks in his book “The Inevitable” about total tracking and surveillance as an unavoidable force that will prove beneficial to the world.
Investments & Considerations:
The main area of investment here is real world analytics. Startups in the space can be doing just data supply and/or processing the data into insights that help governments, cities and businesses understand the factors affecting resources, risks and sales. Few investments were made in aerial imaging from satellites (ICEYE, Descartes Labs, Tellus Labs) and others in drone startups doing data capture/processing (3D Robotics, Precisionhawk, Slantrange, Identified Technologies) as well as those who combine it with hardware (Kespry). Visual data supply/analytics is a continuation of the much developed $130B big data market that is expected to grow beyond $200 Billion by 2020. The visual aspect could also prove to become the most important part of it as experts predict 80% of all data available will be in image and video capture.
Opportunities exist across the investment spectrum and across industries as new companies can find horizontal or vertical niche visual analytics. Industries like transportation, retailing and manufacturing are very interesting sectors because of the extra efficiencies needed that computer vision algorithms can solve for. Proprietary data access combined with data network effects ensure defensibility for the startups in this space, as they are monetized by spurring new insights that solve commercial problems and enable more informed decision-making. To achieve that, startups would either want to place their own cameras or access the data from partner camera networks.
Theme 3: Visual tasks automation for better productivity and higher customer value
Driving factors:
Because vision software can identify patterns in pixels and divergences from them, they can be used in B2B to ensure quality and inspect for faulty objects. Cameras can effectively do the job of human inspectors, diagnosticians and QAs. There are widespread predictions today that automation will prevail in the next few decades and that consequences may include big improvements in productivity and efficiency. One only needs to look at Airports today where security, passport control and ticket checks are done by machines with embedded cameras in the hopes of increasing speed of movement and passenger safety. Security has been one of the most obvious applications as public officials and private home owners find more value in cameras that tell in real-time that something is wrong (Camio in home security, Intelliscape in city-wide security) than in cameras that just film and wait for people to check on them after incidents happen or even in expensive private security guards that are idle 99% of the time.
Probable scenarios:
Visual task automation is just starting and will see more impact in many industries through robotics in factories and warehouses, through devices in health diagnosing and monitoring, and even through robots that can perform simple services (i.e. waiting tables and assisting customers in retail stores). Robots and devices that link to the cloud or that have chips in them (like Qualcomm’s new machine learning chips and NVIDIA’s GPUs) will handle services that people previously did themselves. In the short term, automation is enabling workers to do things more effectively by working with them side by side. On the long term, reactions are split between hopes of total freedom from boring and arduous tasks for and fears from total economic meltdown from unemployment. Because business leaders may face dilemmas in automation decisions and policy makers may face societal backlash, there is a probability of an automation slowdown.
Investment & considerations:
As big companies start to invest in AI automation — seeing costs and revenues can be impacted — they open up a world of unlimited investment opportunities in startups offering the software and the hardware for it.
On one hand, a valuable startup aims to provide B2B services in a better way than people — for example autonomous cars that drive people around with less accidents (AImotive), smart algorithms that diagnose patients better than doctors (Zebra Medical Vision, Imagia Cybernetics, Imagen Technologies, PathAI) quality inspection in manufacturing (Sight machine), robots auditing inventory in retail (Simbe Robotics), devices judging damages in insurance claims (Tractable.ai, Kespry), drones that monitor oil pipelines and solar farms (SkyX, Sky-futures), camera devices that monitor agricultural crop (Prospera, Blue River Technology). Many of the B2B products promise to reduce errors, but investors need to be weary of false negatives and false positives as this will make or break customer adoption, especially in medical and quality-related applications.
On the other hand, consumer startups who are in the digital assistance space can become valuable by giving people access to services they weren’t able to get before, creating “abundance” for consumers (the Amazon Echo Look that acts as a personal fashion concierge, or the FridgeCam by Smarter which tracks the food in your refrigerator and gives you recipes) with digital assistance being part of the vision of many technology companies including Google.
The best AI solutions will be however the ones that create new kinds of value — on the consumer and business level — that were unimaginable before, such as spotting medical diseases before an actual doctor can or creating new kinds of insurance policies from newly collected data.
Theme 4: Recognizing behaviours and emotions to improve the customer experience
Driving factors:
In a similar way that human interaction is centered around emotions and body language, we can give apps and robots human understanding and expressiveness, thereby improving human-machine interaction and collaboration. After face identification and eye tracking, gesture recognition and emotion recognition are the next rising tide. We know that face identification plays a big role in solving not only the access but the trustworthiness and risk profiling challenge in digital interactions (Onfido), but combined with a bunch of other data (emotional, speech, gesture) it can say much more. Facial expression analysis can be coupled by speech and tone of voice analysis in what is called multimodal emotion recognition. Gesture recognition, on the other hand, is important today for the smart home and smart car market (Eyesight) and plays a large part in creating new ways for people to control technology by just moving the hands, for example.
Probable scenarios:
Human sensing technology can find its way into unlimited use cases as it will augment almost every digital offering when apps, objects and robots begin to deeply understand humans. With the (not so new) knowledge that emotions — not rationality — drive consumer decision-making, both branding and UX design are increasingly driven by an emotion-focused approach. Brands and publishers taking particular interest in emotional data because how consumers feel about the brand is a leading indicator to how much they consume. They are not only interested in catching attention with their content, they increasingly want to make a “connection” that leads to loyalty and sales conversion. They can also push different content for different moods using real-time emotion data. Realeyes, Affectiva and Crowdemotion are all targeting content publishers shows the size and imminence of these solutions and making marketing research insights more accurate and expanding the scale of the research to every user on a camera-enabled device.
Collecting behavioural and emotional intent data improves digital products. Using emotional recognition software like Affectiva’s Emotion SDK and API, developers can identify the current mood of their users and the reactions to their products in order to improve their service and iterate towards greater satisfaction. Video conferencing in business, recruitment processes and personal chats are being taken to a new level when the video software lets the parties know what the other person is feeling (Hirevue). nViso launched a risk profiling tool for financial services. In the smart home and smart environments market, recognizing people’s gestures and other things such as eye gaze can also lead to better interactions between people and devices. Retail stores, for example, can optimize the impact of their store experiences by continuously A/B testing emotional states of shoppers and behaviours inside their camera-loaded shops.
In healthcare, any researcher that is trying to make people “happier” and “healthier” will benefit. One solution to help patients at hospitals or even elderly and sick people in their homes is to install cameras everywhere, in a similar way to how Pointgrab track building occupants’ behaviours to enhance productivity and space design. Furthermore, care robots and virtual assistants being given expressive emotions to appear more considerate and friendly. While Siri will never have emotions herself, she can learn to mimic them if that makes the user talk to her more. “Humanizing robots” is what Affectiva talks about in their blog. Moreover, when mirrors and robots in our home can diagnose our health using facial expressions, they can alert us when something is going wrong and act accordingly.
Finally, the understanding of peoples’ emotions leads to replicating them in simulated online worlds. As we move into a world of digital avatars and immersive social experiences, our emotions will come with us (Mindmaze, Loom.ai).
Investment & considerations:
There is already M&A activity in the emotion recognition space because of the promise of the technology and because the market is set to be as big as $36B in 2021. Recent exits include Emotient to Apple, Innerscope to Nielsen, and Faciometrics to Facebook and despite their small amount size they are significant in telling the future. Gesture control seems like the next big opportunity, with companies like Eyesight and Ultrahaptics having already raised respectively $30.9M and $39.68M to bring about the next human-computer interfaces.
Sensory AI, while seeming risky seems very rewarding on the long-term as the technology makes its way into every device and application. Risks and challenges in this space are many and they include privacy, creepiness factors, and mistrust of technology as they develop expressive capabilities that are specific to humans — thus empathetic technology will take time to prove itself more beneficial than invasive.
While the future seems rosy with hundreds of potential applications, these solutions are still in early stages and have not reached consumers yet. To survive they either need to target specific business solutions or integrate themselves into other hardware. Emotion technology, for example, is itself bound to be commoditized on the long term and only those applying it alongside other sensory data input to solve a specific problem across sectors or those working on tools to increase usefulness of a set of third party apps will win.
Theme 5: Real world objects becoming hyperlinks to the internet
Driving factors:
When experts speak about the Internet of Things (IoT) and the convergence of the digital and the physical, they rarely talk of the ability of our smartphones to look at things through a camera and access related digital content or services and that’s mainly because those things need not be connected to the internet. When identified by vision software and viewed through a camera-enabled smartphone screen, products and objects in the real world can become tokens to anything on the internet. With the advent of accurate object recognition, people will prefer to point at something instead of searching for it on Google or downloading an app to access related services. The “camera will replace the keyboard”, as WIRED magazine repeats from Silicon Valley chatter. This trend is made clear by Google, Pinterest, Amazon and Ebay all going into visual search, with the two latest acquisitions being Moodstock and Corrigon (though for small amounts).
Probable scenarios:
The physical and the digital will start to connect here, through the camera — this is QR code enhanced. Once identified by the consumer’s camera, the product itself can lead to a service that enhances its consumption (Pointing your phone at a Chevy Camaro car tells you that it can be found at a nearby dealership and costs 15k). Similarly, smart product packaging can become a link to media content and social conversations around that product i.e. personalizing Pepsi cans will go beyond just writing names on them into a full digital experience. Just like songs identified by Shazam have links to apps and more information about the song (lyrics, videos etc.), the objects identified by the camera can have that too.
Some consumer use cases include:
· Point and Search (Google Lens)
· Point and understand (i.e. Blippar, Pinterest Lens, Foodvisor)
· Point and get entertained — mostly branded products linking to branded content, but also “social” content (i.e. This is a designer chair, watch the artist’s story behind it)
· Point and shop (i.e. Snap Fashion with different use cases like buying an item on a fashion runway by pointing at it, buying your friend’s jacket or buying an item displayed on the stores mannequin while walking by the shop).
· Point and invest (Dabbl)
· Point and make it come alive (QuiverVision, Zappar, Booksandmagic, Gamar)
· Point and get similar products (i.e. Pinterest visual search, ViSenze, Visii, Mirrorthatlook)
· Point and see if you need to take action (plant is dying, dog is unfed, battery is drained, heater is using too much power, food is about to expire)
· Point and learn. (i.e. Virtuali-tee by Curiscope)
· Point and remix/add your touch (i.e. First Snap Selfie filters, but in the future I would be able to change anything in the real world according to my personal taste and creativity)
· Point and make a copy (i.e. KeyMe)
· Point and compare (i.e. comparing two laptops I see at the mall by taking successive pictures of them)
· Point and get reviews (i.e. pointing at a bottle of wine at the supermarket and getting ratings and articles about it)
· Point and go back in time (i.e. could be to get my memories at this place I returned to, or historical facts/stories about a monument)
· Point and connect (i.e. get to know people and chat with people around you at a festival with facial recognition)
· Point and see who owns this (i.e. Pointing at a building while searching for an apartment to rent to get in touch with landlord/agencies)
· Point and diagnose (Skin Analytics)
· Point and get analytics (i.e. point at Metro Card and see 12.5 pounds left)
· Point and translate (Google Translate, Waygo)
Investment & considerations:
The first investment area here lies in APIs and libraries will help to develop object recognition. While general APIs like Clarifai and Google’s vision API exist, specific APIs are ones can build specialized libraries of pictures with very accurate identification at any case (fashion like Snap Fashion and Everybag, urban places like Fringefy, and car brands like Blippar’s promoted use case).
The second investment area is in apps built on top of these platforms, which follows from generic APIs into specific use cases for the technology — it is a nascent space that involves consumer adoption risks because consumers don’t understand yet that their smartphone camera is a gateway to information and services. Moreover, the app distribution strategy is still not clear here — the fact that mobile apps will all want to include camera experiences leads to the question of if someone will own the camera platform for all these apps (The Google/Snap/Facebook camera platforms as a place for third-party apps that can handle tasks like identifying a car brand to tell you more about its features) or if every camera application will still be a native app with the user owning multiple apps that identify specific products. The second option is looking more likely, but means that the app discovery problem will keep on existing as consumers don’t want to have a specific app for every real-world token. The alternative for new visual search apps is to somehow leverage other people’s audiences or find a way to acquire customers through existing infrastructures.
Theme 6: Media content is becoming shoppable
Driving factors:
As brands and publishers compete among each other in the zero-sum battle for attention, content marketing became a must. Nevertheless, we see report after report that big investments in content do not bring sales conversions and that the ROI from content is nearly impossible to measure. To counter these worries, there is a clearly big push by the technology giants towards shoppable content. YouTube released Shopping ads, a new advertising format that lets viewers click and buy products directly from videos. Facebook is experimenting with shoppable ad formats — while these are today still manually created by advertisers, we can expect that brands start making shoppable product placements inside content. Pinterest’s last 150M round to focus on shoppable pins and visual search to attract more advertising spend. Amazon increasingly making movies and magazine content part of their prime subscription might mean that one day all this content becomes shoppable. We are also seeing small interesting startups popping up such as Like2knowit which has partnered with Google in 2016 to bring “Shop the look” technology to influencers
Probable scenarios:
Understanding what objects are inside photos and videos makes searching for them with text or voice easier. Similarly, understanding what objects are inside magazine pictures or movies will also help users discover, consume, find similar items and shop better. By adding hotspots to photos and videos, shoppable media technology promises to remove friction for users who can now shop directly from the image or video without having to navigate to other websites thereby shortening the journey from discovery to purchase, which could lead to fast adoption by brands and e-tailers. Therefore, content creation becomes a more trackable investment and social platforms become a more effective advertising channel. Content producers and publishers can better monetize their creations and advertisers can start effectively measuring content ROI, a classic win-win situation driven by emerging technology. Consequently, new ecommerce players who are better content marketers than they are website designers can become the new top sellers. We may finally see F-Commerce become a real thing after years of debate because “shoppable content” is not far as full ecommerce from the main experience that Facebook and other social platforms are offering.
Investment & Considerations:
Many tools and platforms being created to make content on the web “shoppable” such as Clicktivated and Styla. We still don’t know what the optimal user interface here is but the prediction is that the tools that can create the experience with the least friction without interrupting the viewing experience will definitely outcompete others. In addition, product placements and the influencer space may become a bigger investment theme as Facebook and Youtube start tightening the grip around it and buying/partnering with more platforms that they do not want to build themselves (Youtube recently acquired Famebit). There will also be much value in building the infrastructure and networks around these shoppable media interactions among brands/content creators/influencers/consumers outside of Facebook and Pinterest.
Nevertheless, making content shoppable by itself is not a straightforward path to sales conversion — the content needs to be contextual, relevant and personalized to increase the probability of the sale. As more and more brands jump on the shoppable content bandwagon, there should be a general consensus in the industry on the balance between the traditional objective that is storytelling and the new objective that is selling. Going towards a world where every piece of content aims to sell something will remove the pleasure from media — which is why the UI may be the biggest challenge here. Lastly, although we know it is the future and there are a couple of interesting case studies out there, there needs to be much larger success stories on shoppable media for it to be really considered by big brand marketing teams — who will in turn need to figure out how to scale this.
Theme 7: AR Headsets promise more immersive experiences and higher productivity
Driving Factors:
Unlike Virtual Reality which is designed to disconnect users from the real world and place you in a dreamlike environment, Augmented Reality bring the real world to life in new ways — users will experience their city, their friends, their school and their work through an amplifying lens that makes everything more interesting and more fulfilling that adds computer generated content such as information and 3D imagery. Because reality is not separate from our understanding, there is no point of augmenting reality if it is not in service of augmenting our senses, our perception and our intelligence — that includes seeing things that are not really there but are useful for us in the context.
AR is expected to be a $50–80Billion in less than 5 years, tech giants Microsoft, Apple, Google Facebook and others have started racing each other to create headsets that make the AR experience as real and useful as possible. Oculus Research predicted that AR glasses would replace smartphones as the everyday computing gadget in the near future (5–10 years). Augmented reality headsets/glasses is currently in a platform war. Companies who previously made smartphones are literally taking everything from the smartphone and putting it into the headset, and they all predict the end of the smartphone. Other companies who missed on the smartphone era like Microsoft and Facebook are now in the game to take on AR. An additional driving factor is that there are new kinds of data that a headset can capture that a phone can’t — for example electromyography (Emteq), brain electrical data through direct links (Mindmaze), sight data showing where the user is looking through eye recognition (SMI) which makes them ideal personalization enablers.
Probable scenarios:
There are 50+ different headsets on the market or in production today, including Magic Leap’s stealth product which is funded with $ 1.4B. Most experts and investors think that we will not have consumer-ready glasses that do everything your phone does and more before 2020 and they will be rather expensive. While Hololens signaled a release data in 2019, there are different rumours about Apple either launching a full version in 2020 for around 2020, or maybe even going for a far less very basic version in 2018. The limitations and complications for do-all AR headsets are many: technical issues (FOV, frame rate, computing power, battery), design issues (devices should be light, little and good-looking, like a standard reading glass), social issues (wearing an AR glass should be accepted by all people), prices issues ($3000 is the current price of a Microsoft HoloLens). The balance between acceptable price and acceptable performance has not yet been reached. The prediction is that none emerge as winners and most of these will end up specializing. Some companies are launching cheap B2C specific use-case glasses (Snapchat will launch Spectacles 2.0, Zappar launched Zapbox) which helps hasten adoption and even manage consumer expectations.
On the other hand, many new companies are launching expensive B2B headsets (Daqri Smart Helmet is at $15000, Varjo aims to launch for $10,000). B2B headsets will grow faster than B2C not only because the margins are higher but also because people don’t have a problem with wearing clunky, unstylish headsets at work like with going out in the city. ABI Research projects that 60% of the AR market is going to industrial and commercial uses. While Google Glass Enterprise is launching headset solutions for factory workers, Meta aims to replace workdesk computers with headsets. Bloomberg Tech explains: “Imagine an office without computer monitors, cubicles or chairs. Everyone is wearing headsets that project their work in the form of 3D holograms. Meta, an augmented reality startup, is making that reality. Starting a few months ago, the company started ripping out everyone’s monitors and replacing them with AR headsets.”
Investment & considerations:
The large gaps in the technology first makes way for IP-based startups that can increase user engagement and make the experience more “real” for consumers. These are bound to become acquisition targets for headset manufacturers and technology providers such Facebook, and Apple. The next few years will be about filling the gaps in product design and performance to reach immersive stylish AR glasses (like the ones Facebook patented) and then lenses on the longer term. Startups in this space may be working on accurate positional tracking (Eonite), as well as on interactions like letting people use their hands inside AR (Leap Motion, Manomotion), letting them feel heat and cold (TEGway/Thermoreal), feel weight, feel taste etc.
On the B2B side, hardware and applications that clearly are solving problems such as in therapy and surgical intervention (Medical Realities, ProximieAR, Philips’ image-guided therapy), or in industrial solutions (Atheer) can more easily find their way into value creation. Information and waveguides layered on top of the world in a workspace (WaveOptics, ScopeAR’s WorkLink) promise to increase productivity by teaching and showing people what to do. Collaborative tasks will also be enhanced such as working together on a product (Bigscreen), viewing analytics and spreadsheets (Virtualitics), and even online meetings (Microsoft’s “Holoportation”). Much like AI applied to maps enabled anyone to become a taxi driver without previous knowledge, we may also see AR becoming an instruction/training tool that “uberizes” the workforce of many other industries and spawning new platforms like Uber. Other industries most likely to be in need of AR tools are entertainment (movie making and mobile gaming) commerce (“try before you buy”) and education (immersive courses and stories).
The direct-to-consumer side will need time to follow as headsets become less clunky, and we should note that there is always the risk that these headsets might never be fully adopted by consumers mainly because of social risks (Social rejection, digital fatigue) and technology risks (miniaturization challenges, poor UI). There is much uncertainty in the user interface and the digital fatigue that can arise from having HMDs on all the time and everything in the real world is screaming at users to “unlock” or interact with. Potential solutions exist such as AI-optimized contextual feed (similar to how Facebook’s newsfeed started optimizing what to show you when showing everything became impossible), but will take some time to materialize with because of it’s reliance on computer vision advances and IoT adoption.
Theme 8: Mobile AR is not as hyped as HMDs but is much more imminent
Driving factors:
The big race between major tech players is towards building stylish and untethered wearables that are not too expensive, yet mobile smartphone AR will come much before headsets and is expected to rise very soon as Apple releases the iPhone 8, which will be equipped with AR capabilities, following a release of a software tools called ARKit. The Verge explains “ARKit enables what Apple refers to as world tracking, which works through a technique called visual-inertial odometry (VIO). Using the iPhone or iPad’s camera and motion sensors, ARKit finds a bunch of points in the environment, then tracks them as you move the phone. It doesn’t create a 3D model of a space, but it can “pin” objects to one point, realistically changing the scale and perspective. It can also find flat surfaces, which is great for putting digital props on a floor or table”. Google also launched the ARCore SDK for Android developers and Vuforia has a similar SDK which is hardware agnostic. This is only to stay that developers now have great tools to start building applications for mobile AR devices.
Probable scenarios:
With the launch of the new iPhone this year, market observers forecast that around a quarter billion people will be walking around with AR capabilities in their camera phone in the next year, and it will be a short time before they notice the benefits around entertainment, information and retail. Mobile AR will be very basic at first and will revolve mostly around placing objects like furniture and home décor in homes with (with retail stores IKEA, Wayfair and visualization tools (Sayduck, Augment), around consumers trying on makeup, accessories and clothes to see how they look (Metail, Inkhunter, Sephora), information on real world objects and points of interest, outdoor location-based games (Ingress & Pokemon Go by Niantic, and crowdfunded game Father.io), indoor entertainment and tricks (demos on @appleARworld and @madewithARkit on twitter). Even movie directors such as Peter Jackson’s have opened AR studios to experiment with the technology. As the space evolves we expect to begin to see hundreds of interesting applications that we cannot even imagine today, in the same way that millions of apps have been created after mobile technology became mainstream with the iPhone.
Investment & considerations:
Although mobile AR is the pre-requisite for AR headsets, it is still very early in the lifecycle of the technology. Only a few phones on the market will have depth sensing and AR capabilities in the next year or two. It will be interesting to wait and see the impact of ARkit and the consumer adoption of AR-enhanced apps. First movers don’t necessarily have advantages because the user experience might not be that great. The really interesting new native apps might only start appearing towards mid-2018, although existing apps like Wayfair with a big number of users can start adding AR experiences. In terms of companies that will take advantage of existings platforms like Facebook’s AR camera, it would be interesting to see if Facebook would share revenues with developers on their platforms because this means that the developers can monetize existing audiences without spending much marketing money. For now, investing in tools that enable developers to create more engaging experiences is the better strategy, with startups such as ManoMotion which offer an SDK for mobile AR developers, Zappar which offer brands the tools to create simple AR experiences for their customers and other startups that may compete with ARkit and offer vision/positioning tools but are hardware-agnostic. Lastly, consumer mobile AR experiences can be coupled with physical objects / specialized toys — such as Reach Robotics’s MekaMon, Merge Labs’ Merge Cube, and Yangshu Wenhua’s NeoBear .
Theme 9: Projections and volumetric holograms are the alternate AR experience
Driving factors:
While AR headsets like Hololens create content illusions on a screen, projections and 3D volumetric display take the objects outside the screen and form a real world representation. Holograms coming out from devices mean that you can project anything in 3D into the real world from objects such as information on a board, chair you want to buy to a content such as football game inside your living room. On the Gartner Hype Cycle, volumetric display is at the end of its innovation trigger. There been many advances in the last few years and research such as the one coming out of KAIST are promising much better results today (the approach is said to improve projections by 2600x). The camera company RED very recently announced the first holographic phone Hydrogen that will launch on the market in 2018, in partnership with Leia 3D.
Probable scenarios:
The concept of taking apps and content outside the screen shatters many limitations. The move from 2D to 3D imaging is massive, especially without the need for headsets but just the naked eye. On the consumer side, since the digital sphere is easier to design efforts will be focused on making our reality a more enjoyable place not through the production of more real “stuff”, but through the creation of more interactive content that can be layered on top of the real world. On the business application side, we can imagine training in surgery and product design as major applications. 3D holograms can potentially be coupled with motion sensors to trigger interactive experiences from devices that have neither screens nor buttons. Futurist Micheal Mascioni talks about ambient interactivity for public spaces as a probable trend in out of home immersive experiences. Kevin Kelly again talks about “screening” — the presence of interactive interfaces all around us — as another inevitable trend, and the logical progression is from screens to projections and holograms due to the increased level of immersiveness.
Investment & considerations:
Startups Lampix and Lightform project light on flat surfaces to create digital experiences outside the screen. Startups working on volumetric holography technology are few as this market is nascent and include Leia 3D, Realview Imaging and Voxon Photonics. Partnerships like RED & Leia 3D will start multiplying when consumer and businesses are more aware of the potential in the technology applied. On the consumer side, startup Looking Glass has raised a Series A to develop personal hologram machines and has 4 products. Time Warner has invested in 8i to bring celebrity holograms to life as the potential to disrupt the entertainment industry with more immersive experiences became clear. On the commercial side, the time to market for commercial applications though is unclear as depends on technology advancement pace. Holograms, unlike headsets, and mobile AR is not something that big tech are publicly pursuing today — although Apple has patented a hologram display system with gesture input in 2014, as reported by TechCrunch. RED and Leia’s new phone strategy should be watched carefully but might face the same difficulties as Headsets due to the price tag of $1200 that is not supported by a rich content ecosystem. Volumetric display is today the more risk-infused contrarian approach to Augmented Reality and for volumetric technology startups to survive until market traction they have to combine state of the art technology with corporate partnerships.
Theme 10: A rich content ecosystem is a prerequisite for consumer AR mass adoption
Driving Factors:
After three years of hype, VR has not quite reached its potential because of the lack of content. When the AR headsets are released there needs to be enough apps and content that make people want to use it. Luckily for Augmented reality, 3D content creation that has already started within the VR ecosystem can spillover into AR. Content creation tools have emerged to help designers and developers bring 3D content to life (Unity3D, Aurasma, Wikitude). Real world objects are being brought to the online world as 3D content through tools (Metail, Uncorporeal Systems, Trillenium, Hexa3D.io, Matterport), even humans (8i, Loom.ai, Wolf3D). Microsoft has also released Microsoft Paint 3D to support Hololens content creativity and democratize 3D design. Startups that are operating in social media content creation tools include Spektral and Tipit.tv that put a green screen behind people so that any background environment can be added. They also include Selerio in UK and Uru in NYC who are building a tool for content creators and influencers to add branded products in their videos i.e. an adventure focused influencer can remix his real environment and add Mountain Dew products or Patagonia themed equipment. While Mirriad is a company that does post-production video changes, Selerio aim to do it in real-time broadcasts.
Probable scenarios:
Filters and faces are just the beginning of a transformation in social media and messaging. With the launch of Facebook camera platform, Zuckerberg told the audience about this along with his mission to “help people express anything”. After Facebook copied Snapchat filters they went one step beyond and launched AR studio for developers. This means that he will want to make content creation easier for the designers, influencers and publishers on the platform. The apps Mirage and Skignz, for example, are new apps aimed towards social interactions as people can leave digital content like drawings and texts in the real world — in the sky or on objects — that others can find based on location. Evolving this concept into instant messaging or live broadcasts and the user can think “I’d like to give you a flower” and that could happen with a digital representation of a flower. We can also foresee this as an interesting trend in education and training as “teachers” can explain things by making them appear in their videos in 3D just like they would writing on a board.
Investment & considerations:
On the consumer side of the market, it is hard to predict what will happen in this market as this is Facebook’s and Snap’s turf. In August 2017, Facebook acquired the German startup Fayteq and even Google acquired AImatter/Fabby which both allow video modifications with computer vision technology. These acquisitions show that Facebook and Google are looking for new deep-tech IP and talent to create new opportunities in the AR market. In addition, Zuckerberg’s “help people express anything” mission is a great indicator for the the future of social apps inside and outside of Facebook’s efforts. This could mean the emergence of more design tools that enable professional end-users to create content, but then those will be followed by tools that would make this creation process frictionless for consumers — think of how Instagram democratized Photoshop-like image editing. Hopefully, we can also see new independent AR media platforms emerging alongside Facebook which included Augmented Reality in its 10-year plan and are looking to monopolize the content tools to win the AR war early — let’s not forget that publishers and advertisers are looking to break out of the monopoly that the tech giants have on mobile media content and
Theme 11: Advertising and commerce set to follow attention into AR
Driving factors:
Wherever people are turning their attention to, advertising follows. Brands have been creating native advertising content for years inside media platforms. And if people start spending most of their day time in AR as Tim Cook predicts, if the real estate inside Augmented Reality is bigger than inside a mobile app, and if AR is more interactive than a web or mobile app — then AR is definitely the next big transformation for advertising. It seems that every website for studios helping brands create immersive experiences in VR and AR promises more engagement and more conversion. Snap has expanded its sponsored face filters into sponsored world lenses and geo-filters so that advertisers can create AR content beyond the face. Blippar has launched Augmented reality digital placements (ADRPs) which open interactive AR experiences from rich media banners. Snatch is offering publishers a location-based treasure hunt game and has been gaining good traction. Zappar offers Zapworks as an AR experience creator for brands. Few others startups offer advertising platforms and AR ad formats (MyBrana, Skignz, Vyking, Vertebrae). On one hand, AR developers and content creators want to monetize. On the other hand, brands will need a platform for native AR advertising where they can seamlessly insert their branded products and create their own immersive experiences. The ecosystem is already forming well.
Probable scenarios:
Just like shoppable content is becoming a thing in videos we can expect the same trend where the lines between AR apps, advertising and commerce blur. The hyped promise of AR applied in retail is that it allows customers to discover and visualize products in their home. Speaking about their sponsored lenses, Snapchat’s head of commerce recently said “But you can imagine this being able to show a new BMW in your drive way or a Burberry handbag on your desk. This is where this will go”. Flipping this over, it won’t be long before a product in real-life seen through a mobile screen opens an AR portal into both an interactive test and a virtual store. Yihaodian, a Chinese grocery store opened up “new locations” in parking lots and parks with virtual aisles that people can walk and shop through using just their phones. By 2020, Gartner estimates that 100 million consumers will be shopping in augmented reality. Advertising is a fast moving market as there are innovations every year and thousands of Adtech companies spurring, yet we know that the general direction in the future is that advertisements won’t feel like advertisements, and that’s a big step forward from today’s mainstream interruptive pre-roll videos and banner ads. We will probably see that immersive advertising will increase the level of creativity and gamification needed in marketing and sales, and take “rich media formats” to a whole new level. AR advertising analytics will be much more thorough because using cameras and headsets enables the tracking of interactions and engagement metrics involving body language, eye tracking, emotion and speech.
Investment & considerations:
Grand View Research estimates that $30 billion will be invested in retail AR applications by 2020. There not yet any known successful case studies showing that consumers will adopt, but we do know that Snap has acquired the Israeli startup Cimagine which has technology for “try before you buy” and that it also has 8 patents related to object recognition and AR. We expect the space will quickly get crowded with all brands, old retailers and new apps coming in because it’s expected to increase commerce conversion rates directly. Amazon has already started to experiment with things like visualizing TVs in living rooms before buying to reduce returns and save costs.
On the advertising side, startups like Selerio and Uru who are aiming for branded video modifications have the potential to become the AR video advertising platforms of the future because they make it easy for content creators and brands to collaborate. We also believe that brands can get really excited about inserting their products into targeted video content on Youtube, Facebook and any video platform without the added friction of getting involved in the production process and dealing directly with content creators. Our preferred thesis in ecommerce is to invest in applications that allow for frictionless consumption, thereby reducing the efforts needed for all parties involved to zero and bringing in ultimate convenience shopping all the way from discovery to purchase.
Companies met in the research process: VR/AR association London, Medical Realities, Realities Centre, nViso, RealEyes, Calipsa, Manomotion,Selerio, Zappar, Emteq, Peekabu Studios, Booksandmagic, Crowdemotion, Facebook
Please contact me on jeljamous.mba2018@london.edu so that I send you an Excel list of all 153 companies mentioned in this report with details on what they do, location, funding amounts and investors…and let’s talk about the 30+ other promising startups in the market we’re analyzing at Aurelius
