The Evolution of Augmented Reality (AR) in China
Interview with Alibaba AR Lead Jiang Jiayi
Augmented reality (AR) continues to be a hot area in tech these days, with potential to build the future platform for computing and spawn a hundred-billion-dollar market. In China, this yields a highly competitive environment with all tech companies, ranging from giants to start-ups, battling for a share in it.
Today, we sit down with Dr. Jiayi Jiang (蒋佳忆), a top AR expert at Alibaba, to elaborate on the development of augmented reality technologies and use cases in China. Dr. Jiang shares with us his story and helps us understand the following:
- What does the AR industry look like in China?
- What is Alibaba’s AR strategy
- What Alibaba’s vision for the next generation of Retail?
- What are AR related strategies for other Chinese tech giants, Tencent and Baidu?
- What are the start-ups doing, what are some other applications?
- What’s the difference between the Chinese and the U.S. markets?
Interview by Tyler Xie, Shaolong Lin, Carina Zhang, and Adam Bao
Dr. Jiang’s Start with AR
Tyler: Dr. Jiang, thanks for joining us today. First, can you share your experience with AR?
Dr. Jiang: I started to work on AR at the end of 2015. There are only a few engineers or subject experts that come from a few labs and get hired by a large tech companies. And those companies also hire engineers and experts away from each other. That’s true in both China and the US.
Then there was this well-known start-up, HiScene. They are now top 3 in AR in China and just raised $15 million in series B. I joined them when they just started. For more than a year of my time with HiScene, my job focused on developing fundamental algorithms, especially SLAM (Simultaneous Localization and Mapping), which is a core algorithm in AR.
I learned a few things during my algorithms development work at HiScene. First, SLAM algorithms heavily rely on the control of the hardware. That’s why when Apple released its AR Kit earlier this year, the industry was discussing that it’s difficult for other companies to compete. The reason is that Apple has excellent control on the core hardware. They know what capabilities of CPU, GPU, gyroscope and other hardware they want to support AR. And Apple has the capability to control the entire production for those hardware. That’s why they can offer the best AR SDK (software developer kit) in the market.
Also, almost all major SDK suppliers, like Metaio which was acquired by Apple, and Cimagine which was acquired by Snap, and many others, have been focusing on mobile SDKs since last year. So, the trend is the development of the fundamental algorithms is slowing down, as more advanced hardware and chips allow for better AR performance.
AR at Alibaba: A New Form of Customer Engagement
Dr Jiang: Then Alibaba reached me out to me with their business needs. Alibaba is an ambitious and open-minded company, and they don’t have to use their own algorithm. If they like some company’s algorithms and think what they make is unique and irreplaceable, they can simply buy it. So, I came to Alibaba and started to develop AR-related applications. Since I first came here and up until now, I have been involved in basically all the businesses and services of Tmall. I led some of them and participated in others. Alibaba Group still does not have as clear of an idea on how to leverage AR in core businesses compared to big data and artificial intelligence.
For now, very few companies have provided an AR application with good experience and commercialization. Pokémon GO is one of those, but it is not their algorithm that is impressive. Rather, it is their good IP and innovative way of using it. Alibaba then followed up with Catch Yoo!, which has been the primary AR application since last year’s Ali’ Double 11 Shopping Day, for the purpose of engaging customers. Other than that, I don’t see other teams announcing their decision to do something with AR. AR is more like something new to experience or a new way of interaction.
My job is to make sure that Alibaba neither gets outdone by competitors nor falls behind when it comes to any business model, strategy or fundamental capabilities of AR. This is not Alibaba’s mandate by the way, rather this is how I interpret it.
My priority now is to follow the development of the entire industry closely, including the changes in fundamental capabilities like SDKs and the broader ecosystem. I will contact good content providers and support them with the fundamental capability they want. But we won’t let those independent software vendors (ISVs) develop their own super app. ISVs can only provide content based on different kinds of frames and approaches. For example, editing the content/model of interaction using our editor or based on the web but not writing the whole algorithms.
We also contact hardware companies including Huawei and Xiaomi in China, and some modular providers such as Lumus or uSens in Israel. These modules can give mobile devices the capability to support AR. Once the Apple AR Kit, I predict, achieves a coverage of 20%-30% of mobile users by the end of the year, Alibaba will put more resources on AR.
Tyler: Can you give us some examples of applications? Based on what I know, Taobao had an application called Buy+ for a very short time last year. What is that?
Dr Jiang: I did not participate in this project but I can talk about this from the technology side. Buy+ aimed to let customers buy things in virtual reality (VR). They even readied the payment process through the headset. But there were still a few problems they needed to face: first, most customers don’t have an AR/ VR headset. Even if some customers have it, the whole experience is not good enough. Traditional online shopping is way more efficient than VR shopping. Another thing was just how expensive it was to make the virtual store. So, this was more for PR purposes.
Alibaba is willing to try those new technologies to see how they can be leveraged for our core businesses. We are willing to research and test all the fancy technologies available in the market, even if they are only for marketing purposes rather than solving a real problem. But we do take cost into consideration.
Tyler: How about AR Livestream? Is this another such technology or use case?
Dr Jiang: Alibaba brought up the AR livestream for the Taobao Maker Festival. It’s quite simple: customers standing in the front of a big screen can interact with others in the screen. It’s more like a marketing strategy for attracting more customers, which is one of the most important use cases for AR.
You may find that, in those shopping centers, more restaurants and bubble tea shops are opening than other retail stores. This means fewer shoppers choose to go to physical stores. So those stores need a way to attract shoppers to get in. AR is a good gimmick!
Once customer get in the store, AR provides a good UI for customers interacting with goods. You scan the product to see some AR effects to get more information by yourselves rather than listening to the staff. And the simple AR effects and contents are preloaded and easy to manage.
What is “New Retail?”
Tyler: Alibaba recently brought up this concept of “New Retail”. How does that relate to AR?
Dr Jiang: Although AR the technology itself doesn’t associate with New Retail, it can be handy in the context of New Retail. Alibaba’s New Retail emphasizes connecting online and offline retail in three aspects: goods, members and services. Connecting goods means the goods customers see online and in the physical stores need to be the same. Connecting members means that Alibaba needs to be able to serve and manage customers, or members online and offline seamlessly. Connecting services means that the services different customers received online and in the physical stores need to be in the same system.
The services here are very broad. For example, one of the services is supply chain management. Alibaba-developed Cainiao Logistics is aiming to control the whole supply chain of e-commerce both online and offline. Further, Alibaba may consolidate the ERP and CRM systems for small online businesses to provide a system of services for both customer and business sides.
However, in my opinion, there are more and more channels for customers to buy stuff. For example, besides Alibaba’s Tmall and Taobao, there are JD.com, Yihaodian.com, food delivery apps and Wechat’s ecommerce. And there will be more. So that’s what worries Alibaba. Because if more people are buying things outside of Alibaba, their purchase data are not feeding to Alibaba. Alibaba wants to know what and how people are buying stuff, even not from its own ecommerce stores. So that’s the reason behind this New Retail.
In this sense, AR is a good gimmick to attract customers online to offline and back, and then allow them to receive online services from the physical stores.
Tencent Builds Brands While Baidu Builds a Platform
Tyler: What are Alibaba’s competitors Baidu and Tencent working on in the field of AR?
Dr. Jiang: Besides BAT (Baidu, Alibaba, Tencent), second-tier companies including Xiaomi, NetEase, and JD.com as well as Toutiao, Didi Chuxing, Meituan, and a number of technical companies such as HiScene and Sight+ that focus on SDKs, have produced some results in AR. The one with the largest move is Alibaba’s Buy+, which I mentioned earlier.
During the Olympic Games last year, Tencent collaborated with HiScene to design a virtual torch relay for QQ. A large amount of money was spent on celebrity endorsements, and the activity itself did end up having some influence. But as far as I see, the only access to AR in Tencent’s product system is the one located in the “scan” tab on QQ’s homepage.
In terms of AR, Tencent has positioned itself similarly to Alibaba: both focus on interactive marketing. QQ Mobile, with 800 to 900 million registered users, around half of whom are active daily, allows users to scan all kinds of things. Starting last year, I know they collaborated with Pepsi, as well as with movies such as The Fate of the Furious. Moreover, WeChat’s embedded web browser has an X5 core, which provides some capabilities in AR. I would say that WeChat is technically ready to embrace AR, but there have not been any concrete moves.
Tencent also created an AR lab, and has recruited people to develop some underlying technologies. I, however, do not think favorably of any lab that revolves around AR, as it is already too late for that. In particular, software companies will not be able to develop a universal SDK, given that even Google is still struggling with the underlying technologies. Chinese companies may well rely on their own SDKs, but these are bound to be replaced by other SDKs in the future; this is an ending that even Alibaba might not be able to avoid.
Baidu has an AR lab as well, which employed, in its prime, over 50 people. Since Baidu does not have a brick and mortar business as large as that of Tencent and Alibaba, it brands itself mainly as a technical company ambitious to build an open platform and system, which it hopes other companies will turn to. I think that in terms of underlying technologies, Baidu might be slightly better than other companies, but even so it is by no means incredibly advanced. I am not familiar with the companies it is partnering with.
AR Verticals: Education, 3D Content and Apparel
Tyler: So, besides BAT and HiScene, are there other players or start-up entering the market?
Dr Jiang: Of course. One of the companies I know is NeoBear (小熊尼奥), which made a lot of money last year. They are working on early childhood education in AR, which I think is a market that has fewer barriers to entry. For instance, it doesn’t require high quality effects — children feel happy as long as they see some cartoon characters jumping or standing on the table. They are not like us who are used to playing those 3D video games on PC or mobile phone. In this sense, children are a good target user group. And of course, it’s parents who are paying. Parents are willing to pay as long as their children are happy. NeoBear also produced an AR device called Magnifier NEC, which is a AR featured magnifier.
Recently I came across a start-up called Owlii (猫头鹰视界) which received funding from Shunwei Capital. Owlii is an AR content provider, which I think has the potential to be a successful one. They are one of two companies I know who have the capability to develop high quality AR content. The other is Plex, which received funding from Alibaba. They focus on light field movie special effects.
These AR content providers have good opportunities. Big companies actually can’t afford to provide AR contents, because their customer bases are too large and each customer has different requirements. Those big companies can’t afford to focus on individual customers. But if AR/VR is the next generation of UI or even develops into its own computing platform, there will be massive demand for 3D content. So those who can provide 3D content quickly and at a low cost will have good chance to succeed.
Another start-up is Haomaiyi(好买衣), whose core competency is 3D apparel content modeling. Most companies can model a few 3D clothing items per day, at a cost of about $200 per item. Haomaiyi, however can produce 500–1000 items of 3D clothing per day, putting the cost down to $10 per item. With this content modeling capability, Haomaiyi is almost unbeatable. There is no one in the market who can do that, not even Alibaba.
China vs. the U.S.
Tyler: Can you talk a little more about the difference between the Chinese and U.S. markets?
Dr. Jiang: I haven’t done any projects in companies overseas, so I can only talk about what I know from my circle. It seems that big names like Facebook and Google are using AR as a tool for generating more engagement and info from users, since they have good capabilities in the fundamental algorithms and device control. Taking Google Lens as an example, Google is using AR to make your camera another search gate for generating more searches. You can search what you want to know through a camera. And if this AR camera is on the back of a phone, it’s a camera for exploring the world; if it’s a front-facing camera, it’s exploring the users — getting more information from the user and generating more accurate advertising, like what Snap is doing.
Microsoft is focusing more on creating tools for AR content providers and other businesses, for example, like HoloLens. Other tech giants like Google, Facebook and Snap are using AR more like a gimmick to generating more users and engagement. This is different to China, where more and more companies are producing AR contents as we talked about earlier.
Tyler: Very insightful, thanks so much for sharing Dr. Jiang!
The opinions expressed in this article are Dr Jiang’s own and do not reflect the views of Alibaba
