Amazon Patents Provide Clues to New Home Robot
The robotics community is abuzz with speculation following reports that Amazon’s R&D arm Lab126 is hiring senior researchers for a home robot-producing project codenamed “Vesta.” The company is also said to be recruiting employees who will test the robot in their homes later this year.
No one knows if Amazon envisions an enhanced roomba or a precocious R2D2 V2.0. Company Founder and CEO Jeff Bezos was recently spotted at a tech conference walking Spotmini, the Boston Dynamics robotic dog known for autonomously opening doors: Might the bot take the form of a pet?
Conversational skills
One thing is certain: Amazon will cram Alexa into the robot. The company’s star virtual assistant has evolved into a know-it-all that can play music and audiobooks, compile to-do lists, set alarms, stream podcasts, and provide real-time weather, traffic, news and other personalized information. Although most people only use their virtual assistants for music streaming, controlling smart lights, timers and weather forecasts, according to an IFTTT survey, Alexa is backed by so much cutting-edge AI tech that she’s become too smart for just a smart speaker.
With Alexa as the brains, the bot may be more humanoid, both in physical form and in personalization and communication scope and style. Sarah Osentoski, co-founder of the robotics company Mayfield that makes companion robot Kuri, told Wired, “When you have something that’s talking to you and that’s driving around your house you start to expect a lot more. You start to expect the intelligence of a 3-year-old or a 5-year-old.”
That could be why Amazon is scrambling to update Alexa with more powerful capabilities. At the recent World Wide Web Conference in Lyon, France, the Head of the Alexa Brain group Ruhi Sarikaya said Alexa will soon launch three new features: an enhanced memory system that remembers everything users say, a more natural conversational system, and automatic activation of over 40,000 third-party skills.
Facial recognition
To recognize its owner and family members, navigate rooms and hallways and so on, Amazon’s new robot will certainly require robust computer vision skills. In 2016, Amazon acquired Orbeus, a Silicon Valley startup that specializes in image recognition. Amazon hired nearly all of Orbeus’ technical staff, and picked up its facial recognition system and scene recognition system patents.
Leveraging facial recognition, the robot will be able to capture images and record videos automatically. Amazon has patented approaches for creating high quality images with less blur and noise. The bot for example might be put in charge of photographing a birthday party, or compiling family photo and video albums.
The new robot might also employ multimodal dialogues to better watch, listen, speak, and physically interact with humans. Dr. Zhou Yu, an assistant professor at University of California in Davis, told Synced she is enabling intelligent assistants such as Siri and Alexa to recognize actions such as facial expressions and give appropriate responses. Since 2016, Amazon has supported her research with US$100,000 in annual funding.
Locate its owner
The Amazon home bot will be mobile, and navigate autonomously with cameras. It will need to recognize and track its user or users in the environment. Some of this tech can be borrowed from the Echo Smart Speaker, which uses real time sound data to determine a user’s physical position. Amazon’s robot could push that technique by incorporating dynamic visual data.
Lab126 has been granted several patents for localization techniques which can determine where a user is. One patent describes a system that utilizes a sound location technique to estimate an audio-based sound source position, and then pinpoints the user’s position from analysis of optical images or depth maps generated by multiple sensors, such as LiDar and structured light.
Although the patent was originally filed for a mysterious Lab126 project — rumoured to be a since-abandoned in-home augmented reality entertainment system — Lab126 is likely to apply this technique in the home robot.
Gesture recognition
While Amazon has prioritized voice commands in human-machine interfaces, their home bot will have to go beyond that, and watch for and respond to physical cues from the user, such as hand gestures.
Amazon has already been granted patents for hand signal detection. One is for a room computing system that uses sensors to detect and respond to hand poses. To detect a hand pose, an observed pose is compared to a hand pose dataset. When you wave goodbye to your Amazon robot as you head to work in the morning, will it wave back? It will if you want it to.
In any case the home bot will understand you are leaving, and will get on with its day, which may involve assigned tasks such as security monitoring. The new Amazon Key service allows in-home delivery and secure home access for guests. An interior security camera connects with Amazon Key, and it would seem natural to apply that task to the robot.
Go outside to run errands
Autonomous vehicles are a heated innovation area attracting interest from tech companies like Amazon, which this year patented an autonomous ground vehicle (AGV) that can leave its home, pick up a package from a depot and bring it back.
Will the Amazon robot have this capability? It is possible. Tye Michael Brady and Ethan Zane Evans, the researchers behind the technique, point out “the AGVs may be owned by individual users and/or may service a group of users in a given area (e.g. in an apartment building, neighborhood, etc.).” When the robot knows a delivery truck is approaching, it will navigate autonomously to meet the truck at the pickup point.
Your AGV bot could even meet up with other neighbourhood AGVs to return a rake or borrow a cup of sugar.
The new Amazon home bot will likely be a wheeled, mobile, Alexa-based robot with an array of cameras and possibly other sensors. Will it have arms and suction grippers to do household chores? Possible, but Amazon seems to be aiming for multiple capabilities across a wide range of functions, not a housekeeper like “Andrew” in the film Bicentennial Man. The potential is huge and the possibilities endless.
Only time will tell whether Bezos and Lab126’s creation will be a rolling Echo or a game-changing, “wow” product.
* * *
Journalist: Tony Peng| Editor: Michael Sarazen
* * *
Subscribe here to get insightful tech news, reviews and analysis!
* * *
The ATEC Artificial Intelligence Competition is a fintech algorithm competition hosted by Ant Financial for top global data algorithm developers. It focuses on highly-important industry-level fintech issues, provides a prize pool worth millions. Register now!
