Google I/O News: Android, Assistant, Actions, Home, TensorFlow, and Augmented Reality

The 10 most exciting developments so far: Actions on Assistant, Calls from Home, Proactive, Visual Responses, AutoML

It’s an exciting day for Google developers, especially for those who have been working on Actions on Google, Assistant, and Tensorflow. Here are the top 10 most interesting announcement so far if you’re working on speech recognition, chatbots, conversational apps, intelligent assistants or agents, or personal assistant applications.

1. There are now 2 Billion active Android devices

That’s phones, cars, watches, tablets, and other devices. That’s a large ecosystem for developers to interact with users. In fact, Audi and Volvo announced today that Assistant will be running in their cars. That adds an even larger users base.

2. Actions now part of Assistant

If you’ve been developing Actions on Google to create interactive experiences for Google Home, they are now available in Google Assistant; that’s 100M more users starting today. In addition, for users on Android (and now iOS devices), developers can create interactive experiences that merge speech input as well as images, video, and text. This is a truly integrated multi-modal platform that now has user scale.

3. Hands free calling on Home

Amazon enabled calling between Alexa devices and the Alexa app a few days ago. However, Google announced today that you will be able to actually make calls from Google Home using your own phone number. That means Google Home essentially becomes a fancy bluetooth speaker. We’ll have to wait a few months to see how this works.

4. Identity and Transactions on Assistant

Assistant will now pass your identity easily to Assistant applications and allow you to use Google Pay to make purchases. The demo showed the process of using voice and visuals in an assistant application to pass identity, and payment information to order a sandwich and drink. It was simple and seamless and showed off voice and visual input working very well together.

5. Proactive Assistant

Google will be (selecting) enabling proactive alerts on Google Home (and I assume Assistant) to allow select applications to push alerts to the device. It’s not clear if this will be available to 3rd party developers (yet) via user permission, but Google will obviously have to tightly manage this.

6. Visual Responses from Home (and Assistant?)

Casting visuals from your Actions or Assistant application to Chromecast devices is here. “Hey Google, show me my calendar today on the TV” will bring up your calendar. This adds news and exciting possibilities for creating interactive applications that you can talk to AND see. I can immediately imagine interactive games or hearing and seeing recipes while cooking.

7. TPUs on Google Cloud

Long rumored, Google has launched their Tensorflow Processing Unit (TPU) as an option for Google Cloud users. Previously, these were only available to Google. It’s unknown how these compare to Nvidias latest product, but you can be sure it’s going to be fast for Tensorflow jobs.

8. TensorFlow Light

TensorFlow Light leverages a new neural network API to tie directly into silicon resources. The SDK will be available later this year. The idea is that you can run some AI jobs on the device, not only in the cloud.

9. AutoML

This is a very interesting development for those of us already working with neural nets. AutoML helps you select the best neural net model and structure by running billions of simulations to self-select the best performing model and layers. Not many details were provided, but if you’ve spent any time trying to optimize the layers and structure of a neural net, this sounds fantastic. This also moves us toward future AI technologies where AI is training AI.

10. Augmented Reality in Google Maps

Google has added a feature in Google Maps that interacts with the camera on your device. You can simply show Google Maps what you’re looking at, and it will recognize the business or location and give you contextual options. For example, you could show Maps the sign over a venue for a band on a specific date and it might give you options to listen to the band’s songs or buy a ticket for the show. This is powerful and exciting.

Like what you read? Give Jonathan Eisenzopf a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.