Dazbo’s Google Cloud Next ’24 Recap: Keynote

Published in

Google Cloud - Community

9 min readApr 16, 2024

Shall I? Shan’t I?

It’s been a couple of days since Google Cloud Next ’24 wrapped up, and I’ve seen recaps appear on Medium already. So I ask myself: “Should I bother this year?”

I’ve decided “Yes” for two reasons…

I’ve been doing recaps of these events for a few years, so I’d hate to break my streak! (Check out here, and here.)
I find writing stuff down helps me learn and remember. So even if no one else finds this useful, I will!

This year’s Google Cloud Next was in Las Vegas. Alas, this is the Next in the last few that I haven’t been able to attend in person. 😭 So, this wrap-up is based purely on watching the virtual content. And in case you weren’t aware, you can view all the recorded sessions at cloud.withgoogle.com/next.

Summary of Announcements

I’ll update this list as a view more sessions.

New investments in sub-sea cabling and data centres.
AI Hypercomputer: A3 Mega VMs, powered by NVIDIA H100 Tensorcore GPUs. Twice as powerful has the previous iteration.
AI Hypercomputer: GA of TPU v5p. Google’s most powerful TPU yet. These have 4x the compute capacity of the previous generation of TPUs.
Preview: Hyperdisk ML — next generation block storage optimised for AI workloads.
Vertex AI on GDC.
GKE Enterprise support for GDC.
AI Model support (including Gemma and Llama) on GDC.
Preview: Google Axion. A custom ARM-based CPU. Claims 50% better performance and 60% more energy efficient than comparable current-gen x86 VMs! Google are migrating many services to Axiom.
Intel 5th Gen Xeon processors.
Public preview: Gemini AI 1.5 Pro in Vertex AI. Google’s multimodal foundational model. It can parse 1m tokens of information!
Gemini AI 1.5 Pro now integrated with Gemini Code Assist.
Supervised tuning for Gemini models.
Preview: Gemini Cloud Assist, which helps with the entire development lifecycle, including design and optimisation.
Public preview: Grounding of Gemini models with Google Search! This significantly reduces hallucination.
Vertex AI Agent Builder: rapidly speed up the creation of multi-modal AI agents.
Google Vids will be released to Workspace labs in June. This is an AI-powered collaborative video creation app, as part of Workspace.
Imagen 2.0 is now GA in Vertex AI. Google’s most advanced text-to-image model.
Public preview: Text-to-Live Image. This creates animated video-like images from a text prompt.
Public preview: Gemini in Looker.
Public preview: Gemini in Threat Intelligence. Tap into Mandiant’s frontline threat intelligence using using natural language prompts.
Public preview: Gemini in Security Operations. Summarise and explain findings, recommend next steps, and even write and execute remediation playbooks.
Public preview: Gemini in Security Command Centre. Evaluate security posture, and summarise potential attack paths and risks.

The Irony Isn’t Wasted On Me

I’ve watched the keynote, and I’m summarising it here. Manually. Without AI.

Opening Keynote: The New Way to Cloud

You can see the full keynote here.

Keynote Quick Thoughts

It’s all about AI. Shocking.
The biggest announcements are around Gen AI capabilities.
I think the keynote mentioned AI agents 1,806,402 times. (Okay, I’m exaggerating slightly.)

Introduction

Google are at the forefront of the AI platform shift. More than 60% of funded Gen AI startups, and nearly 90% of Gen AI unicorns are Google Cloud customers.

The keynote opens with an introductory video talking the power of AI today. (“AI you say? I’m shocked. Shocked, I tell you!”) The video talks about things we can do with AI now, like:

Using satellites to reduce methane emissions.
Turning DNA into code to make… Crop-resistant corn!
Spoting and filling potholes.
Spoting diseases earlier.
Scanning 100K lines of code in 2 minutes, in order to spot and fix bugs.

So this is “The new way to Cloud.”

So far, so cool.

Google has announced partnerships with 100s of leading AI partners

The early keynote includes a brief introduction to some of the topics of this year’s Next:

Over 300 customers and partners will be sharing their Gen AI success stories at this event.
Some discussion around the launch of Gemini and the advancements since its launch.
Google Distributed Cloud and Edge, to support highly confidential and edge workloads.
Cross-cloud networking now provides secure, low-latency connectivity of Google’s AI services to any application on any cloud.
Chrome Enterprise Premium Browser.
Multimodal Gen AI Agents will transform how we interact with the applications and the web. Agents are intelligent entities to do things like: customer agents, to help a shopper find the perfect dress; or helping an employee pick the right health benefits.

The AI Stack

The keynote talks about Google’s AI stack:

Note the rebranding of Duet AI to “Gemini for Google Cloud”.
AI Hypercomputer: an integrated AI infrastructure platform for offering AI at scale. There are a number of announcements related to GPUs, TPUs, and AI-optimised storage.

The keynote includes announcements around:

Google Distributed Cloud, which has a number of capability enhancements around GKE, Vertex AI, and AI model support. GDC now has both “secret” and “top secret” accreditations. Mobile operator “Orange” referenced as an organisation running across 26 countries and using GDC to keep data localised to each country.
Google Axion. A new custom ARM-based CPU that offers considerably higher performance and lowe energy consumption than caparable current gen x86.
Gemini 1.5 Pro in public preview. It has the world’s largest context window. In a single shot, it can process: 1M tokens, 1 hour of video, 11 hours of audio, and over 30K lines of code.
Grounding of Gemini models with Google Search! This significantly reduces hallucination. Or you can ground with data from your own databases.
Vertex AI Agent Builder — to rapidly speed up creating AI Agents. Gemino Pro can create free-flowing conversations with text, voice, images and video as inputs. But also, it can even provide real time interactions in voice! Natural language can be used to train the AI agents, e.g. to describe topics that are verboten. You can configure transcription and summarisation. And response quality can be improved using vector search. Also, modular extensions can be integrated to complete standard customer workflows, e.g. booking a flight.

Shopping AI Agents

The keynote then demonstrates a shopping AI Agent, and the ability to upload a video and ask it:

Find me a checkered shirt like the keyboard player is wearing. I’d like to see prices, where to buy it, and how soon can I be wearing it?

The response is near instantaneous on the website. And then we see a demo of interacting with a voice AI agent which continues the interaction and completes the transaction. That’s pretty cool!

A Few Google Workspace Updates

Then the keynote moves onto Gemini for Google Workspace. Use it to:

Answer questions.
Create notes in meetings.
Extract insights from reports.
Create images to insert in presentations.
Real-time translation.

Announcements related to Google Workspace:

A recent benchmarking study shows Google Meet now outperforms Zoom and MS Teams for overall video performance.
Chat summarisation and real time translation now available for Google Meet.
AI Security add-on can automatically classify and protect company data.
Gemini in Google Chat can provide summaries of long conversations.

We see a demo of reviewing proposals, comparing them, and asking questions, e.g.

Does this offer comply with our compliance rule book?

Employee Agents

Next, we talk about how to create a multi-modal AI employee agents using Vertex AI:

Create a custom model with Vertex AI.
Connect the custom model to your company data and web data.
Ground in enterprise truth, e.g. with BigQuery and AlloyDB.

Then we see a demo of how you can use a Vertex AI employee agent to summarise an employee benefits enrollment email, as well as a one hour benefits video. The agent is able to reason across text, video and the prompt, and provide a summary. Furthermore, the agent is able to compare the proposed plan to a previous plan, and make inferences.

Creative AI Agents

Now we move on to Creative AI Agents. Carrefore are using Creative AI Agents for marketing; they built a new marketing studio using Vertex AI, in just five weeks. Now they can build personalised campaigns in just a few clicks.

Creative agents uses Gemino Pro to look at existing material, documents and brand images, to infer a brand identity. We can generate multi-modal content; we can create live images, and even podcasts!

Then there was the announcement of Google Vids, the AI-powered collaborative video creation app, as part of Google Workspace. Aparna then demos creating a recap video of the Next event, using Google Vids:

Creating a video recap in seconds, using Google Vids

Then we have announcements of Imagen 2.0 Text-to-Image, including new editing modes to edit a generated image. And there’s the new Text-to-Live Image, which is now in preview:

Data Agents

So many agents!!

AI Data Agents us to ask natural language questions of our data. Gemini in BigQuery is now in Preview, and allows AI-powered data preparation, analysis and querying. BigQuery can be integrated directly with Vertex AI. So now we can perform multi-modal analysis across all of documents, images, videos, audio, and structured data.

One extremely cool thing about this demo was that the agent built a forecast dynamically, using BigQuery ML. And then uses vector embeddings to find products that look like a supplied image.

Code Agents

Surprise… More agents.

Google’s AI code assistant is now called Gemini Code Assist. (No more Duet AI.)

The keynote talks about how Gemini Code Assist can be used with a code base anywhere… On-prem, GitLab, GitHub, BitBucket, etc. Furthermore, Gemini Code Assist supports data residency requirements in multiple regions. It is now integrated with Gemini 1.5 Pro, and can leverage the new 1-million token context window.

The demo was cool… Show the visual mockup of a new UI to Gemini Code Assist, and it generates the code, leveraging our entire (huge) code base, and aligned to our code standards.

Security Agents

Please… No more agents!

These AI agents assist security operations teams, radically increasing the speed of security investigation and response.

There were a number of announcements relating to integration of Gemini into security products:

Public preview: Gemini in Threat Intelligence. Tap into Mandiant’s frontline threat intelligence using using natural language prompts.
Public preview: Gemini in Security Operations. Summarise and explain findings, recommend next steps, and even write and execute remediation playbooks.
Public preview: Gemini in Security Command Centre. Evaluate security posture, and summarise potential attack paths and risks.

Wrap-Up

Thomas Kurian wraps-up by saying:

Our open platform offers choice at every layer.

Chips (CPUs, TPUs, GPUs) for training and serving.
Your choice of models.
Your choice of development environments.
Databases, including vector.
Your choice of business applications.

We’re creating a new era of generative AI agents, built on a new, truly open platform for AI. And we’re reinventing infrastructure to support it.

What’s Next?

(See what I did there?)

I’ll watch a bunch of sessions I’m interested in, and provides some useful nuggets and summaries soon. I’ll put these in some separate articles, rather than just adding to this one.

Before You Go

Please share this with anyone that you think will be interested. It might help them, and it really helps me!
Please give me claps! You know you clap more than once, right?
Feel free to leave a comment 💬.
Follow and subscribe, so you don’t miss my content. Go to my Profile Page, and click on these icons:

Follow and Subscribe