What is the Google Vertex AI Agent Builder?

A practical example: Google Search with Slack.

Google’s new Vertex AI Agent Builder simplifies the creation and deployment of AI-powered agents, making AI development more accessible to businesses and developers. This blog describes its capabilities by creating a Slack RAG application.

NOTE: This article is an introduction to the Google Vertex AI Agent Builder. We are not affiliated with Google. This article is also not to be understood as a review, but informational material as part of our first look experience.

Google has recently launched Vertex AI Agent Builder, a new tool designed to make it easier for developers to create and deploy AI-powered agents. This platform is part of Google Cloud’s Vertex AI suite and integrates with existing products like Vertex AI Search and Conversation.

The Vertex AI Agent builder is the first major and serious entry in the realm of RAG. Which is surprising, as a good chunk of RAG — Retrieval Augmented Generation — is about searching and finding information. A metier, Google is famously good in. So, I’m quite excited about the prospect of testing the Agent builder.

This article will provide a high level introduction to what the Google Vertex AI Agent Builder is, how it helps to create production-ready generative AI applications and a hands-on-example on how to create a RAG search engine on top of your Slack messages.

What is the Google Vertex AI Agent Builder?

Vertex AI Agent Builder is a platform that helps developers build and deploy AI agents, regardless of their level of expertise. It offers a no-code console for creating agents using natural language, as well as more advanced, open-source frameworks for experienced developers.

One of the main features of Vertex AI Agent Builder is its ability to ensure that AI outputs are factually accurate and relevant by grounding them in enterprise data. This is achieved through systems like Vertex AI Search and Retrieval Augmented Generation (RAG) APIs, as well as custom solutions for more complex needs.

Security and compliance are also important aspects of the Agent Builder. It adheres to industry standards like HIPAA and ISO 27000-series to ensure that AI applications are secure and reliable. This alone is quite powerful, as this enables a range of Enterprise-Grade applications which are otherwise underserved.

What’s quite unique is, that the platform guides users through each step of agent creation, from defining parameters to training the model, making the development process more accessible and user-friendly.

According to Google AI agents can help streamline business processes by automating routine tasks, freeing up resources and improving productivity. While this is obviously a marketing-induced sentence, it is grounded in reality. AI Agents are in fact believed to assist in these tasks.

One of the main value propositions of the Vertex AI Agent Builder is it’s quite intriguing low-code user interface. By making AI tools more accessible, Vertex AI Agent Builder could be a good tool for businesses to get started with AI — even without vast AI experience.

One final benefit to note: Its integration with Google Cloud infrastructure makes it a powerful platform for developing, deploying, and managing AI agents effectively, as you get the full Google Cloud Platform at your fingertips.

Summary: Features of the Vertex AI Agent Builder

No-code AI agent creation: Develop conversational AI agents without writing code, using natural language inputs.

Data grounding: Connect AI agents to trusted data sources like Google Search and enterprise data using APIs for accurate and relevant responses. Google provides connectors to some external tools for a full RAG experience.

Efficient testing and deployment: Use comprehensive tools for performance evaluation and tuning, with scalable deployment across enterprise-ready infrastructure.

Secure and compliant: Ensure AI operations adhere to HIPAA and ISO standards, with data privacy management and responsible AI model usage.

Advanced AI capabilities: Integrate document parsing, ranking, grounded generation, and fact-checking APIs for custom solutions.

Scalable vector search: Build applications that handle billions of vectors, combining vector and keyword search for relevant responses.

LangChain integration: Use the open-source Python framework for creating and deploying high-performance AI agents on Vertex AI. This part is especially intriguing, as this allows to use some parts of the Google ecosystem but extend it with building blocks from other AI platforms. Eg. use the AI agent builder connectors and vector search, but an OpenAI model.

How to use the Vertex AI Agent Builder to connect AI to your Slack messages?

Now that we know what the Vertex AI Agent Builder is capable of doing, let’s but it to test.

We are going to implement a RAG system which uses our messages in Slack to answer questions. Why Slack you might ask? Well, Slack is one of the tools were probably enormous amounts of a companies knowledge is stored. If we can make this knowledge accessible — by utilizing AI — we would create tremendous value.

Before we get started though, we need to set up access control for our data source. More information about this admittedly quite cumbersome topic to be found here. To just get it done, navigate here click on the “edit” button and select “Google Identity” as Identity provider for your data sources.

1. Get your Slack workspace ID: To get started, you need to get your Slack workspace ID. You can find it as follows:

• From your desktop, open Slack in a web browser using URL URL (ex. app.slack.com).
• Once the page loads and you sign in, the URL will be in the following format: https://app.slack.com/client/TXXXXXXX/CXXXXXXX.
• Your workspace ID is the string beginning with T.

Slack Client Id

2. Create a Slack app: Next, you need to create a Slack app to access your workspace messages. Follow these steps:

• Visit this slack tutorial site to quickly create a Slack app and access token.
• Click on “Create App” and follow the install wizard. This will create a Slack application with some default scopes. (Note: For production use, please edit the scopes to what you really need.)
• Once the app is created, click “Install in workspace”
• Click on “OAuth & Permissions” and copy the “Bot User OAuth Access Token”.

3. Add a data store to your Agent Builder

• Navigate to the Agent Builders Data Store Settings.

4. Click Create data store, select Slack and enter your Slack authentication information. Click Authenticate.

5. In the next screen, select what you want to sync. You can select from Files, Messages and Conversations

Note: Google Agent Builder can’t sync private channels, multi-party instant messages, and 1:1 instant messages as of time of this writing.

Slack Data Store Configuration

6. Click continue and select a location for your data store. Enter a name for your data store.

7. Select a synchronization frequency for your data. Click Create. Note that as of time of this writing, there are three options:

• Daily
• Every 3 days
• Every 5 days

Vertex AI will now start ingesting your Slack messages. You can check the status of your ingestion by going to the Data stores page and clicking on your data store name.

Slack Data Store

In general, syncing data seems to take a while. So, be patient.

Create a Vertex AI Agent Builder App

Now that we’ve set up our data sources and created an index pipeline, we can create an app that uses this data. Vertex AI provides 4 different applications:

• Search: A search application that uses the data store as well as web search to answer users’ questions. This is what’s similar to ‘traditional’ RAG.
• Chat: A chat-bot-like application, where you define different chat paths. More like pre-LLM chatbots — but enhanced with the power of LLMs. Compared to “just” search, chat allows for more complex interactions and user guiding.
• Recommendation: A recommendation application that uses the data store to recommend items.
• Agent: This option ins quite intriguing as it allows to define an LLM-agent system, with multiple agents by just providing a high-level description of what the agents should do and what steps they should engage in to reach their goals. This is quite powerful. We will dive deeper into this in a future article.

NOTE: As of time of this writing, only Search and Agent seem to be compatible with external connectors like Slack.

For this example, we will create a search application.

1. Navigate to the Agent Builders Console and click on “Create new app”
2. Leave the default settings, add a name and select a region. Click “Continue”.

Create a new app

3. In the next screen select the data store you created earlier. Click “Create”.

Select Data Store during app creation

4. Once the app is created, you can test it by navigating to the Preview tab of the application. As we built a “Search” app, the preview is a nice and simple search bar.

Slack Search App Preview

Advanced features of the Search App

While the created app is at first glance just that — a search bar which finds information — it is actually enriched with Googles Gemini AI LLM models. They summarize search results and provide the possibility to ask follow-up questions (which is enormously helpful in a search context).

When navigating to the “Configuration” page of the app, you can find the “Advanced” option which allows to enable/disable these LLM options.

Advanced Search App Configuration

While I highly recommend keeping them enabled — there is a price tag attached to this.

• Default Search Pricing: 2$ per 1000 queries
• With Enterprise Features enabled: 4$ per 1000 queries
• With LLM enabled: plus additional 4$ per 1000 queries

With all features enabled you are at 8$ per 1000 search queries.

It’s best to test the search behavior with and without these features to decide whether you need them.

Deploying a Vertex AI Agent Build app

Now that our app is up and running, we want to make it accessible to our users. Google offers 3 different deployment options:

1. Web app: A hosted page that can be accessed via predefined URL. This is the easiest option, as it requires just a click on “Enable”. Drawback: It’s just the search input box. So not that useful for most people.
2. Widget: A widget that can be embedded in your website. This is quite powerful, as it allows to integrate the search functionality directly into your website. This is especially useful for customer facing websites.
3. API: An API that can be used to integrate the search functionality into your own applications. This is the most powerful option, as it allows to integrate the search functionality into your own applications.

How to deploy Vertex AI as a Widget?

For the sake of providing a complete example, the following steps are required to integrate the Slack search app into your own website:

1. Navigate to the Agent Builder Console
2. Select the App you just created.
3. Click on “Integration” in the left-hand menu.
4. Click on “Widget”. There you need to enter your domain, click “Add” and then “Save”.
5. This will create a JS snippet which looks similar to this:

<!-- Widget JavaScript bundle --> <script
src="https://cloud.google.com/ai/gen-app-builder/client?hl=en_US"></script>

<!-- Search widget element is not visible by default -->
<gen-search-widget
configId="<config-id>"
location="eu"
triggerId="searchWidgetTrigger">
</gen-search-widget>

<!-- Element that opens the widget on click.-->
<input placeholder="Search here" id="searchWidgetTrigger" />

Add this snippet to your website. This will create a hidden search widget, which will be opened by clicking in the input field. Please note that the triggering element does not necessarily have to be an input. It can be any HTML element — just match the id and the triggerId of the widget.

6. Finally, you also need to create an OAUTH token and pass it to the widget element.

// Set authorization token.
const searchWidget = document.querySelector("gen-search-widget")
searchWidget.authToken = "<JWT or OAuth token provided by your backend>"

Creating an API integration works quite similarly — just navigate to “API” instead of “Widget” in the integration tab.

Conclusion

In this blog post we saw the capabilities of how to create a Google Search like application based on data in our own systems — in this case Slack.

What’s quite intriguing is that you can create all of that with just UI. Absolutely no code required. This is quite powerful, as it allows to create AI applications without the need of a data scientist or AI expert.

In this article we had a look at three main elements of the Vertex AI Agent Builder: External Connectors, Data Stores and Search Applications.

The external connectors provide a low-code way to connect to external systems and data sources, while the data stores seem to be vector stores which can subsequently be connected to the applications. The search application itself is easy to integrate in existing websites and even provides a powerful API integration.

In one of the next articles we are excited to explore “Agent” Vertex AI applications, as they promise to be much more flexible than the “Search” app and therefore more powerful. We also like to delve deeper into the “Chat” app offered by Vertex AI, because while it was easy to configure the Search app, it was — at the end — just search. And therefore a bit disappointing, compared to modern RAG systems with chat capabilities. So, stay tuned for our next, more advanced entry into the Vertex AI Agent Builder ecosystem.

Further Reading

Integrating enterprise knowledge with LLMs

Improving Retrieval Augmented Generation: A Step-by-Step Evaluation of RAG Pipelines

Advanced RAG: Recursive Retrieval with llamaindex