Tutorial: Vertex AI Agent Builder for Developers

Recently, I worked with a Corporative Housing Society on automation projects, involving AppSheet, Sheets and AppsScript. They brought to me a problem that focussed on talking with your data. They had a Bye-Laws PDF that contained all the rules and regulations of a Corporative Housing Society.

Every time a resident wanted to search inside the document for a particular topic, they had to search through the entire document. To reduce time spent on searching, I came up with a solution where you can talk with your Data using Vertex AI’s Agent Builder.

In this blog, we will be building a ChatApp with Vertex AI Agent Builder that allows us to talk to our data.

So let’s get started.

If you want a video version of the blog, you can check out the video tutorial given below.

Enable API’s

Before we build the chat app, we have to enable a few APIs to get started. We have to enable the Dialogflow API and Vertex AI Search and Conversation API.

1. To Enable the Dialogflow API, follow these steps:
— While you are in your GCP project, search for the Dialogflow API in the search bar.

On searching for the API, go ahead and click on Enable to add it to your GCP Project. You can check out the Documentation and Pricing for the API.

2. To Enable the Vertex AI Search and Conversation API, follow these steps:
— On enabling the Dialogflow API, search for Vertex AI And Conversation Console, where you will be asked to select your Billing Account.

On clicking Set Account, you will be taken to the Vertex AI Agent Builder where you have to click on Activate the API.

On enabling the API, you will be taken to the Agent Builder where you can create your first chat app.

Setup your Data

Now that we have enabled the APIs that we require to build the chat app, it's time to set up the data source that the chat app is going to be using. We are going to start off by uploading the data to the Google Cloud Storage by creating a Bucket followed by creating a new Data Store that will be connected to our bucket.

To upload your data to the Google Cloud Storage, follow these steps:

1. Navigate to the Google Cloud Storage and click on Buckets.

2. On reaching the Buckets page, go ahead and click on CREATE.

3. On creating a new bucket, you have to enter some details to set up the bucket.

Once you enter all the details, go ahead and click on CREATE. To upload your data to the bucket you can either click on UPLOAD FILES or just drag and drop your Documents.

Now that the PDF has been uploaded to a Cloud Storage Bucket, let's go ahead and create a Data Store that is connected to our data. To know more about Data Stores. check out the documentation link given below.

Data stores | Vertex AI Agents | Google Cloud

Once you upload your data to the Cloud Storage, navigate back to the Agent Builder and click on Data Stores. To create a new Data Store follow these steps:

1. On clicking CREATE, you will be asked to select a data source, which in this case is going to be Cloud Storage.

2. On selecting Cloud Storage, you will be asked to specify the type of data we are importing, which in this case is Unstructured data.

On selecting the type of data, you have to specify the path of the folder where the PDF is stored. To get the path go back to your bucket and click on the copy button next to the name as shown below.

3. The last step includes specifying the location of your data store followed by giving it a name.

Once you have entered all the details, go ahead and click on CREATE and your data store will get created.

Create the Chat App

Now that we have the data ready, API’s enabled, its time to go ahead and create the chat app. To do that navigate to Apps inside of Agenet Builder and click Create a New App.

On clicking create, you will be provided with multiple app types that you can choose from. For this tutorial, we will be using the Chat App type, so you can go ahead and click on Select.

Once you select the Chat App, you have configure your agent settings by filling in some details such as the Company name, Agent Name and Location type. After entering the details, click on Continue.

Now that the agent is configured, its time to specify the data for the app. Once you click on Continue, you will be shown the various Data Stores that you have and select the one that you want for the app and click on Create.

Congratulations! You have successfully create your first Chat App using Vertex AI’s Agent Builder.

Note: It could take upto 4 hours for your agent to access your newly added documents while they are indexed, but usually gets done before that.

Test your Agent

Just like any software, conversational agents need thorough testing. This helps identify bugs in their code, understand their current capabilities, and most importantly, simulate real interactions with users to refine the overall customer experience.

While you are in the Agent Builder, go ahead and click on the Chat App we just created, which will show you the following interface.

This is the Dialogflow CX console where you can Build, Test and Manage your agent. To Test your agent click on the Test Agent button, which will bring up the Chat Interface.

I am going to be asking my agent the following questions:

1. What is the sinking fund ?
2. What are the rules for Subletting my flat ?
3. What are the service charges for the Society ?

If your agent responds with generic phrases like “I didn’t understand,” it could be because it’s still processing recently added information (domains, URLs, or documents). This indexing process can take up to 4 hours.

Here you can see that Chat App is able to answer questions based on the data specified earlier. To learn more about testing your agent, check out the Documentation link given below.

Dialogflow CX Console overview | Google Cloud

Publish your App

Now that the Chat App is up and running, it's time to publish it and embed it into our website. To publish your Chat App, click on the Publish button which will prompt you with the following.

You have to specify the agent environment, followed by the Access type and the UI Style that you want. Once you have all of this set, go ahead and click on Enable the unauthenticated API.

You will be provided with the code which you can put into your website and your chat app will be accessible on your website.

Conclusion

In this tutorial, we looked at how we can build a Chat App using Vertex AI’s Agent Builder. We utilized Google Cloud Storage for data storage, Data Stores for indexing and analyzing our data, and then implemented Dialogflow CX to configure the Chat App.

References:

Create a Generative Chat App with Vertex AI Conversation | Google Codelabs

Feel free to reach out if you have any issues/feedback at aryanirani123@gmail.com.