Building a Smart Retail Shopping Assistant PART 3

Integrate your retail assistant to a web app using Agent Builder API

Image of an app displaying “black shoe with high heel” as a search result

Background

In the previous installments of this series, we constructed a powerful knowledge-driven analytics engine and then smoothly integrated it into a conversational agent using Agent Builder. Now, the final piece of the puzzle is to make this intelligent assistant accessible to your customers through a web-based chat interface. We’ll achieve this by deploying a web application on Cloud Run and leveraging the Agent Builder API to create a dynamic conversational experience.

Part 1: Building a Smart Retail Shopping Assistant PART 1

Part 2: Building a Smart Retail Shopping Assistant PART 2

Why Cloud Run?

Cloud Run is a serverless platform that makes it easy to deploy and scale stateless containers. It’s a perfect fit for our web application because:

Serverless Simplicity: Cloud Run abstracts away infrastructure management, letting you focus on your application code.

Automatic Scaling: It scales up or down automatically based on incoming traffic, ensuring consistent performance and cost-efficiency.

Customizable: You can easily tailor the runtime environment to your needs, including specifying memory and CPU allocations.

Connecting the Dots: Agent Builder API and Cloud Run

Cloud Function Setup

We’ll utilize the Cloud Function you created in Part 2 for the conversation engine. We will write a function that acts as the bridge between our web application and Agent Builder. It handles the detectIntent requests, sending user input to Agent Builder and returning the agent’s response.

1. To get started with this, navigate to Cloud Functions in Google Cloud Console.
2. Click CREATE FUNCTION on the top of the page.
3. Select 2nd gen as the environment, “detectintent” as the function name, “us-central1” as the region, HTTPS as the trigger type and “allow unauthenticated” for now.
4. Click the NEXT button.
5. Select Python 3.11 from Runtime and Source code set to “Inline Editor”.
6. Replace the contents of the main.py and requirements.txt from the github repo files main.py and requirements.txt respectively.
7. Click DEPLOY and change the permissions accordingly (as needed by your application). Your deployed endpoint looks like this: https://us-central1-*****.cloudfunctions.net/detectintent_python

Source Code for the Cloud Functions Python App, main.py:

import functions_framework
from google.cloud import dialogflowcx_v3beta1 as dialogflow
import argparse
import uuid

from google.cloud.dialogflowcx_v3beta1.services.agents import AgentsClient
from google.cloud.dialogflowcx_v3beta1.services.sessions import SessionsClient
from google.cloud.dialogflowcx_v3beta1.types import session

@functions_framework.http
def hello_get(request):
    request_json = request.get_json(silent=True)
    print(f"Request JSON: {request_json}")
    if request_json and 'project_id' in request_json and 'location_id' in request_json and 'agent_id' in request_json and 'texts' in request_json and 'language_code' in request_json:
        project_id = request_json['project_id']
        location_id = request_json['location_id']
        agent_id = request_json['agent_id']
        texts = request_json['texts']
        language_code = request_json['language_code']
        return run_sample(project_id, location_id, agent_id, texts, language_code)
    else:
        return 'Missing required parameters in request JSON', 400

def run_sample(project_id, location_id, agent_id, texts, language_code):
    session_id = texts[0].split(":_+_:")[0];
    return detect_intent_texts(agent, session_id, texts, language_code)


def detect_intent_texts(agent, session_id, texts, language_code):
    """Returns the result of detect intent with texts as inputs.
    Using the same `session_id` between requests allows continuation
    of the conversation."""
    session_path = f"{agent}/sessions/{session_id}"
    client_options = None
    agent_components = AgentsClient.parse_agent_path(agent)
    location_id = agent_components["location"]
    if location_id != "global":
        api_endpoint = f"{location_id}-dialogflow.googleapis.com:443"
        print(f"API Endpoint: {api_endpoint}\n")
        client_options = {"api_endpoint": api_endpoint}
    session_client = SessionsClient(client_options=client_options)

    for text in texts:
        text_input = session.TextInput(text=text.split(":_+_:")[1])
        query_input = session.QueryInput(text=text_input, language_code=language_code)
        request = session.DetectIntentRequest(
            session=session_path, query_input=query_input
        )
        response = session_client.detect_intent(request=request)

        print("=" * 20)
        print(f"Query text: {response.query_result.text}")
        response_messages = [
            " ".join(msg.text.text) for msg in response.query_result.response_messages
        ]
        print(f"Response text: {' '.join(response_messages)}\n")
        resultprint = f"Response text: {' '.join(response_messages)}\n"

    return resultprint

Explanation

The Cloud Function handles requests from the web application, which contain user input and session information.

It uses the Dialogflow CX library to interact with the Agent Builder API, specifically the detectIntent method.

Intent: The detectIntent method analyzes the user’s query to identify the underlying intent (e.g., “product_search,” “recommendation_request”). This intent then guides how the agent should respond.

Session: A session is a unique conversation identifier. The Cloud Function maintains session state by passing the session ID with each API call. This ensures that the agent remembers the context of the conversation and can respond appropriately.

Test the Agent Builder API Integration

Now that the Cloud Functions application is deployed, go ahead and test it in your Cloud Functions Console, in the TESTING tab, with your request data. If you do not see the action to directly test the function, you should get a CURL command that you can execute from the Cloud Shell Terminal to see your Cloud Function endpoint in action.

Request Data:
Remember you can get the AGENT_ID from your Agent Builder Application created in part 2 of this series.

{
"project_id": "YOUR_PROJECT_ID",
"location_id": "us-central1",
"agent_id": "YOUR_AGENT_ID_FROM_THE_AGENT_BUILDER_APP",
"texts": ["YOUR_TEST_SESSION_ID:_+_:I AM LOOKING FOR YELLOW SHOES"],
"language_code": "en-us"
}

Response:

Response from the agent

Repeat testing with the same SESSION_ID and you will see the conversation flow.

Web Application

For this we will bootstrap a Java Cloud Run Application from Cloud Shell Editor. Run the below command to clone the repository code and replace the placeholders for your PROJECT_ID, ENDPOINT for the Cloud Function created in the above section.

git clone https://github.com/AbiramiSukumaran/Retail-Agent-Web-App

Once the repo is cloned, the project structure looks like this.

Project Structure

Note: You have to update your PROJECT_ID and AGENT_ID in HelloWorldController.java and the Cloud Function Endpoint in the DetectIntent.java files.

Explanation

Frontend (HTML, CSS, JavaScript): This is the user interface for the chat application. It includes a chat window where users can type their queries and see the responses from the agent.

Backend (Spring Boot Controller):
This component handles the logic of interacting with the Cloud Function. It receives user input from the frontend, sends it to the Cloud Function, and then displays the agent’s response in the chat window.

API Integration

1. The Spring Boot controller makes HTTP requests to the Cloud Function endpoint.
2. It passes the user’s query and session information in the request payload.
3. The Cloud Function, in turn, sends this information to Agent Builder using the detectIntent method, retrieves the agent’s response, and returns it to the controller.
4. The controller then displays the response in the web application’s chat window.

Deployment to Cloud Run

To cover the 2 steps, below we will run the gcloud run deploy command from Cloud Shell Terminal:

1. Package your Spring Boot application as a container image.
2. Deploy the container image to Cloud Run, configuring the necessary environment variables (e.g., Cloud Function URL, Agent ID).

cd Retail-Agent-Web-App
gcloud run deploy --source .

The terminal will prompt you for some details like app name, region, authentication (I selected unauthenticated for this demo app purpose) etc. Provide the details and wait for the app to be deployed in Cloud Run. In a few minutes, your app will be deployed with the endpoint that looks like this:

https://retailagent-********-uc.a.run.app/

Going Live!

Demo

Once your application is deployed to Cloud Run, you’ll have a fully functional web-based chat interface powered by your intelligent retail shopping assistant. Customers can interact with your agent, ask questions, get personalized recommendations, and enjoy the smart shopping experience.

Performance Considerations

You can adjust a few things to get better responses from your data

1. Fine tune the Vector Search parameters and choose the right distance method that works for your data and use case (in the Part 1 Blog of the series).
2. Index the embeddings to get performant searches in the Vector Search step (Part 1 Blog).
3. Adjust the prompt and add some more context to make the Gemini results as detailed or abstract as you want (Part 1 Blog).
4. Add more steps and flows to the Agent Builder app to make the app discover intents and respond accordingly with multiple sources and contexts, if required (Part 2 Blog).
5. Provide more examples for each category of data about which, you think, the users might be requesting information (Part 2 Blog).

Operational Considerations

• Error Handling: Implement robust error handling in both the Cloud Function and web application to gracefully handle network issues, API failures, or invalid responses.
• Security: Secure your Cloud Function endpoint with appropriate authentication and authorization mechanisms to prevent unauthorized access.
• Session Management: Properly manage session state in your web application to ensure that conversations are maintained across multiple user interactions.

The Future of Conversational AI in Retail

This series has demonstrated how you can leverage the power of Google Cloud technologies like AlloyDB, Cloud Functions, Agent Builder, and Gemini to build a cutting-edge retail shopping assistant. This is just the beginning — as conversational AI continues to evolve, the possibilities for enhancing customer experiences and driving business growth are virtually limitless.

We hope this series has inspired you to explore conversational AI using Agent Builder and its potential in various industries. If you wish to tell us what you built, reach out. To stay top everything that is new with Google Cloud, become an Innovator and potentially an Innovator Champion.