Exploring Google Cloud Reasoning Engine
Concept
As I am exploring tools and options to develop Generative AI applications I found out that GCP has this Reasoning Engine service (also named by LangChain on VertexAI). By the docs it is mentioned as
LangChain on Vertex AI (Preview) lets you leverage the LangChain open source library to build custom Generative AI applications and use Vertex AI for models, tools and deployment. With LangChain on Vertex AI (Preview), you can do the following:
- Select the large language model (LLM) that you want to work with.
- Define tools to access external APIs.
- Structure the interface between the user and the system components in an orchestration framework.
- Deploy the framework to a managed runtime.
https://cloud.google.com/vertex-ai/generative-ai/docs/reasoning-engine/overview
The service itself was still in preview when I wrote this story so it may be different or even have more features.
The reason I have huge interest is the orchestration that Reasoning Engine has and at the same time it will manage the runtime for me which allows me to accelerate deployment and in the same time add more application logic.
So the main concept for this story is that I want to modify my earlier Imagen and Gemini-pro-vision test case to utilize Reasoning Engine. Please find the previous writing (without Reasoning Engine) below.
Architecture
This is the main idea of the architecture that I have for Apps Client which is the Frontend for user interface, most of the prompt will be sent to Agent Process-1 which is a Reasoning Engine deployment with two tools (weather api, and vertex search function). Last but not least the integration with Vertex Search to run queries to my sample data in BigQuery.
Steps
Following are the ideas on how things work behind the scenes and steps to re-create the application.
[One] I need to prepare the sources of the external services / datastore to be consumed. This means I need the openweathermap api key and prepare Vertex Search and BigQuery Dataset.
This is a sample dataset that I used (randomly created)
And a Vertex Search Application to ingest the data. I use these two following documents to create a datastore and create an application that uses the datastore.
I will need the information of the engine id (not the datastore id) for my agent. Please go to the Agent Builder and retrieve the ID
[Two] Create the Agent (Reasoning Engine). This is more straightforward and managed by Google. Two things needed are Google Cloud Storage Bucket for staging deployment and initialize Vertex AI SDK for Python.
Through this Agent and deployed to Reasoning Engine I can utilize the tools and expect a response that is already restructured for human readability which is neat.
The only thing that I need to prepare is the requirements.txt and run the application
python agent.pyWhen I deploy the application I will need the reasoning engine ID so I will need to store the id correctly. In the example I can retrieve from the end of the URL of reasoning_engine result
Then I can use ReasoningEngine on the client SDK to call the agent.
[Three] Let’s create a simple UI with the help of Strimlit. This application is similar with the previous test of Imagen and Gemini-Pro-Vision please find more details on
The concept of the prompt can be broken down as follows (please disregard the messy code and concept, this is only intended to explore gemini and reasoning engine).
- User start with uploading an Image (with the assumption that the application already have the user profile)
- Gemini will extract the location (city) where the image is taken.
- With the location information then we can have further content: Current Weather through openweathermap, and some enrichment such as summary and highlights. All of this will be the output of the prompt from Reasoning Engine.
- Adding personalization by creating a dynamic image based on user profile + location + weather.
- Adding last important information, any promotion that we have (stored in BigQuery)
- Call to action button
