How to Use Vertex AI, Langchain, and Google Cloud Functions for AI Applications
Introduction
Vertex AI is a fully managed machine learning (ML) platform that lets you train, deploy, and manage ML models and applications at scale. It is a comprehensive platform that covers the entire ML lifecycle, from data preparation to model deployment and monitoring.
Langchain is a powerful and versatile JavaScript library designed to simplify the process of working with natural language processing (NLP) and machine learning models. With its easy-to-use API and extensive functionality, Langchain enables developers to harness the power of NLP in their applications without having to delve deep into the complexities of machine learning algorithms.
Google Cloud Functions is a serverless computing platform provided by Google Cloud. It allows developers to create and deploy functions that automatically scale with the number of requests, without the need to manage any underlying infrastructure. This event-driven architecture enables developers to write single-purpose, stateless functions that are triggered by various events, such as HTTP requests or changes in the data.
By integrating Langchain with Google Cloud Functions, developers can easily build and deploy AI-powered applications that scale effortlessly, while reducing the operational overhead and costs associated with managing servers. In this article, I will walk you through the process of using the LangChain library with Google Cloud Functions, helping you leverage the benefits of serverless computing to create efficient and scalable AI applications.
️️⚠️ Warning: Please note that this article focuses on the technical aspects of integrating Langchain with Google Cloud Functions and does not address security concerns. Each solution architect should implement their own security measures according to their organization’s standards and best practices. The examples provided in this article are for educational purposes only and should not be used in production environments without applying the necessary security precautions.
Prerequisites
Here’s a list of the necessary tools, accounts, and knowledge required for this tutorial:
1. Google Cloud account: To work with Google Cloud Functions and Vertex AI, you’ll need a Google Cloud account. If you don’t have one, you can sign up for a free trial at https://cloud.google.com/free.
2. Basic Python knowledge: This tutorial assumes that you have a fundamental understanding of Python, as we’ll be using it to write our Google Cloud Function and work with the LangChain library.
3. Familiarity with Google Cloud Functions: While not strictly necessary, it’s helpful to have some prior knowledge of Google Cloud Functions, its concepts, and its operations. You can learn more about Google Cloud Functions at https://cloud.google.com/functions.
With these tools, accounts, and knowledge in place, let’s get this party started.
Setting up Vertex AI & Cloud Functions
- Enable Vertex AI recommended APIs at https://console.cloud.google.com/vertex-ai
- Create a Google Cloud Function:
a. Log in to your Google Cloud account and navigate to the Google Cloud Console: https://console.cloud.google.com/.
b. Select your desired project or create a new one by clicking on the project dropdown at the top of the page.
c. In the left-hand menu, click on the “Navigation menu” (hamburger icon), then navigate to “More Products” > “Serverless” > “Cloud Functions.”
d. Click on the “Create Function” button to start creating a new Google Cloud Function.
2. Configure the Google Cloud Function:
a. Environment: Change to “2nd gen” for using the next-generation of Function-as-a-Service.
b. Function name: Give your function a unique and descriptive name. This name will be used as an identifier within the Google Cloud Console.
c. Region: You can leave the default or select your preference.
d. Trigger: For this tutorial, we’ll use the HTTPS trigger with no authentication. Change the selection to “Allow unauthenticated invocations” However, in a production environment, it’s essential to implement proper authentication and security measures.
e. Open the “Runtime, build, connections and security settings” and scroll down to “Runtime environment variables”
Use the “Add variable” to add:
Name: OPENAI_API_KEY
Value: <YOUR_OPENAI_KEY>
Click the “NEXT” button
a. Runtime: Choose “Python” as your runtime environment and select a suitable version (This tutorial was built on Python 3.12) from the dropdown menu.
b. Default CODE: No need to change for now
import functions_framework
@functions_framework.http
def hello_http(request):
"""HTTP Cloud Function.
Args:
request (flask.Request): The request object.
<https://flask.palletsprojects.com/en/1.1.x/api/#incoming-request-data>
Returns:
The response text, or any set of values that can be turned into a
Response object using `make_response`
<https://flask.palletsprojects.com/en/1.1.x/api/#flask.make_response>.
"""
request_json = request.get_json(silent=True)
request_args = request.args
if request_json and 'name' in request_json:
name = request_json['name']
elif request_args and 'name' in request_args:
name = request_args['name']
else:
name = 'World'
return 'Hello {}!'.format(name)3. Save and deploy:
Click the “Deploy” button at the bottom of the page to save and deploy your function. The deployment process may take a few minutes.
After the deployment ends, your function will be Active and you can copy the URL of the endpoint.
Congratulations, You can use tools like curl, Postman, or your browser to send the request to the endpoint and check the results.
Installing and Configuring Vertex AI & Langchain
To install the LangChain library, you need to include it as a dependency in your project. Since we’re using the inline code editor in the Google Cloud Console, you can add the Langchain dependency to your `requirements.txt` file:
Click on “Edit” at the top of the screen, then “Next” to get to the code editor area. In the Google Cloud Console’s inline editor, click on the `requirements.txt` tab and update the code:
functions-framework==3.*
langchain
google-cloud-aiplatform* This tutorial was built with langchain version: 0.0.347
Now that you’ve declared the Langchain library, you can use it in your Google Cloud Function. Go back to the `main.py` file and use copy this code:
import functions_framework
from langchain.llms import VertexAI
llm = VertexAI(temperature=0.5,model_name='text-bison')
@functions_framework.http
def hello_http(request):
request_json = request.get_json(silent=True)
request_args = request.args
question = (request_json or request_args).get('question', 'Hi')
answer = llm(question)
return answerNow you have a working Google Cloud Function that utilizes Langchain library and the VertexAI API to answer any prompt question. Let’s break down the code and explain each part:
1. Import required modules:
We’re importing the necessary modules from both the Google Cloud Functions Framework and the Langchain library. langchain library will take care of the VortexAI API with the help of google-cloud-aiplatform.
2. Initialize the VertexAI instance:
We create a new instance of the `VertexAI` class, Set the model to `text-bison` and the temperature to 0.5. The `llm` object will be used to communicate with the VertexAI API.
3. Create the Google Cloud Function:
We define an HTTP-triggered Google Cloud Function named `helloHttp` that takes an incoming request (`req`) and sends a response (`res`).
4. Extract the query:
We extract the user’s query (the text to be translated) from either the request’s query string or body using the parameter `question`.
5. Call the VertexAI API:
We use the `llm()` method to send the prompt question to the VertexAI API and await the response, storing it in the `answer` variable.
6. Send the response:
We send the `answer` as the response of the Google Cloud Function, which will be returned to the user.
Remember to save and deploy your changes by clicking the “Deploy” button in the Google Cloud Console. And to change the ‘name’ to ‘question’ on your curl command.
Handling CORS in the Google Cloud Function
In this section, let’s demonstrate how to configure your Google Cloud Function to allow cross-origin requests, thus overcoming CORS errors that may occur when your function is called from a different domain.
As before, let’s start with the code and then I will explain the changes:
import functions_framework
from langchain.llms import VertexAI
llm = VertexAI(temperature=0.5,model_name='text-bison')
@functions_framework.http
def hello_http(request):
if request.method == "OPTIONS":
headers = {
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Methods": "GET, POST, OPTIONS",
"Access-Control-Allow-Headers": "Content-Type, Access-Control-Allow-Headers, Authorization, X-Requested-With"
}
return ('', 204, headers)
headers = {
'Access-Control-Allow-Methods': 'POST',
'Access-Control-Allow-Origin': '*'
}
request_json = request.get_json(silent=True)
request_args = request.args
question = (request_json or request_args).get('question')
if not question or not isinstance(question, str) or len(question) == 0:
return ("Invalid question", 400, headers)
answer = llm(question)
return (answer, 200, headers)1. Handle OPTIONS requests:
When a client makes a cross-origin request, it might first send an HTTP `OPTIONS` request to check if it’s allowed to access the resource. In Google Cloud Function, we check if the incoming request has the `OPTIONS` method, and if so, we respond with additional CORS-related headers:
- `Access-Control-Allow-Origin`: This header specifies which origins are allowed to access the resource. By setting it to `”*”`, we’re allowing any origin to access your Google Cloud Function. You can easily set it to your frontend domain.
- `Access-Control-Allow-Methods`: This header specifies which HTTP methods are allowed for the actual request.
- `Access-Control-Allow-Headers`: This header specifies which HTTP headers can be used during the actual request. We’re allowing several common headers, including `Content-Type`, `Access-Control-Allow-Headers`, `Authorization`, and `X-Requested-With`.
After setting these headers, we send a `204 No Content` response to the `OPTIONS` request.
2. Set CORS headers:
Before processing the incoming request, we set several CORS-related headers. These headers allow cross-origin requests and specify various rules for handling such requests:
- `Access-Control-Allow-Methods`: This header specifies which HTTP methods are allowed for the actual request. In this case, we’re allowing only the `POST` method. You can add `GET` if needed.
- `Access-Control-Allow-Origin`: This header specifies which origins are allowed to access the resource. By setting it to `”*”`, we’re allowing any origin to access your Google Cloud Function. You can easily set it to your frontend domain.
3. Process other requests:
If the request method is not `OPTIONS`, the Google Cloud Function proceeds with processing the incoming request, extracting the search query from the request body, and using the Langchain library to perform the query to the model. The response is then sent back with a `200 OK` status.
By adding this section to the code, we’ve enabled Google Cloud Function to handle cross-origin requests and avoid CORS errors when called from different domains.
I also made a few other changes to the original code. Let’s break down these changes and explain their purpose:
1. Add input validation:
We’ve introduced input validation to check if the `query` variable contains a valid string before proceeding with the translation. This validation helps prevent errors and ensures that the function only processes valid requests:
If the validation fails, the function sends a `400 Bad Request` response with the message “Invalid question”.
2. Update response status code:
When sending the answer in the response, we’ve explicitly set the status code to `200 OK` to provide a clear indication of a successful operation.
These changes to the original code help improve the function’s reliability and user experience by providing better input validation, more descriptive response messages, and a clearer indication of successful operations.
Best Practices and Tips
1. Optimize performance: To ensure that your serverless functions perform efficiently, try to keep the function code lightweight and focused on a single task. Also, consider using caching strategies for frequently accessed data to reduce the latency of your function.
2. Error handling: Implement proper error handling in your Google Cloud Functions to ensure that unexpected issues are caught and handled gracefully. This can help you maintain the stability and reliability of your application.
3. Security: As mentioned earlier, the examples provided in this tutorial do not include security measures. It’s essential to implement proper authentication, authorization, and input validation to protect your serverless functions from malicious attacks and unauthorized access.
4. Monitor and log: Set up monitoring and logging for your Google Cloud Functions using tools like Google Cloud Monitoring and Google Cloud Logging. Regularly reviewing logs and monitoring function performance can help you identify and resolve issues more quickly.
5. Function scaling: When using serverless functions with the Langchain library, consider how your functions might scale with increasing demand. Make sure you understand the scaling behaviour of Google Cloud Functions and any associated costs to avoid unexpected charges.
6. Stay up-to-date: Keep both your Google Cloud Function’s runtime environment and the Langchain library up-to-date to benefit from the latest features, bug fixes, and security patches.
7. Testing: Thoroughly test your serverless functions, both locally and in a staging environment, before deploying them to production. This can help you identify and resolve issues before they affect your users.
8. Documentation: Document your serverless functions and their usage, including any specific configuration options, inputs, and outputs. This documentation can help your team members and future developers better understand and maintain your serverless functions.
Conclusion
In conclusion, this article has demonstrated how to integrate the Langchain library with VertexAI and Google Cloud Functions to build powerful and scalable natural language processing applications. We’ve covered the necessary tools, accounts, and knowledge required for this tutorial and provided step-by-step instructions for setting up Google Cloud Functions, installing and configuring LangChain, and writing a serverless function that leverages the capabilities of the LangChain library and VertexAI models.
Furthermore, we’ve discussed how to handle CORS errors in your Google Cloud Function to allow cross-origin requests, and we shared best practices and tips for optimizing performance, security, and scalability when using Langchain with Google Cloud Functions.
I encourage you to explore the potential of integrating Langchain with Google Cloud Functions in your projects and discover how this powerful combination can help you create efficient and scalable solutions for various AI use cases across different industries. Remember to always implement appropriate security measures and follow best practices to ensure the reliability and stability of your applications.
Happy coding!
P.S: More about this subject in my Youtube video: Deploying LangChain on Cloud Functions
