Generative AI app development using Vertex AI and Golang

Vertex AI is a unified platform for managing and orchestrating machine learning (ML) workflows with a comprehensive set of tools and services that make it easy to build, train, and deploy ML models at scale. In this article, we are going to use Vertex AI pre-trained large language model PaLM 2 for Text (aka. text-bison) to create an outline for a scientific essay using your input topic and a predefined prompt. You can easily tweak it to fit other needs and explore the power of PaLM 2 generative AI features.

Inspired by Deploying a Google Cloud Generative AI App in a Website with Cloud Run, I write this article focusing on application development aspect in Go (Golang) instead of commonly available Python examples.

Prerequisites:

• Google Cloud Platform (GCP) account. Signup for free here
• Download and install Cloud SDK
• Go

Text Generation Model

Before going to the code, we need to create a new prompt using Generative AI Studio.

Go to the GCP Console and choose Vertex AI in the Artificial Intelligence section. Choose Language at Generative AI Studio section, and Text Prompt.

On the right pane, set Model to text-bison@001. Create the prompt, tweak Temperature, Token Limit, Top-K and Top-P parameters. You can add inputs and outputs to tune the model.

The following prompt and parameters will be used for the code later:

Prompt:
Create an outline for a scientific essay. Include the maths, experimental possibilities, alternatives, benefit and risk.

Input: ${input}
Output:

Parameters:
Temperature (temperature): 0.8
Token Limit (maxOutputToken): 512
Top-K (topK): 40
Top-P (topP): 0.8

Note: You can change to different model, prompt and parameters you like. This is just an example.

Setting up local development environment

Make sure you have a GCP account, a project, and Cloud SDK installed on your development machine. Use Application Default Credentials (ADC) to store your credential locally. This credential will be automatically used when the code calls Vertex AI API.

gcloud auth login
gcloud config set project ${YOUR_GCP_PROJECT}
gcloud services enable aiplatform.googleapis.com
gcloud auth application-default login

You will be redirected to the browser to authenticate, copy/paste authorization code back to your gcloud CLI.

Warning: ADC is for development only, you should use Service Account attached to GCP resource in production environment.

IAM Permission

IAM permission aiplatform.endpoints.predict is required to make prompt requests. Role roles/aiplatform.user (Vertex AI User) is a predefined role with that permission, you need to grant this role to your GCP user or group.

Note: Grant permission to the Service Account if your service is deployed to GCP.

Application Development

There are multiple approach for the application to access Vertex AI API:

1. Call the API directly
   When using Generative AI Studio, go to button <>VIEW CODE. Generative AI Studio can generate Python, Python Colab and Curl. You can manually construct Go http call using the same Curl parameters. This is ok for development purpose but for production environment you need extra steps to extract Service Account and pass it as Authorization token.
2. Google Cloud AI Platform Go library
   This is a full version of Vertex AI Platform library with a lot of features and models. Essentially a superset of what we need.
   Google Cloud Go documentation
   Go Library doc
3. Google Cloud AI Go library (Beta)
   This library has Generative Language similar to Python version. It is still in Beta when writing this article, and subject to change.
   Google Cloud Go documentation (v0.1.0 apiv1beta2)
   Go Library doc (v0.1.0 rev Jul 10th 2023)

Let’s go with Option #2, I plan to add Option #3 when v1 is ready.

Go ahead to initiate go.mod if you have not done so. We need to download a few dependencies:

go mod init yourmodulename
go get cloud.google.com/go/aiplatform
go get google.golang.org/api
go get google.golang.org/protobuf

We use the model, prompt and parameters we’ve defined earlier:

package main

import (
   "context"
   "fmt"
   "log"
  
   aiplatform "cloud.google.com/go/aiplatform/apiv1"
   aiplatformpb "cloud.google.com/go/aiplatform/apiv1/aiplatformpb"
   "google.golang.org/api/option"
   "google.golang.org/protobuf/types/known/structpb"
)

const INPUT = "warp drive" // Input topic to generate the essay

func main() {
   ctx := context.Background()
  
   // As of Jul 2023, only available in us-central1
   c, err := aiplatform.NewPredictionClient(ctx, option.WithEndpoint("us-central1-aiplatform.googleapis.com:443"))
   if err != nil {
      log.Fatalf("Error 1: %v", err)
   }
   defer c.Close()
 
   params, err := structpb.NewValue(map[string]interface{}{
      "temperature":     0.2,
      "maxOutputTokens": 512,
      "topK":            40,
      "topP":            0.8,
   })
   if err != nil {
      log.Fatalf("Error 2: %v", err)
   }

   // Add input to the prompt
   instance, err := structpb.NewValue(map[string]interface{}{
      "prompt": fmt.Sprintf("Create an outline for a scientific essay. Includes the maths, experimental possibilities, alternatives, benefit and risk. input: %s. output:", INPUT),
   })
   if err != nil {
      log.Fatalf("Error 3: %v", err)
   }
  
   req := &aiplatformpb.PredictRequest{
      // Replace your-gcp-project to your GCP Project ID
      // Notice the model text-bison@001 at the end of the endpoint
      // If you want to use other model, change here
      Endpoint:   "projects/your-gcp-project/locations/us-central1/publishers/google/models/text-bison@001",
      Instances:  []*structpb.Value{instance},
      Parameters: params,
   }
   resp, err := c.Predict(ctx, req)
   if err != nil {
      log.Fatalf("Error 4: %v", err)
   }
  
   respMap := resp.Predictions[0].GetStructValue().AsMap()
   fmt.Printf("resp: %v", respMap["content"])
}

Yup. That’s it!

To further explore, try PaLM for Chat chat-bison for multi-turn conversation use cases. Code completion, take a look at code-bison. And more Generative Model here.