Use Google Palm with Golang

The python language is very popular. Data science, data pipeline, API backends (…), Python is everywhere. Even on the generative AI studio, Google Cloud proposes CURL and Python code snippets, no other languages!

Despite its ease and popularity, I continue to dislike (and to discourage the usage of) Python for something other than data exploration & data science.
It’s only a personal opinion. I prefer type safe and compiled languages for efficiency (memory, speed, container size,…) and Golang is my favorite playground.

However, Google Cloud proposes only Python code snippets and leaves other languages in the fog.

So, how to use Palm LLM with Go?

Let’s do this.
The following code is extracted from my GitHub repository

Google Cloud API call in Go

As any Google service, an API is exposed to access the service. You can see it in the CURL example displayed in the view code section of Generative AI studio

As any API, there is 3 importants parts

• The HTTP verb. Here it’s POST
• The URL to access the API
• The authentication to the API

The URL building

The URL construction is straight forward, but it’s important to make it dynamic for

• Different projects
• Different regions (only us-central1 is supported for now)
• Different models

func (p *PalmClient) createPalmURL(region string, projectId string, modelName string) string {
 return fmt.Sprintf("https://%s-aiplatform.googleapis.com/v1/projects/%s/locations/%s/publishers/google/models/%s:predict", region, projectId, region, modelName)
}

This URL is generated when the PalmClient is created

The authentication to the API

The authentication part could be tricky. But, hopefully, Google Cloud client library provides a simple way to abstract this complexity by creating a Google Cloud OAuth HTTP client instead of a HTTP default client

import httpGoogle "google.golang.org/api/transport/http"
...
func NewClient(region string, projectId string, modelName string) *PalmClient {
  ...
  ctx := context.Background()
  p.client, _, err = httpGoogle.NewClient(ctx)
  ...
}

This HTTP client is initialized when the PalmClient is created

The JSON request and response

The input/output part is made with JSON request and response when you use the API.
This part can be partly automated generated and required a few adaptations.

The JSON Request

{
    "instances": [
        {
            "content": string
        }
    ],
    "parameters": {
        "temperature": float,
        "maxOutputTokens": int,
        "topP": float,
        "topK": int
    }
}

The Go implementation

type Content struct {
 Content string `json:"content"`
}

type Parameters struct {
 Temperature     float64 `json:"temperature"`
 MaxOutputTokens int     `json:"maxOutputTokens"`
 TopP            float64 `json:"topP"`
 TopK            int     `json:"topK"`
}
type PalmRequest struct {
 Instances  []Content   `json:"instances"`
 Parameters *Parameters `json:"parameters"`
}

The JSON response

{
  "predictions": [
    {
      "safetyAttributes": {
        "blocked": bool,
        "scores": [],
        "categories": []
      },
      "citationMetadata": {
        "citations": []
      },
      "content": string
    }
  ]
}

The Go implementation

type PalmResponse struct {
 Predictions []struct {
  SafetyAttributes struct {
   Categories []string  `json:"categories"`
   Blocked    bool      `json:"blocked"`
   Scores     []float64 `json:"scores"`
  } `json:"safetyAttributes"`
  CitationMetadata struct {
   Citations []any `json:"citations"`
  } `json:"citationMetadata"`
  Content string `json:"content"`
 } `json:"predictions"`
}

Invoke the text generation

Now, you have the main components (URL, Authenticated HTTP client, and request/response object), you can use them to generate text

func main() {

 region := "us-central1"
 modelName := "text-bison@001"
 projectId := "<projectId>"

 prompt := "Write poem on cats"

 palmClient := palm_client.NewClient(region, projectId, modelName)

 // Use the default parameters
 response, err := palmClient.CallPalmApi(prompt, nil)

 if err != nil {
  fmt.Printf("error during the API call: %s\n", err)
  return
 }

 fmt.Printf("The initial prompt is \n%s\n", prompt)
 fmt.Printf("The generated answer is \n%s\n", response.Predictions[0].Content)
 return
}

Replace <projectId> with your own project ID, and customize the prompt as you wish!

Run the code with the command go run .

Conclusion

Generative AI is wonderful tools to use and to integrate in different applications, contexts and pieces of code. Python should not be the only one to benefit from that feature!

Now, you have a code base in Go and you are able to invoke Palm API in 2 lines of code with your own prompt.
This client is not final and requires adaptations if you want to use Chat models or Embeddings feature for instance. But having the main buildings block, it will be easy and simple to adapt them to your own usage.

Enjoy the new gen AI API, in Go, your favorite language!