Using Vertex AI Gemini from GAPIC libraries (C#)

Introduction

In my previous Using Vertex AI Gemini REST API post, I showed how to use the Gemini REST API from languages without SDK support yet such as C# and Rust.

There’s actually another way to use Gemini from languages without SDK support: GAPIC libraries. In this post, I show you how to use Vertex AI Gemini from GAPIC libraries, using C# as an example.

What is GAPIC?

At this point, you might be wondering: What’s GAPIC? GAPIC stands for Google API CodeGen. In Google Cloud, all services have auto-generated libraries from Google’s service proto files. Since these libraries are auto-generated, they’re not the easiest and most intuitive way of calling a service. Because of that, some services also have hand-written SDKs/libraries on top of GAPIC libraries.

Specifically for Vertex AI and Gemini, there are hand-written SDKs for Python, Java, Node.js, and Go. You’d prefer these hand-written SDKs over GAPIC versions.

For other languages, you can either rely on the REST API or you can use the GAPIC library for that language, if it exists.

REST vs. GAPIC

As an example, in C#, there is a GAPIC library for Vertex AI. It’s mostly through the PredictionServiceClient ofGoogle.Cloud.AIPlatform.V1 library. For other languages, you'd search for similarly named classes and libraries to see if there's GAPIC support.

When you have the choice to use the REST API or the GAPIC library, which one should you use? It’s really up to you but keep in mind:

• The REST API is probably more intuitive but you need to handle authentication and you need to parse the returned JSON to strongly typed objects on your own.
• The GAPIC library provides authentication and strongly typed classes but you need to figure out how to construct the right protobuf classes, so it can be a little unintuitive.

Gemini REST API from C#

As a reminder, in my Using Vertex AI Gemini REST API post, I showed how to use the Gemini REST API from C# to describe an image of a cat. The full sample is in my GitHub repo in GenerateTextFromImageGcs.cs.

Gemini GAPIC library from C#

Now, let’s build the same sample but this time using the GAPIC library.

Define prompt and image

Define the prompt and the image stored on Cloud Storage:

string text = "Describe this image in detail in a few paragraphs";
string imageUrl = "gs://cloud-samples-data/generative-ai/image/320px-Felis_catus-cat_on_snow.jpg";
Console.WriteLine($"Text: {text}");
Console.WriteLine($"ImageUrl: {imageUrl}");

It’s an image of a cat 🙂

While we’re at it, also define some constants we’ll need later:

const string ProjectId = "genai-atamel";
const string Location = "us-central1";
const string Publisher = "google";
const string Model = "gemini-pro-vision";

Create a prediction service client

Create a PredictionServiceClient that you'll use to make requests to Vertex AI:

// Create client
var predictionServiceClient = new PredictionServiceClientBuilder
{
    Endpoint = $"{Location}-aiplatform.googleapis.com"
}.Build();

Construct the request payload

Next, construct the request payload with the right parameters. This can be tricky to get right:

// Initialize request argument(s)
var content = new Content
{
    Role = "USER"
};
content.Parts.AddRange(
[
    new() {
        Text = prompt
    },
    new() {
        FileData = new() {
            MimeType = "image/png",
            FileUri = imageUrl
        }
    }
]);

var generateContentRequest = new GenerateContentRequest
{
    Model = $"projects/{ProjectId}/locations/{Location}/publishers/{Publisher}/models/{Model}",
    GenerationConfig = new GenerationConfig
    {
        Temperature = 0.4f,
        TopP = 1,
        TopK = 32,
        MaxOutputTokens = 2048
    }
};
generateContentRequest.Contents.Add(content);

Send the request and parse the streaming response

Sending the request is a simple call:

// Make the request, returning a streaming response
using PredictionServiceClient.StreamGenerateContentStream response = 
    predictionServiceClient.StreamGenerateContent(generateContentRequest);

Note that the authentication is handled for us, so we don’t need to do anything special to pass the auth token or anything like that.

The response is streaming, you can print it out as it comes back:

Console.Write($"Response: ");

// Read streaming responses from server until complete
AsyncResponseStream<GenerateContentResponse> responseStream = response.GetResponseStream();
while (await responseStream.MoveNextAsync())
{
    GenerateContentResponse responseItem = responseStream.Current;
    Console.WriteLine(responseItem.Candidates[0].Content.Parts[0].Text);
}

You can see the full sample in my GitHub repo in GenerateTextFromImageGcs.cs.

Run the sample

Run the sample:

dotnet run

Prompt: Describe this image in detail in a few paragraphs
ImageUrl: gs://cloud-samples-data/generative-ai/image/320px-Felis_catus-cat_on_snow.jpg
Response:  This is an image of a cat in the snow. The cat is standing on a
snow-covered field, and it appears to be looking off to the side. It is a
short-haired tabby cat with a white belly and paws. The cat's fur is brown and
black, with some white patches on its chest and belly. The cat's eyes are
yellow, and its nose is pink. The cat's tail is long and fluffy, and it is
curled up at the end. The cat's body is slightly hunched, and its ears are
perked up. The cat's fur is matted and dirty, and it appears to be in poor
condition. The cat's surroundings are covered in snow, and there are no other
animals or people visible in the image.

You’ll realize that the response will stream as it’s available. That’s nice.

Summary

Ultimately, the language of your choice determines what SDK/library to use to talk to Gemini on Vertex AI:

• Hand-written SDKs for Python, Java, Node.js, Go.
• GAPIC libraries for other Google Cloud supported languages like C#.
• REST API for everything else.

As always, if you have any questions or feedback, feel free to reach out to me on Twitter @ meteatamel.

Originally published at https://atamel.dev.