Why Gemini Web Chat is much better than Gemini API?

So you try Gemini Web Chat for creative writing. It works really really well.

Now you wanna automate it with Gemini API, but the results are about 1/2 as good. Very frustrating…

In this post we will go into the magic behind Gemini Web Chat and how to automate it without going through Gemini API. Let’s go down the rabbit-hole.

Fear and Loathing with Gemini

Recently I had to create an email marketing campaign. I wanted to personalize my emails with Gemini. When using Gemini Web Chat the results were glorious, but when I tried to do the same using Gemini API, the results were much worse. In both cases I used Gemini Pro model and same prompt.

Generate cold email.
Target Audience: Small Startups
Goal:  Get reply
Value Propositon: Find best Developers
Recepient: John Smith, CEO of Acma, Inc.
Keep it spartan and concise.

Gemini Web Chat (Pro) returned:

Subject: Finding Top Talent That Scales Fast, Like Acma

Hi John,

Your success at Acma is inspiring! Did you know finding the right 
developers can be 2x faster with our targeted talent pool & AI matching?

Just a quick question: are you actively growing your dev team?

No pressure, just curious if we could help save you time & land rockstars.

Best,
[Your Name]

But when I did same thing in Gemini API, using request:

POST: https://generativelanguage.googleapis.com/v1beta/models/gemini-pro:generateContent?key=

{
     "contents": [
        {
         "parts":[{
           "text": "My Prompt goes here"}]},
      ]
}

The results were much more inferior:

Subject: Unlock Your Startup's Potential with Top Talent

Dear John Smith,

As the CEO of Acma, Inc., I understand the importance of finding 
exceptional developers to drive your startup's success.

Our platform connects startups with the best software engineers tailored 
to your specific needs. We leverage AI and a global network to provide a seamless and efficient recruitment experience.

We believe that every startup can access the talent they need to thrive.
Let us help you find the perfect match for your team.

Schedule a quick call to learn more: [Insert scheduling link]

Best regards,
[Your Name]

It is not bad, but it is not as concise or direct as the Web Chat. And it doesn’t have much human feel to it. I ran the same prompt multiple times on Web Chat and through API and results are consistently better on Web Chat.

But what About ChatGPT

After playing around with prompt on Gemini and not be able to get good results with API, I turned to ChatGPT.

AND…. I had the same issue. Doing ol’ fashioned google search on the internet I discovered that I was not alone:

Different results: ChatGPT3.5 vs API (gpt-3.5-turbo)

The most common suggestion with ChatGPT is to use System Prompt.

You are ChatGPT, a large language model
trained by OpenAI, based on the GPT-3.5 architecture.
Knowledge cutoff: 2021-09
Current date: {Today's Date}

But that does absolutely bollocks (Pardon my French). So after a couple of days of trying to adjust System/User prompts, I decided to go back to Gemini. For creative writing, Gemini felt much better than ChatGPT.

The Solution to rule them all

In the end, the best solution was to automate Gemini Web Chat using Html Parser. I used Puppeteer and short C# script:

using PuppeteerSharp;

IBrowser browser;
IPage page;


// Start chrome using command line:
// "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --remote-debugging-port=9222 --user-data-dir=/temp
browser = await Puppeteer.ConnectAsync(new ConnectOptions
{
    BrowserURL = "http://127.0.0.1:9222",
});


var pages = await browser.PagesAsync();
page = pages[0];

//Make sure user is logged in and on the right page
if (!page.Url.StartsWith("https://gemini.google.com/app"))
{
    Console.WriteLine($"Go to gemini Chat \"https://gemini.google.com\" and login.");
    return;
}

//Prompt user for what to ask Gemini
Console.WriteLine($"Ask Gemini:");
string prompt = Console.ReadLine();

string responseText = await SendMessage(prompt, page);

Console.WriteLine("\nResponse from Gemini:");
Console.WriteLine(responseText);

Console.WriteLine("\n\nPress any key to exit");
Console.ReadKey();

async Task<string> SendMessage(string prompt, IPage page)
{

    //Enter prompt into UI
    var selector = "rich-textarea > div > p";
    await page.QuerySelectorAsync(selector).EvaluateFunctionAsync("(el, content) => el.textContent = content", prompt);
    await Task.Delay(1000);

    //click button
    selector = "div[class*=\"send-button-container\"] > button";
    var elementHandle = await page.QuerySelectorAsync(selector);
    await elementHandle.EvaluateFunctionAsync("(el, content) => el.click()", "nothing");

    selector = "message-content[class*=\"model-response-text\"]";
    
    //wait for response to appear
    await page.WaitForSelectorAsync(selector);
    
    //It may take a couple of seconds for UI to draw response
    await Task.Delay(5000);

    //get response
    elementHandle = await page.QuerySelectorAsync(selector);
    var response = await elementHandle.GetPropertyAsync("innerText");

    string responseText = null;
    if (response != null)
        responseText = response.RemoteObject.Value.ToString();

    return responseText;
}

Complete code including full console app is here:

GitHub Repository

Here is what happens when you run this console app:

IMPORTANT: Before you run console app, you need to perform the following steps:

1. start chrome using command line:

"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --remote-debugging-port=9222 --user-data-dir=/temp

2. In chrome , go to: https://gemini.google.com/

3. Login to Gemini chat if not already login.

4. Execute console app and it will “puppeteer” your chrome.

Automating generation of Personalized Emails with Gemini Web Chat

Using the code in previous section, I can now create a google sheet that I read with a script and for each row call Gemini Web Chat API.

Works Great!