Get Started with Chrome Built-in AI : Access Gemini Nano Model locally
An AI model within your browser. Yes that’s exactly what Chrome Built-AI is all about. The team has been at work to infuse AI APIs and models inside of Chrome via Chrome’s built-in AI APIs and models, including Gemini Nano.
This would mean that you can invoke a Gemini model locally in the browser. Thinking of giving that Chrome extension of yours, some AI superpowers and yet running it locally? You should give Chrome Built-in AI a try.
In this article, we are going to look at:
- Chrome Built-in AI capabilities at the time of writing (likely to have breaking changes as its early days).
- Key Configuration steps to getting started with Chrome Built-in AI.
- Sample code that demonstrates Sentiment Analysis feature that you can build within your web application and it uses the Gemini Nano model integrated within Chrome AI via the Prompt API. All processing is done locally.
- Join the Google Chrome Built-in AI Challenge and win prizes
Chrome Built-in AI capabilities
Note: It’s an Early Preview program and this means that you will face breaking changes across releases, APIs that could change and more. Be ready for that.
At the time of writing, the following built-in APIs are available in the preview program along with the links to their specifications. Each of these links describes the feature, how you can configure/enable it in the browser, sample code to see the feature in action and a playground.
You can refer to the APIs page to understand the current status of each of these APIs, exposed to Chrome Extensions and more.
Suggestion: Start with the Prompt API, that allows you to send across a natural language query to the local Gemini Nano model in the browser.
Getting Started with Chrome Built-in AI
The Chrome Built-in AI is currently an Early Preview program and ideally you should go through the Getting Started post on this, that highlights the background, need for these capabilities and more. This will set you up with solid grounding on what these features are all about, why they are important and the possibilities with it.
If I may summarize the key points about what Chrome Built-in AI is about, I will reproduce a part of the article here:
Chrome Built-in AI is “web platform APIs and browser features designed to integrate AI models, including large language models (LLMs), directly into the browser. This includes Gemini Nano, the most efficient version of the Gemini family of LLMs, designed to run locally on most modern desktop and laptop computers. With built-in AI, your website or web application can perform AI-powered tasks without needing to deploy or manage its own AI models.”
At a high level, the architecture looks like this:
(Optional Step) Join the Early Preview program
The first step would be to join the early preview program, so that you get a welcome email and are alerted about updates as they happen. This is not a necessary thing to start using these features, as we will see in a while.
Steps to setting up the Prompt API
I am using the steps provided in the Prompt API document to get started. Keep in mind, that there will be specific steps required for the other APIs that are highlighted/mentioned in the previous section and for which links were provided. But for now, this will suffice.
- There are specific hardware requirements (available for desktop platform only at this moment) to run this. I produce it from the document:
2. Install Chrome Canary. At the time of writing, you will need version 128.0.6545.0 or above.
3. Enable Gemini Nano and the Prompt API : There are a couple of flags to set for this (do this in Chrome Canary):
- Go to chrome://flags/#optimization-guide-on-device-model. Select Enabled BypassPerfRequirement
- Go to chrome://flags/#prompt-api-for-gemini-nano. Select Enabled
- You might also see other capabilities in the settings like Reader , Writer API, Summarization API, etc. If you’d like, Enable them too.
- (Important) Relaunch Chrome at each step.
A sample screenshot from my Chrome settings for the above setting parameters is shown below:
4. Confirm availability of Gemini Nano : This step involves opening up the Chrome DevTools and I will produce the instructions from the guide below.
There is a troubleshooting guide too and I strongly recommend that. I struggled a bit first with Chrome Dev and then moved on to Canary and got better results. Keep in mind that it needs to download the Gemini Nano model as shown in step 2 of the screenshot above and this took about 3–4 minutes for me.
If all looks good, you are all set to try out the Prompt API and you can visit the Prompt API playground. Run it in the Chrome Canary browser, where you have done all these settings so far and ensure that it works.
Sample Application using Prompt API
The application that I wanted to demonstrate was to do sentiment analysis via the Gemini Nano local model in the browser.
So I start off with a sample page that has a bunch of reviews posted for a specific blog post as given below:
We have a Analyze Sentiment
button at the top. What I want this to do is the following:
- For each review, take the Review Text.
- Invoke the Chrome Built-in AI Prompt API to do a sentiment analysis
- Write the Sentiment analysis done by the LLM in the
Sentiment
column
At a high level, the code for the Prompt API is as follows:
my_session = await ai.languageModel.create({
systemPrompt: "You are an expert reviewer of comments
who analyses and identifies what attributes
were liked or disliked in the review."
});
const response = await
my_session.prompt("Analyze the review for
sentiment and identify in a list
what was liked and disliked." + reviewText);
On running this, we see a sample run below:
The entire code for the page is shown below:
The full source code for the repository is available over here:
Google Chrome Built-in AI Challenge
The Google Chrome Built-in AI Challenge details are available here.
If you have an idea for a web application or Chrome Extention that uses one of the built-in AI APIs, consider participating in it. Its currently on and you need to submit your entries by Dec 4, 2024 and has some fantastic prizes.
This would be a good way to try our the Built-in AI features in Chrome and if you’d like submit the entry to the contest.
Get Inspired
I’d like to share a few other videos that I have found useful and ones that could spark some ideas. Take a look.
First up is Addy Osmani’s video that hgihlights the various APIs. A great way to understand what is available.
Next up, check out Pascal Birchler’s fantastic blog post on how Web AI can be applied to WordPress for a range of activities from summarization, helping you write better and more.
Happy Building with Chrome Built-in AI. If you’ve come across some blog posts, videos that show interesting ways in which developers are integrating Chrome Built-in AI, do share in the comments.