Getting Started: window.ai in Chrome
Imagine having a powerful AI model ready to assist directly in your browser without any hefty downloads. In this demo, we’ll walk through the steps to work with Gemini Nano in Chrome using window.ai.
Background
Chrome has introduced Gemini Nano, an AI model designed for on-device use, along with an API for prompting it. While browser-based models have been around for a while, they come with limitations, particularly the need for users to download large models. This can significantly impact bandwidth and user experience.
Enter window.ai and Gemini Nano. window.ai is the API for communicating with a local model, and Gemini Nano is the advanced model designed to operate efficiently on user devices.
Getting Started
⚠️ ️Attention: Updated 1/8/2025. This article has been updated to reflect the recent changes in the Chrome API. This API is experimental and may undergo significant changes in the future. Please comment and let me know if you’re having any issues 😅
Before we dive in, let’s ensure you can access this feature. As of the writing of this article, it is not available in the main Chrome release (this might have changed by the time you’re reading this). To check if the AI is available, run the following command in the Chrome Dev Console:
(await window.ai?.languageModel.capabilities()).available
If it returns ‘readily,’ you can skip to the section “Using window.ai.”
Enabling window.ai
As of January 8, 2025, Chrome doesn’t support this API in the stable release. You’ll need to download the Chrome Dev build and enable specific flags.
Download the Chrome Dev build here. Once you’ve installed it, open the browser and follow these steps:
Step 1: chrome://flags/#prompt-api-for-gemini-nano
- Select ‘Enabled’
Step 2: chrome://flags/#optimization-guide-on-device-model
- Select ‘Enabled BypassPrefRequirement’
- Restart Chrome
Step 3: chrome://components
- Click ‘Check for Update’ on ‘Optimization Guide On Device Model’ to download the model.
Using window.ai
API Overview
The API is dead simple:
// checks to see if capabilities are supported by browser
await window.ai?.languageModel.capabilities();
// creates session with gemini nano model. Note: options arg optional
const session = await ai.languageModel.create();
// command for executing prompt
await session.prompt("What does NASA stand for?");
// command for executing prompt, but returns a ReadableStream object
await session.promptStreaming("What does NASA stand for?");
// destroys the session
await session.destroy();How to use
Like the API, usage is very straight forward. You simply create a session, and pass your text prompt to session.prompt. You can demo this in the Chrome Dev Console by just dropping the following in:
const session = await window.ai.languageModel.create();
await session.prompt("What does NASA stand for?");That was easy! Check out my working demo. I encourage you to take this further and see if you can implement streaming.
Wrapping up
This article is intentionally brief for two reasons:
- The API is straightforward, so there’s no need to complicate it.
- This feature is highly experimental and may change or never reach the main Chrome build. It’s also specific to Chrome, though other browsers might implement similar APIs with their own models.
As a next step, you can use this API to build a voice assistant. Check that out here.
I do encourage you to dive deeper into the topic. If you’re interested in learning more about window.ai, Gemini Nano, or using models in the browser, check out the following resources:
To stay connected and share your journey, feel free to reach out through the following channels:
- 👨💼 LinkedIn: Join me for more insights into AI development and tech innovations.
- 🤖 JavaScript + AI: Join the JavaScript and AI group and share what you’re working on.
- 💻 GitHub: Explore my projects and contribute to ongoing work.
- 📚 Medium: Follow my articles for more in-depth discussions on LangSmith, LangChain, and other AI technologies.
