Implementing Function Calling using Genkit

About this article

Genkit was announced at Google I/O 2024. In this article, I introduce how to implement Function Calling using Genkit.

What is Genkit?

Genkit is an open-source framework designed to assist in the development of applications equipped with generative AI features. For those familiar with LangChain, you can think of Genkit as a framework that provides one interface for multiple generative AI models, much like LangChain.

Genkit is developed by the Firebase team, and its framework interface is user-friendly for Firebase users.

The programming languages supported are JavaScript, TypeScript, and Go.

What are the advantages of using Genkit?

One of the biggest advantages of using Genkit is that it allows us to use Developer tools locally. By running the following command:

genkit start -o

then, the Developer tool will launch on localhost.

Genkit local emulator

Developing generative AI features currently requires significant focus on the AI orchestration. Using Developer tools makes the development the AI orchestration much more faster.

Genkit is developed by the Firebase team, so it is designed to integrate easily with Firebase services. You can easily implement things like Firebase Authentication and the newly announced vector search feature in Firestore for implementing RAG.

Implementing Function Calling

I implemented a feature to scrape the web page and summarize content in Japanese. Although the entire code is presented below, it can be implemented in about 50 lines of code. I will explain each key part below.

Full Source Code

import { defineTool, generate } from "@genkit-ai/ai";
import { configureGenkit } from "@genkit-ai/core";
import { defineFlow, startFlowsServer } from "@genkit-ai/flow";
import { gemini15Pro, googleAI } from "@genkit-ai/googleai";
import * as cheerio from "cheerio";
import * as z from "zod";

configureGenkit({
  plugins: [googleAI({ apiVersion: ["v1beta"] })],
  logLevel: "info",
  enableTracingAndMetrics: true,
});

const webLoader = defineTool(
  {
    name: "webLoader",
    description:
      "When a URL is received, it accesses the URL and retrieves the content inside.",
    inputSchema: z.object({ url: z.string() }),
    outputSchema: z.string(),
  },
  async ({ url }) => {
    const res = await fetch(url);
    const html = await res.text();
    const $ = cheerio.load(html);
    $("script, style, noscript").remove();
    if ($("article")) {
      return $("article").text();
    }
    return $("body").text();
  },
);

export const summarize = defineFlow(
  {
    name: "summarize",
    inputSchema: z.string(),
    outputSchema: z.string(),
  },
  async (url) => {
    const llmResponse = await generate({
      prompt: `First, fetch this link: "${url}". Then, summarize the content within 300 words in Japanese.`,
      model: gemini15Pro,
      tools: [webLoader],
      config: { temperature: 1 },
    });
    return llmResponse.text();
  },
);

startFlowsServer();

configureGenkit

In Genkit, initial settings are made using the configureGenkit function, similar to Firebase's initializeApp. To implement Function Calling, it appears necessary to specify apiVersion as v1beta for now.

The logLevel is set to info, but debug can be used during debugging for detailed logs.

configureGenkit({
  plugins: [googleAI({ apiVersion: ["v1beta"] })],
  logLevel: "info",
  enableTracingAndMetrics: true,
});

defineTool

Define the function used for Function Calling with defineTool. Genkit allows for specifying the types of input (inputSchema) and output (outputSchema) values using zod. In this case, cheerio is used to parse web content.

const webLoader = defineTool(
  {
    name: "webLoader",
    description:
      "When a URL is received, it accesses the URL and retrieves the content inside.",
    inputSchema: z.object({ url: z.string() }),
    outputSchema: z.string(),
  },
  async ({ url }) => {
    const res = await fetch(url);
    const html = await res.text();
    const $ = cheerio.load(html);
    $("script, style, noscript").remove();
    if ($("article")) {
      return $("article").text();
    }
    return $("body").text();
  },
);t

The functions implemented with defineTool can be shown in the Tools menu, as shown in the screenshot below, allowing for the individual testing of each tool.

Genkit Tools menu

defineFlow

In Genkit development, callable functions are implemented as Flows. They are callable in both local and remote environments such as Firebase.

export const summarize = defineFlow(
  {
    name: "summarize",
    inputSchema: z.string(),
    outputSchema: z.string(),
  },
  async (url) => {
    const llmResponse = await generate({
      prompt: `First, fetch this link: "${url}". Then, summarize the content within 300 words in Japanese.`,
      model: gemini15Pro,
      tools: [webLoader],
      config: { temperature: 1 },
    });
    return llmResponse.text();
  },
);

The functions implemented with defineFlow can be accessed from the Flows menu, as shown in the screenshot below, allowing for local execution.

Genkit Flows menu

Actual Behavior

Here is a screenshot of the actual behavior.

You can follow the execution process of the Flow in detail from the inspect tab. This allows for detailed debugging to see if Function Calling was executed correctly.

Genkit inspect tab

Summary

Using Genkit, it was straightforward to implement Function Calling. The ability to implement this in about 50 lines and easily deploy to Cloud Functions for Firebase is very appealing.

The source code can be accessed from the following link for your reference.

GitHub - tanabee/genkit-summarize-webpage