Crit-Bot Hotline
Call today and get an AI safeguard for free
Part of a series on prototyping with generative AI
Moving chatbots out of chat and into phones can get interesting. People tend to have colorful opinions about robocalls, but maybe a gpt4 phone call can hold someones interest in a whole new way? Also, if we make a publicly accessible bot, how can we set up some simple safeguards when an AI model has none? This article explains how to prototype a Crit-Bot like the one mentioned in this article written by Savannah Kunovsky and myself. Test it out…
Call (870) 399–0391 to get your very own critique!
If you’re just here for generative AI tips, scroll down to Hear No Evil and check out how to use OpenAI’s FREE moderation endpoint.
Setting up our hotline bot
We are going to be using Dialogflow to set up our basic phone line. We need to specifically set up a Dialogflow CX project and create a new agent (chatgpt gives great setup instructions if this is unfamiliar). When we create our agent, WE MUST SET IT’S LOCATION TO GLOBAL to enable usage of Dialogflow CX’s Phone gateway. Don’t miss that step!
Now we start using Dialogflow CXs flow map. We need to set up a couple pages and intents. We can start with Hello. We can use the Test Agent pane as we go to see how its progressing.
Our agents Start page wants to look something like the above image. It uses a Default Welcome Intent where we give it a few ways to say “Welcome to the hotline.” Immediately after the welcome statement the conversation is automatically routed to the critique page. The boolean true route does this. It’s a common pattern in CX templates.
Agent: Welcome to the Critique Hotline
In this section we immediately ask them to “Share your idea or insight.” Then we wait for our users reply. It’s a phone call, so we need to tell our agent how long to sit in silence before it asks again (event handler: sys.no-input-default). Go to Agent settings > Speech and IVR > No Speech Timeout and set a reasonable time (I set mine to 7 seconds).
Share your idea or insight. <waits 7 seconds>
Now for a bit of a hack. We take any statements that don’t match “no thank you” and send them to openai via fulfillment. First, we need to set up and serve a webhook. Below is an example of Dialogflow CX fulfillment as a node-js app. I used a firebase function, node-js, set as “type”: “module” in package.json. It’s much simpler if the function is in the same Dialogflow project in google cloud.
Add an openai function to our app along with the following system command:
You are a very sarcastic wine drinking snobby design critic. You are especially hung up on poor use of words and tired and common phrases. Your remarks are always cutting, very short and consise, and to the point. After you brutalize someone with your critique, you offer suggestions and make one very short positive comment. If possible.
import { Configuration, OpenAIApi } from 'openai';
const configuration = new Configuration({
apiKey: "XXXX-XXXX-XXXX"
});
const openai = new OpenAIApi(configuration);
...
// in this instance the function is part of an exported module named chat.js
async getAI(msg) {
console.log('TO OPENAI >>> \n'+msg);
messages = [
{"role": "system", "content": "You are a very sarcastic wine drinking snobby design critic. You are especially hung up on poor use of words and tired and common phrases. Your remarks are always cutting, very short and consise, and to the point. After you brutalize someone with your critique, you offer suggestions and make one very short positive comment. If possible."},
{"role": "user", "content": msg}
]
const response = await openai.createChatCompletion({
model: "gpt-4",
messages: messages
});
return response
}Then build that call into the fulfillment webhook itself:
// This is set up as a firebase function in the SAME PROJECT as our dialogflow CX service
import functions from 'firebase-functions';
import chat from './chat.js';
export const dialogflowFulfillment = functions.region('us-east1').https.onRequest((request, response) => {
// first check text, then check voice-to-text
const msg = request.body.text? request.body.text: request.body.transcript;
const tag = request.body.fulfillmentInfo.tag;
return new Promise(async (resolve, reject) => {
// Tag contains the "Intent" name
if (tag) {
let text = "";
if (tag === "Critique") {
const aiRes = await chat.getAI(msg);
text = aiRes.data.choices[0].message.content;
}
console.log('REPLY:'+text);
const jsonResponse = {
fulfillment_response: {
messages: [
{
text: {
text: [text],
},
},
],
},
};
response.send(jsonResponse);
resolve();
} else {
reject("No intents tagged");
}
}).catch((err) => { console.log(err); })
});Next, we go to manage > webhooks > + create to point Dialogflow towards our new webhook. On the creation screen we need to be generous with “Webhook Timeout” and set it to 30 seconds (not possible with Dialogflow Essential). Now, back in the build map, select the critique page, and click on the event: sys.no-match-default. That triggers when Dialogflow can’t match the statement to any of its expected intents (how it categorizes user input). Toggle on “Enable webhook” for fulfillment and select the webhook we just created (I named it “Our New Webhook”). The “Tag” field helps the webhook determine the type of content Dialogflow needs.
User: “Meet people where they’re at” or “Never has a bluer cow walked more finely than under the pressure of a true moon.”
You may have noticed I toggled on “Return partial response.” By doing so, you can put some instant replies in your sys.no-match-default event. This is important because on the phone, they may state their idea and then its just silence for sometimes dozens of seconds. They’ll think its broken, but those solid GPT-4 replies just take awhile… for now.
Agent: Give me a minute… <waits 30 seconds for openai response>
<12 seconds later>
Agent: <the AI’s stylized critique!>
We could hang up, or ask them if they want to continue.
Agent: Shall we go another round? Yes or no.
Nothing fancy here. Give them a Yes or No option to go again, or end the call. If they pick yes, route them back to the critique page where it will repeat one of its introductory phrases: “Share your idea or insight.” At this point the basic dialog… flow… is done.
Turn On The Phone
To put this experience on a phone we go to manage > integrations > CX Phone Gateway and click the manage button (only available if the Dialogflow Agent we’re using was set up as global). Click create and select the available country code, then click request. Once it gives us a phone number, it should be ready to take calls!
Hear No Evil
Ok, we built a working Generative AI Critique Hotline! We’re about to share the phone number with the entire lovely internet, and we want our bot to keep out of trouble. This prototype uses GPT-4, which does a decent job at replying to disturbing prompts due to its internal safeguards, so we could just let it reply? Risky practice. We already have access to LLMs with fewer or even no safeguards (see Based GPT), so we shouldn’t always assume guardrails exist.
We are going to implement our own guardrails, even for quick prototypes. To do this we can send a caller’s transcription to Openai’s moderation endpoint. It’s quick and importantly, they make it available for FREE!
Add the following function to our app:
// Send content to the FREE openai moderation endpoint
async contentIsSafe(msg) {
const res = await openai.createModeration({
input: msg
});
let response = res.data.results[0].flagged;
if (response === true) {
// FLAGGED, List off categories and their flag status
console.log('FLAGGED STATEMENT:\n'+res.data.results[0].categories);
}
return !response;
}Back in our fulfillment webhook we can update the “Critique” handler to first pass transcribed messages through our new contentIsSafe() function.
// export const dialogflowFulfillment = functions.region('us-east1').https.onRequest((request, response) => {
// ...
if (tag === "Critique") {
// Check that the user message is safe to reply to
const safe = await chat.contentIsSafe(msg);
if (safe) {
// It's safe, REPLY!
const aiRes = await chat.getAI(msg);
text = aiRes.data.choices[0].message.content;
} else {
text = "That statement was flagged by the moderation api.";
}
}
// ...
//});The bot implemented on the (870) 399–0391 number at the beginning of the article uses this “hear no evil” method. It adds roughly .5 seconds to the fulfillment roundtrip. Most of the delay is just waiting for the GPT-4 reply.
Or… Speak No Evil
Another way we might handle this: send both the prompt and our AI response to the moderation endpoint right before replying to our caller. Just in case our bot’s “shadow self” sneaks in.
That’s it for this prototype. Stay safe out there
