Bimpe AI: How We Built an AI Agent for WhatsApp
Version one of BimpeAI functioned as an automated personal task chatbot on WhatsApp. We chose WhatsApp due to its significant market share in messaging, prompting us to forgo creating our native chatbot. This decision led us to our current focus: empowering restaurants to process orders and manage customer information.
My co-founder, Elisha, has tasked me with abstracting this article to avoid revealing too much about our technology. As a result, I will focus less on technical documentation and more on creating a product guide. If this gains traction, Elisha may consider creating a technical version as part 2.
What is an AI Agent?
I consider an AI agent to be an automated engine capable of independent thought and task execution. Perhaps incorporating natural language processing makes it even more impressive, but that’s my general definition for this article. When you think of robotic automation, do UI Path or Zapier come to mind? That’s what we included in our first version.
We developed a chatbot that seamlessly integrates with your email and calendar, processes emails, provides summaries, composes emails in natural language (including voice recording), and takes notes. Here’s a video demonstrating how it works.
How do you put an AI on WhatsApp?
There is a straightforward tutorial on creating a WhatsApp Business account here. This reduces the explanation to some easy steps.
Generate an access token from the WhatsApp API setup tab, then securely copy and paste it. The access token is initially temporary and needs to be renewed every 24 hours. We used this process for 30 days to test the BimpeAI model until it was ready to be deployed, and then we applied for a permanent access token.
How do you connect your WhatsApp token to OpenAI?
Watch this video if you understand Python —
How to build an agentic flow, how does that work?
To ensure the effectiveness of this agent flow, it is crucial to clearly define the objectives. At Bimpe AI, our focus was on addressing common professional tasks such as composing emails, scheduling calendar meetings, and setting reminders. We initiated the process by creating a flowchart specifically tailored to these tasks.
Functions and actions
We extract functions based on specific keywords to trigger actions. For instance, when the words “Send” and “email” appear together in a string, it indicates the intention to send an email, prompting the use of the Gmail API. Similarly, the word “Book” or “Schedule” in any natural language context triggers the calendar API and initiates the integration. We also identify synonyms and similar words to activate preset actions for individual users.
Edge case scenarios
Natural language can be tricky at times, especially because LLM lacks context understanding and memory. To address this, we engaged our beta testers to explore edge cases and wild scenarios, which revealed valuable opportunities for improvement. For instance, we had to identify 5–10 synonyms and phrases for each function to cover edge case scenarios, such as “book a meeting,” “schedule a call,” “set a time,” and “add to a calendar,” which we then added to our agentic function list.
Memory database
As I mentioned earlier, LLMs can be underwhelming without context. Initially, we had to fine-tune the model to ensure more accurate responses for our specific use case. However, as LLM models became more advanced, it became clear that fine-tuning was a waste of time and resources. Instead, we developed a memory database for each user, allowing the LLM to have context and providing users with personalised experiences with their WhatsApp bot. This database includes information such as names, chat history, past email recipients, and more.
Guard railing
The early version of the bot was initially easy to jailbreak. Despite our attempts to fix this with prompt engineering, the bot continued to revert to its base models after some social engineering role-play. While we didn’t solve this in version one, we eventually found success with a different approach. This topic deserves an article of its own.
How do you connect Google OAuth APIs?
I am going to try not to sound too technical here. Google OAuth can be a challenge to navigate, but you can check out this playlist if you are technical.
We set up some API checklists from the flowchart designed earlier.
- Gmail Write
- Gmail Read
- Gmail send
- Calendar write
- Calendar read
- Reminders
We were able to connect all of these endpoints on the Google OAuth 2 documentation.
Major challenges along the way
Google OAuth limitation
After passing the Tier 2 CASA verification test, our API calls were no longer rate-limited by Google! It did take us about 6 weeks to complete and it did have an impact on the team morale, but we made it through. Kelig’s article was incredibly helpful during this process.
Beta testers raised concerns about privacy
We had 100 beta testers for the MVP, and 90 of them raised privacy concerns about our product and AI in general. They expressed their reluctance to have AI integrated into their emails and calendars, despite Google Gemini’s recent announcement of a similar integration. We considered implementing a data minimisation privacy framework, but we are hesitant to repeatedly explain privacy concerns to every user at scale.
Flawed B2C model
We did a financial projection on how much monthly subscription we’d need to make to run the company properly, build our own custom gpt, and goto market, and it didn’t add up. It means we can’t scale without venture capital.
Final thoughts
We pivoted Bimp AI into B2B after talking to business owners and prioritising responses from business owners in London.
BimpeAI now helps restaurants collect and manage orders directly from their social channels. We help restaurant owners reduce the cost of customer acquisitions and cut platform fees by selling directly to their customers while still having access to their customer information. Bimpe AI makes it easy to connect with existing order platforms as well.
