Multi-modal Gemini Pro Vision Slack chat bot
In my last article, we built a PaLM-based chat bot. But since then, the state of art has moved onwards and it’s time to make some improvements to our chat bot and move it over to Gemini.
We also want to some super powers to the bot, namely multi-modal dialogue support. As always, a fully deployable example is provided.
Deploying the bot
To deploy the bot, check out the code from GoogleCloudPlatform/pubsub2inbox and follow the README instructions.
This time deploying the bot is even easier, as there is now a Slack manifest for getting the permissions correct the first time.
Once deployed and added to your Slack workspace, you can discuss with the app directly or add it to a channel and tag it. Some examples of what it can do:
Normal text conversation
Explaining an image
Explaining a PDF
You can also provide a prompt to be added to a discussion (prompt parameter in Slack processors processMessages part).
Update: Now the bot can also integrate with Vertex Search (or any other REST service) directly using Gemini function calling! See example here: Integrating Vertex AI Search
That’s it — you can customize the bot as you wish. Interested to hear about your use cases! You can reach out to me on X for example: twitter.com/rosmo