Get GPT-3 To Read Your Slack, So You Don’t Have To

I built a prototype to gather Slack’s unread content and pipe it into OpenAI in order to summarize what you’ve missed. Here’s how it works.

Hi, I’m Taylor Hughes. I’m a software engineer. I have shipped apps and built teams at Facebook, Google, Clubhouse and a bunch of start-ups in between.

I am convinced that the way we deal with inbound workplace communications is unsustainable. My everyday work-life on larger engineering teams has turned into a game of notifications whack-a-mole. I hope that LLMs will help us deal with the onslaught of workplace communications some day.

So I built a prototype that takes all your unread Slack messages and pipes them into GPT-3 for summarization, so at least you have less to read. I chose not to use the official Slack API (for reasons), and this meant I got to learn all about puppeteer and the world of web scraping.

The result is Unread Buddy, an Electron app that runs a browser window with your Slack client in it, sucks out all the unread messages from that, and sends them off to OpenAI for summarization. (There’s no backend / server side for this app — you have to BYO OpenAI API key if you want to try it. The app talks directly to the OpenAI API from the local machine.) This app is not a great app to use, but it’s a proof of concept to show how summarizing Slack conversations might work.

Here are my main takeaways from building this prototype over ~2 weeks:

• It was hard to get all of the Slack unread messages together in one place! This is partially because of the way I chose to do it — web API scraping — and partially because the Slack API separates channels, threads and IMs/“MPIMs” into different API concepts. There’s just a lot of different data you need to bring together to make this work.
• In terms of the actual output of the app, one-sentence summaries were pretty useful for conversations with about ~500 words. But for shorter or longer amounts of conversation, it’s not a great fit: Shorter conversations are just obscured by the summaries, whereas longer conversations lose nuance and important detail.
• It was a pain in the ass to get GPT-3 to reliably include specific tokens in the response in a reliable way. (In this case, user IDs.) So formatting “Joe said X, Jane said Y” nicely involves a lot of regex.
• Signing a Mac app for distribution is crazy hard! Feels like building for iOS 2.0. This guide helped, but I had to use as bunch of extra entitlements and notarization options not mentioned here.

How It Works

Electron with TypeScript, Remix and Puppeteer

The project was made with remix-electron, which configures Remix to run in Electron. This was my first time with both Electron and Remix, but overall it worked pretty great. Remix reminds me of my own weird framework, with file-based routes and everything.

The most confusing part about using Electron for this was having to make the split between “server side” and “client side” in a native app — Electron is a browser, and something has to render the initial HTML (the “server side”) that runs in that browser view (“client side”).

Since the app is driving puppeteer and firing off a bunch of async work in the “server side”, the “client side” needs to refresh a bunch and reload UI state. I used a meta refresh tag to hack this for awhile, and later adapted it to provide an actual “API” to the state of the scrape & summarization.

A normal desktop Mac app would be able to hook directly into the long-running browser process and update the UI directly instead, which would be way less confusing.

Listening to the Slack API, Making Some Additional Calls

The Slack web app (https://app.slack.com/) makes a metric ton of API calls when it starts up. There are two important ones for our purposes: client.boot and client.counts. The boot API response contains an overview of the workspace, the main channels and usernames to render in the UI. The counts API response gives you the unread counts for all the channels and IMs.

I originally planned on just loading the “Unreads” page in the Slack app, and listening to the API calls that resulted passively, without having to issue any additional calls myself. But I discovered two issues: threads are not loaded within the “Unreads” page — you have to go to the “Threads” page — and only the first page of conversations are loaded from both views if you just load the page without scrolling.

So I decided to just issue the queries myself instead. I call threads.getView for the threads list, and then conversations.history is the API for loading each channel’s content.

With all of those things cobbled together, along with querying for user info (users/list and users/info) to get user names, I could finally print out the unread content in order to summarize it. This took by far the longest of any part of this prototype to get working.

Prompt Engineering

By this point I had a Slack conversation in a text format like:

taylor: Hey, quick question
taylor: Wondering if we can throw all the existing plans out the window and just ship a totally different product to production than what we talked about in the meeting last week
joel: wait, what?
taylor: @joel yeah, I heard from Sales that nobody wants the thing we planned to ship, cc: @brent
joel: …

Given this format, we want to get OpenAI to summarize it. It was surprisingly easy, and I used a prompt like:

Given the following chat history:

taylor: …
joel: …

Generate a summary of the conversation, including main points and who made them. Do not modify the user names. Summarize into 1 concise sentence:

This worked pretty well, and OpenAI would return with a result like:

Taylor said sales wants to upend all existing plans and ship something new, and Joel was surprised.

Then the hard part came: I wanted to bold “Taylor” and “Joel” in the above conversation inside the UI. So I switched it to send user IDs instead of names, like “U1234” and “U5032” — this is actually nicer from a privacy standpoint as well, because then we don’t reveal anyone’s real names to OpenAI.

But GPT-3 would respond with things like:

• User 1234 said…
• U 1234 said…
• Users 1234 and 5032 disagreed about…

So it became really difficult to find-and-replace these user IDs in the output in order to bold the names. In order to get this to work, I had to add a little crazy regex to deal with all the possible things GPT-3 could morph the usernames into. Seems like a pain that everybody will have to deal with as a result of calling into LLMs.

Conclusion

Overall, building the prototype was really fun. But I don’t think the product is compelling: I can’t really trust the GPT-3 summary of the Slack conversation most of the time, so I still need to see the underlying Slack messages anyway. It’s not exactly saving my time that way.

But I do hope I can reuse this scraping code for something else more useful!