Having Trouble Keeping Up with Group Chat?
Read our paper or try out our new Slack tool, Tilda! Paper: Making Sense of Group Chat using Collaborative Tagging and Summarization. Slack Tool: tildachat.com
Is Slack eating email? Lately, group chat tools like Slack, HipChat, and Microsoft Teams have gotten more popular for team and workplace communication. People say they prefer group chat over email because it’s more immediate, more personal, and more casual. People also point out that it’s just more fun, with plenty of chit-chat, jokes, gifs, and emojis.
But there have been complaints regarding group chat as well. One issue is that chat is suited for real-time communication, where people can exchange lots of back-and-forth quickly. This means that people who miss out might come back to many messages to go through. Some of those might be important but given the casual nature of chat, many of them will likely not be important. Distinguishing important from unimportant chat can be hard since they all look the same while scrolling around. The problem gets even worse if someone is, say, coming back from vacation or is a complete newcomer.
In our research, we examined this problem — of catching up in group chat — through a series of studies. We talked to people who use group chat regularly to understand their problems with catching up. We created a series of mock-up interfaces for summaries of chat to learn about what people find useful for catching up.
From these findings, we developed a Slack app called Tilda for people to easily mark up their chat with a number of signals that then generate summaries. The tool uses features like emoji reactions and slash commands in Slack to make it easy to mark up chat while chatting. So far, we’ve tested out the tool in a number of experiments and deployments to real teams, with encouraging results. If you’re curious, try out the tool for yourself: https://tildachat.com.
Catching Up On Group Chat
In interviews with people who use group chat actively, we found many who experienced difficulties with catching up in group chat:
1) Despite an “always on” mentality, people still fell behind.
I think there’s a lot of content that I don’t need to consume. I’ve read [that] content switching is distracting and bad for productivity…But I hate having unread notifications.
Almost everyone we talked to had their group chat app open the entire day and checked it continuously. This echoes prior reports that Slack users have the app open on average 10 hours a day per weekday. Some also admitted to checking chat while on vacation. Despite all this effort to stay up-to-date, almost everyone we talked to also described falling behind due to things like too many conversations going on or too many channels to keep up with.
2) Catching up on group chat is hard.
Scrolling is basically the big issue, which is that you’ve got this giant timeline of stuff…You can only scroll and skim back through so many views on the viewport before you start getting tired of looking.
The way most interviewees caught up with group chat was to just scroll up in their chat window. This was hard because all the conversation looked the same at a glance, and important things were interspersed with chit-chat and humor. Other interviewees sometimes just gave up and ignored missed messages, expecting that important things would find them eventually. This also led to problems where people missed out on important messages. As a sender knowing this, people would post the same thing to different places like in both email and chat, and the conversations that played out in these different places would then be hard to trace.
3) Attempts to organize or synthesize chat haven’t worked well.
Acknowledging difficulties with catching up, some people described trying to start processes for organizing or synthesizing chat, like setting up a wiki, collaborative doc, or contributing to a Q&A forum. However, people would fail to update these separate applications, finding the work a documentation chore. People would also forget about them because the summaries were so separate from the chat application.
Designing a Chat Summary Interface
We build some mockups of different ways that chat conversations could be summarized to get a sense of what people would find useful. As you can see below, we varied the mockups to highlight different information and presentation formats, from A) short written summaries, to B) excerpts from the chat, to C) major types of discussion happening, such as Q&A or an announcement, to D) high level signals like participants and topics.
We found overall that people preferred formats that were highly structured, as opposed to free-form text, so that they would be easier to skim. At the same time, short excerpts weren’t that helpful on their own because they often missed useful context to understand what happened. Markers like the major types of discussion were helpful for gaining some of that context.
We also heard from people that they still wanted to read the original discussion for certain things that were interesting to them. So any summary should make it easy to dive in to read more as opposed to trying to be a one-stop static shop.
Tilda: A Tool for Marking Up Chat to Generate Summaries
From the above explorations, we build Tilda, a tool for marking up chat in Slack. The way Tilda works is that while having a conversation, you can mark up the ongoing chat with information, like whether it’s a question or what the topic of the conversation is. The ways you can do this are lightweight and integrated into the chatting experience, including adding an emoji reaction to a particular message to tag it or adding a short note using a slash command in the text box. The work is also collaborative — everyone can pitch in to mark up conversation.
Tilda then takes the information that you add and generates summaries of conversations. The summaries live inside Slack as well, with each note or tag in the summary linking back to where it happened in the chat. Additionally, any edits to the markup in the original chat get automatically reflected in the summary, wherever it’s posted.
You can choose to get a summary delivered to your direct messages with Tilda (where you can customize the channels, summaries, and people you want summaries from), or you can designate public “summary channels” where people can get summary notifications all in one place.
Check out our Best Paper awarded at CSCW 2018, where we report promising results from a number of lab studies comparing Tilda with using Google Docs for keeping notes while chatting in Slack, as well as deployments with real Slack teams, including 2 software startups, 1 journalism team, and 1 research group. Here are some quotes from our deployment:
“We really didn’t have a good system…Tilda made it muuuuch easier for us to fill someone in on something that happened…Overall I think Tilda greatly improved team communication over the week we used it. Conversations had better structure, team members were better kept up to date, and we actually had a way to save…results of our conversations for future use.”
“Before Tilda I would try to scroll…This was very tedious…With Tilda this process was much smoother. I would usually check our Tilda responses channel and skim through the summaries to see what I missed. If a topic seemed interesting, I would expand it all and read through everything. If I was uninterested in the topic I would just move on.”
To try out Tilda, head to tildachat.com. You can also run your own version of Tilda or contribute to the open source code at https://github.com/Microsoft/tilda.
Looking Ahead
We’re excited about a number of things regarding this work.
First off, Tilda is currently still in the prototype phase, so some exciting features don’t exist quite yet. For instance, you could imagine Tilda summaries being used for additional purposes. They could interface with calendar apps, project management apps, or any number of other apps so discussions automatically lead to actions without needing to remember all the different integrations. They could also exported to a separate repository outside Slack, to aid with writing up reports, deeper searching, newcomer integration, or longer-term organization.
Another aspect is the idea of using these chat logs with rich markup towards training machine learning models to help with summarization. Right now there’s a lack of annotated data to help with the task as well as underspecification of what summarizing chat really means. Our work takes important steps in this direction by: 1) describing what makes a good chat summary according to our mockups, 2) breaking down the summarization task into a set of concrete smaller tasks that can be chipped away at by more automated techniques, and 3) providing a simple mechanism and also motivation for people to mark up chat and create data.
