How Slack Lost Over 800,000 Messages
*834,000 — to be precise
--
Our team was a small (< 10) group of young developers working together on projects. We grew out of a Skype group chat almost 3 years ago and moved to Slack, which was growing in popularity at the time.
Since May 2014 we’ve been using Slack religiously and have grown to over 30 members — we send over 10,000 messages a week! I can’t recall a day where I haven’t checked the Slack team and caught up with what’s going on.
For the past few months we’ve been looking forward to hitting a milestone of 1,000,000 (1 million!) messages, which is impressive for what is just a small team of friends. The Monday round-up emails in the recent weeks have been something to look forward to and we’ve been checking the statistics page every day.
2 weeks ago an old member of the group re-joined using a new account so we reached out to Slack support to see if we could change the username of his old account to free it up. They were extremely helpful and gave us a free month of the Slack Standard plan so we had access to the username changing feature.
That was all great until 1 week later when suddenly a load of pinned messages disappeared from the sidebar of our #chat channel. The blame game began as we tried to figure out who was responsible, until we checked our statistics page… Slack was only reporting 6,705 messages in public channels.
I immediately contacted Slack support and they informed us that our public message retention settings were set to delete messages after 7 days, and that there was no way to recover what had been deleted.
This would be fine, however none of the team’s owners say that they changed that setting. On top of that… the response from Slack confirms that other owners would be notified if the setting were updated, which is true:
However nobody received any messages like this (prior to today, where we changed the setting to prevent the deletion of messages):
The only plausible explanation from this point is that Slack automatically enabled the 7-day retention of messages when our team was upgraded to the paid plan trial.
What is there to learn from this? Not much.
We put so much faith in online services nowadays, trusting them with all of our data. While some people are concerned with these companies doing bad things with our data, the real issue in many cases is them making mistakes.
If your team uses Slack, make sure to keep an eye on extra settings when you upgrade – some defaults might not be what you’re looking for.
As much as this was a pain, I still love Slack. This situation was a mistake on our part as well. We’ve learned now to always keep backups/exports of our data, and to audit all settings, both on Slack and other services, on a routine basis.
UPDATE (02/09): Slack’s operations team are able to restore the rest of our missing messages. Thanks Slack!
Hacker Noon is how hackers start their afternoons. We’re a part of the @AMI family. We are now accepting submissions and happy to discuss advertising & sponsorship opportunities.
If you enjoyed this story, we recommend reading our latest tech stories and trending tech stories. Until next time, don’t take the realities of the world for granted!
