How Do You Kill Tribal Knowledge?
One of the biggest problems in fast growing companies is tribal knowledge. The powerful tricks, solutions and hacks that are essential to to the success of your company that live only in the minds of a few individuals? They might as well not even exist. And some day, they will come around and bite you in the ass.
(Short) Story time: A year or so after we launched a product at a former company we had a request from one of our largest customers for a way for them to embed a login screen on their intranet so that users could “log in directly” from an internal page they had created.
I thought the request was a little silly, after all a hyperlink to our product would have worked just fine. But bending over backwards for large customers is something start-ups are prone to do… so one of our product analysts hacked together a code snippet and sent it off to Steve in support to have them test and implement. Hurrah.. it actually worked.
Not a big deal and Steve thought it was so cool he shared it with other big customers as a value add when he worked with them. And in Steve’s defense clearly it was useful because dozens of customers wound up implementing it.
If it’s not documented, no one knows it.
The problem? After 6 years Steve and I were the only people left at the company that even knew this hack had even been attempted and Steve alone knew just how widely it had been shared.
The impact? For 6 years the code was out there and worked without incident. Steve was happy and so were our customers. That is until we decided to refactor some spaghetti code that held our login process together. QA did a great job of identifying all of the test cases that they knew about for logging in and validated that all of our client components worked with the change before we rolled it out.
The moment of truth: The code was rolled out into production and our test automation scripts all came back green. Woohoo! Just after 4AM though the calls started coming in from some of those large customers. No one could log in. A Sev1 ticket was opened immediately and DevOps/Dev was rousted from their beds to troubleshoot. Well, you can guess what had happened…. The changes to our login code broke the login snippet these customers were using on their internal pages. The code that no one in Dev or QA knew anything about and hadn’t tested against.
The fix: Fortunately the code fix itself was easy. We also added comments to the code to alert future developers working on it (and the QA teams writing test cases) to make sure they tested this scenario. The real fix: We wrote up a detailed internal KB article on the code snippet the customers were using and had dev review it. We linked to that from the Jira ticket for the feature and from an internal dev page which described the login process.
The lesson: If it isn’t written down, no one knows it. People leave, our memories are imperfect. If it’s important, it has to be documented.
Documenting Tribal Knowledge is a Process
I’m a big advocate of embedding technical writers with the development and support teams. Many small companies focus on hiring a tech writer to create user manuals and installation guides. Those are certainly important, but they’re primarily external facing in nature. You need to have tech writers focused internally as well.
How do you focus a tech writer internally?
- Ask them to attend your sprint planning and demo sessions.
- Have a tech writer listen in on support calls.
- Invite a tech writer to Support Quarterly Business Reviews.
- Create a channel for dev, QA and support to submit knowledge nuggets to tech writers for review.
- Buy your favorite tech writer a beer.
Ideas and suggestions?
What tools and techniques are you using to gather and document tribal knowledge where you work? Do they help? What would you do differently if you could?
