Why your company doesn’t have an internal Google
Why is it so hard to find the information you need at work?
Whether its meeting notes, architectural overviews or just the reasoning behind why a product is made, it’s almost never just a search away. I’d like to explore that why this is and then perhaps offer a solution.
A quick lay of the land
I’m going to focus on online editors for this article. Offline editors like Microsoft Word are great for individuals but in a team setting you can pretty much forget finding the information you need when it’s spread over dozens or hundreds of computers.
Online editors including Google Docs, Notion and Confluence are a big step forward, giving you a single source of truth for information and all the benefits that come along with it. These include features like real time editing, messaging, and version history. These products will also give you a way to organise your information, for example via folders or tags.
So what exactly is wrong with these products? If all my content is organised and in one place surely it should be easy to find the answer I need. One of these products is even made by Google — the search company.
Unsurprisingly when you are only dealing with a few dozen documents search works fine enough. I say fine enough because it still looks for an exact match in title or content; it’s still far from returning results given a generic question. The real cracks start to show as the number of documents climbs into the hundreds or thousands. You’ll often get back irrelevant, duplicate, or outdated results. But why?
I think it really comes down to three things:
- Users have too much responsibility over how content is organised
- Search is only as good as the content it is searching over
- There is a disconnect between the platforms used for asking questions and your long term knowledge base
These three issues plague every product on the market.
The issues
Organising content
Products like Google Docs and Confluence give you the ability to place documents inside a folder structure. The problem is as your knowledge base grows it gets harder and harder to find the right place for the next piece of content to live. Not only that but often there is more than one correct home for your content.
What happens next is death by a thousand cuts. Time deprived employees find the best place they can for their content to live, but they inevitably make mistakes or disagree on the placement. Content starts to become more spread out and disorganised. Very similar content starts to appear. This floods search with too many results, degrading its usefulness.
To combat this you have to curate content more and more frequently, until it’s a full time job that no one is specifically employed to do.
It’s all about the content
Of course even if your content is organised, you’re not going to find the answer you’re looking for if it hasn’t been written down. You may be the first to have a particular question, more often though you’ll stumble across part of the answer, essentially everything the author could think of at the time.
Then there is a fact of life that every product on the market has completely sidestepped. Information isn’t static. It needs to be nurtured. Currently the only way to keep the content you write up to date is to periodically check through everything you have written and see if it still applies. This is time intensive and impractical, and only gets worse the more you write.
The knowledge base disconnect
Most of the time people will ask their questions in chat tools like Slack and then maybe, just maybe, take the time to store that new knowledge in a knowledge base. This disconnect causes a lot of information to get lost.
Time for some potential solutions
The great thing about a companies internal knowledge base is that it generally isn’t too diverse, you aren’t likely to be writing an essay on the hidden dangers of sword swallowing in your tech startup. If we remove folders entirely, instead focusing on a tagging-like mechanism, it might be possible to leverage this relative homogeneity in order to construct an ML model that learns to group and tag content for you. Eventually the goal is that you will be able to just create content without worrying how it’s organised.
By shifting the responsibility of organising content onto the platform as opposed to the user it would be easier to detect similar or duplicate content. We could warn users of this in realtime, thereby mitigating duplicate content and further improving search results.
To deal with non-existent or partial content we make the ability to ask questions a first-class citizen of our knowledge base. Not just enabling comments but the ability to request content be written.
Other ideas of mine include:
- Giving every user the ability to manage the status of related pieces of content with one click
- Accountability — every piece of information has an owner
- One-click plugins to tools like Slack to be able to bring in non-ephemeral information.
- Change the standard unit of storage (the document) to allow for quick and easy short form content
This is still only a fraction of what needs to be done, but the good news is I’ve spent the last few months building a new product that incorporates these ideas and more, its name is Scribe.
Scribe
I’m building Scribe to try and radically improve a teams ability to create and maintain a knowledge base at scale. It incorporates, or will incorporate, everything I’ve mentioned, and over time much much more. Here are some screenshots:
If you could use a Scribe in your life then sign up for beta access using the link below. I’m only a few weeks away from being able to send out keys to the first round of users which is very exciting!
If you have any ideas or feedback you would like to share please leave a comment, I’d love to hear them :)
