Search at work- do you really have the information at your finger tips?

Featured image on Bing.com homepage on Sep 18, 2021

The red panda may very well have all the info it needs. It is relaxed and content. For we humans, the search for information never ends.

Imagine the scenario below:

You are rushing to make an all-important presentation. You are almost there but missing critical competitive research data. You recall someone in the market research team did share this information in a talk few months back. Hard part — you do not know where this info is! You spend several minutes looking for this presentation in your emails, chat, content management system and everywhere else. You ping your co-workers if they have a link or a copy but no luck. You give up on your search, spend many more hours searching on the internet. If it was a publicly available info you may find it with some effort. If it was internally generated research data, tough luck! Frustrated and exhausted, you give up.

Sounds familiar?

You are not alone. As per the often-cited report by McKinsey (1), “employees spend 1.8 hours every day searching and gathering information.” That is a whopping 9.3 hours/week wasted searching for information.

Finding information in an enterprise — finding a needle in the haystack!

In today’s knowledge economy, organizations are creating a wealth of information and insights at an unprecedented speed. This cumulative business knowledge is critical to business leaders and employees alike for their day-to-day work, to make strategic decisions, and, to gain and maintain competitive advantage.

Millions of hours are spent generating this collective body of knowledge within organizations. Sadly, this information remains unused because employees cannot find it. To make it worse, people, spend a ton of time looking for information, only to be left unsuccessful and frustrated. They spend even more time recreating this information. It is a colossal waste of employee productivity and morale.

History of enterprise search

The history of enterprise search is likely as old as the modern enterprise itself. Thomas Huertas writes ,“Modern enterprise arose during the years 1870 to 1920” (2), when large economic institutions emerged with multiple levels of managerial hierarchy and separation of roles. Effective management of complex business activities was required to succeed in large marketplaces where these enterprises operated.

As the enterprise grew larger, the need to manage information grew as well. Information management and search evolved with technology from paper filings and catalogs to microfilms, to scanned digital libraries on tapes and CDs. Information management was still catalog and archive centric. If you needed something, you needed to spend hours trying to find the document you wanted. With the Internet getting mainstream in mid-nineties, and intranet sites within companies becoming a norm, workplace information search became somewhat easier but added its own complexities.

Challenges in enterprise search

The nature of modern workplace and information needs of employees has been evolving at a fast pace and is accelerating. The quest to find the right information quickly is hampered by many challenges.

1. Too many places to search — each application used to create or store information comes with its own search. If the information lives in a wiki, you must go and search the wiki. For a document on SharePoint, you need to be searching on SharePoint, project management artifacts and work items need to be searched on your organizations project management software. Same for workplace chat and meetings software. This fractured search experience requires users to familiarize themselves with several search appliances. Secondly, in many cases they need to search at multiple repositories before they can get the information they need.
2. Impossible to manually label content at scale: The amount of information generated within enterprises is exploding at an alarming rate. The traditional approach relied on human applied labels and tags to facilitate search. However, most enterprises do not have an organization wide shared vocabulary and taxonomy. Inconsistent information labeling rendered the search ineffective. Secondly, the approach is very hard to scale, given the pace and volume at which of content is getting created.
3. Emotional and Cognitive overload: Knowledge workers are dealing with a lot of information on a daily basis with “constantly changing interfaces, frequent introductions of new versions, and increasing requests to adopt new digital technologies or online services cause more harm than benefit. “ as noted by Anne-F. Rutkowski and Carol S. Saunders in their IEEE paper [1], they further write,” We’re experiencing a new type of burnout in today’s computerized world: emotional and cognitive overload (ECO) with information technology”. This emotional and cognitive overload likely acts as a negative spiral. It may limit people’s ability to remember enough meta information and context to search effectively and get the information they are looking for. Failing to get the right information would add to their digital burnout even more.
4. Information staleness: Not finding the right information is frustrating, getting stale information could be even worse. Outdated information could be misleading. Decisions based on stale information could produce suboptimal or negative business outcomes.
5. Missing feedback loop: One factor leading to outdated information is the absence of a feedback loop back to the content author or maintainer. If I knew that my business analysis document is being searched and read by a lot of people, I would be motivated to keep this information up-to-date. Insights such as, what is being searched, which searches return no results or poor results, which clicks lead to poor engagement, can help organizations discover and address information gaps within their enterprise. Such feedback loops have been missing in enterprise search appliances.

The next generation of enterprise Search

Thankfully, recent enterprise search offerings, Microsoft Search and Google Cloud Search and several others, promise to address many of these long standing pain points in enterprise search. These offerings bring in the familiarity and quality of Google and Bing into enterprise search. Enterprise search can finally leverage decades of advances in the search space, including the cutting-edge AI/ML technologies in area of natural language understanding, semantic search, voice transcription, personalization, instant answers and more.

Conclusion

People spend several hours every day searching for content resulting in lost productivity, effort duplication and employee frustration. Exponential growth of enterprise information, challenges in cataloging and labeling the content, fragmented search experience across multiple applications, and cognitive overload are some of the challenges which need to be addressed. This is set to change with new enterprise search offerings such as Microsoft Search and Google Cloud Search .

In the Part-2 of this article, I will cover how Microsoft Search in Bing helps address some of the long standing pain points in enterprise search.

References

1. McKinsey Global Institute. July, 2012, The social economy: Unlocking value and productivity through social technologies
2. Thomas F. Huertas, “The Rise of the Modern Business Enterprise: The Case of Citibank.”, in the Business History Conference, https://thebhc.org/sites/default/files/beh/BEHprint/v014/p0143-p0158.pdf
3. A. Rutkowski and C. S. Saunders, “Growing Pains with Information Overload,” in Computer, vol. 43, no. 6, pp. 96–95, June 2010, doi: 10.1109/MC.2010.171.