Enterprise search has evolved.
Here’s what you need to know.
On April 20, 1994, Brian Pinkerton created WebCrawler, the very first internet search engine. Unlike the search engines that came before it, WebCrawler allowed users to search for any word on any webpage. And for decades, this traditional type of search — keyword search — was the industry standard. Today, however, search is smarter. Large enterprise use cases have driven this evolution.
The typical data-centric enterprise manages more than 10 billion files annually. That volume of data creates a lot of challenges. In fact, enterprise employees spend an average of 1.8 hours every day searching for the information and data they need to do their job.
Timo Selvaraj, VP of Product Mgmt.
As big data became bigger and more connected, developers used artificial intelligence to make enterprise search smarter. The result? Cognitive search, which Forrester defines as “a new generation of enterprise search solutions that employ artificial intelligence (AI) technologies such as natural language processing and machine learning to ingest, understand, organize, and query digital content from multiple data sources.”
Keyword search simply matches what a user types into a search bar with the data in a document. While that was revolutionary in 1994, here’s what distinguishes cognitive search today.
Powerful Indexing
Cognitive search, unlike keyword search, crawls and ingests both structured and unstructured data. Keep in mind: experts estimate that as much as 80-90% of your data is unstructured, including email, customer surveys and social media.
Cognitive search solutions also enable developers to embed search in other applications using SDKs, APIs, and other tools. This is important because your data isn’t confined to databases: it’s scattered across the enterprise. So, search has to work where your teams work — in Slack, Salesforce, Jira, Amazon Web Services (AWS), etc.
Natural Language Processing (NLP)
Keyword search is basically a matching game played with digital data. If a user types “COVID-19” into the search bar, the results page features thousands of mentions of those exact words — some relevant, some not so much. Cognitive search, on the other hand, uses natural language processing to understand humans’ questions and predict their intent so it can deliver more relevant results.
For example, if a user types “how can I tell if I have COVID-19,” cognitive search understands this natural language query and serves up articles like, “COVID-19 Symptoms and Testing,“ “COVID-19 FAQs,” and “COVID-19 vs. flu.” “Cognitive search understands what you’re looking for even if you don’t,” explains Selvaraj. “So like a very experienced librarian, it can point you to the resources you need.”
NLP technologies are also what make cognitive search capable of responding to search queries via chatbot and voice assistant (think Siri, Alexa, etc.).
Machine Learning (ML) Algorithms
Cognitive search isn’t just more powerful than keyword search, it can also learn to be more powerful than itself. Using implicit and explicit feedback, cognitive search engines continually learn and use what they learn to fine-tune their performance, producing more relevant results over time.