Matching Niche B2B Expertise with Natural Language Processing (NLP)
A peek into the technology that helps us match our clients with experts
The expert network business is, at its core, straightforward. However, clients need insights that are in-depth, specific, and otherwise difficult to access. We deliver that insight by connecting them with niche experts we source and match for each project.
But finding the best experts to match each project isn't easy. Traditionally there's a massive amount of manual searching, vetting, outreach, and human-to-human interaction. As a result, it's historically a white-glove service, where expert networks often keep as much as seventy per cent of the price paid by the client.
Today, expert networks deliver this service with some blend of manual human effort and software. For some providers, it's 90% manual and 10% software-driven. In these instances, the expert network manages a database of experts and has a team of people sifting and searching to find the right expert for each project.
For DeepBench, software plays a much more significant role — allowing our team to be more efficient and effective in their work. By pairing natural language processing (NLP) systems with our team of Client Associates, we can deliver better-fit experts more quickly with less overhead.
Our matching process is broken down into three steps:
- Collecting project information from the client.
- Delivering a recommended pool of candidate experts.
- Refining/extending that recommended pool.
Collecting Project Information
The client first opens a new project by describing what they hope to learn. The more specificity the client can provide, the better. Granular topics, critical areas of expertise, geographies, screening questions, and work experience all help us make more accurate matches in less time.
Once we have the project information, the machine learning system works. It starts by scrubbing the consolidated text provided by the client, removing common words (such as 'the', 'is', 'and', etc.), and saving the remaining keywords in a "bag".
However, not all keywords are equally descriptive of the project. Therefore, we leverage the bags of words against a larger pool of words derived from thousands of other projects run to date to give each keyword an importance weighting — the less frequently a keyword appears in other texts, the more unique and meaningful it is to the project at hand.
We compare how similar two pieces of text are by some function of the overlap between the words in each part of the text and the associated weights from our more extensive pool of words. The higher that number is, the more similar the two pieces of text are. Given language's peculiarities, we also consider redundancies based on word stems (e.g., noting overlaps between 'tall' vs 'taller' vs' tallest', etc.).
For any given project, we can check if we have similar projects using the method above. If we find a similar project, the advisors may be good candidates.
With the same techniques mentioned above, we can also match project descriptions directly to DeepBench profiles that contain titles, organizations, and job descriptions.
Delivering a Pool of Expert Candidates
Concurrent with the NLP processes working in the background, we have a team of Client Associates augmenting the software's output. Including a human-in-the-loop allows us to catch potential errors in expert sourcing, optimize for desired client outcomes, and clarify any minute details to experts and clients as needed.
With this, we assemble a curated pool of candidate experts and hand them off to the client. The client may take one of the following actions: upvote/downvote the profile, message the expert or request a consultation with them. We use these actions to identify which experts were suitable matches. This helps us refine our suggestions by finding other experts with similar characteristics to the strong matches.
Extending The Recommended Pool
The algorithm powering this process becomes more effective over time. With each new project created and with each new expert registered, the amount of data the algorithm has access to continues to grow. This expands its capability for comparison and analysis, allowing us to deliver better expert suggestions.
One of the exciting features we are exploring at DeepBench is dynamically building out our advisor profiles. So, for example, if a client starts a project that is tagged with 'Sales & Marketing' and upvotes or requests a meeting with an advisor that does not have that tag, that advisor is tagged with that topic, and this mechanism could lead another client with a similar project focus to be connected with the advisor.
The expert network space is a fascinating industry to bring commercial applications of artificial intelligence to the market.
DeepBench is thrilled to continue expanding the capabilities of this market and delivering exceptional experiences to clients and experts alike.
