Validating Machine Learning: FedBizFind Captures Over 1% Market Share

Here’s how we took a disruptive idea (which could not be explained in a simple landing page) and iterated into market success. FedBizFind: the faster, easier, better way to win government contracts

Nothing is sweeter than market validation of our DeepFind operational intelligence technology.

In July we launched FedBizFind. It is the first commercial product to use the DeepFind engine. Since then, more than 6,000 of the approximately 500,000 companies registered to do business with the US Federal government signed up to use FedBizFind to find contracts. Even more impressive is the rate of monetization of the freemium model: Conversion to paid subscriptions is over 0.25% of all traffic; mostly from unaffiliated sales. These subscriptions are currently $97 per month or $970 annually. And yes, this is an introductory price that we are offering to early adopters.

Discovering Features to Drive Customer Adoption

Our mission is to enable small businesses to rapidly grow by overcoming the obstacles of doing business with the Federal government. Our FedBizFind solution makes it easier, faster and more profitable to pursue government contracts. In short, we make it easier for our customers to win.

To minimize risk and maximize our ability to scale sales, Englue partnered with Florida-based FedBizAccess (FBA) to bring FedBizFind to market. Our mutual goal is to apply machine learning to create a new approach to match small businesses with Federal contracts. This is a good goal. It’s easy to measure the impact artificial intelligence makes on winning government bids. The entire process should be faster (less time consuming), easier (fewer and simpler workflows) and yield dramatically higher rates of returns for our customers (profitability). Although the market for search-as-a-service for government contracts is very crowded, FedBizFind is the only product that uses machine intelligence to create “unfair advantages” for users. Even our free service is better than the industry norm. The industry incumbent, Deltek’s GovWin, is bloated with features, unable to find contracts using keywords and requires the end-user to be an expert in government acronyms, policies and procedures. GovWin is exactly the kind of competitor we like: Thriving through mediocrity led by a non-technical team that looks at the problem from the government’s (not end-user’s) perspective. We know we could do better. Especially as our partner (FBA) brings sales, marketing and subject matter expertise that our team lacks.

Validating Product Features to Fulfill a Vision

If I asked people what they wanted, they would have told me ‘faster horses.’ – attributed to Henry Ford.

Rather than rely upon expert opinions, we used lean customer discovery methods to isolate the minimum viable product features necessary to gain early market traction. What are the obstacles for small businesses to land government contracts? Why do they pursue them? How can we make it really easy, fast and better than current methods? We developed a portfolio of more than 50 hypotheses to test using a lean canvas approach.

We interviewed more than 300 existing FBA customers and more than 100 non-FBA customers. We used an optimal stopping algorithm to pick “winning features.” This enabled us to isolate solutions to customer pain points in as few as 19 interviews with a confidence above 90%. Once a feature was isolated, we asked seasoned sales reps and experts in Federal contracting to prioritize features based on their perceived value. Often we would iterate the conceptual development of a feature by going back to the sales “experts” to refine the questions we asked to potential customers. Next we triaged the feature set to determine our minimally viable product (MVP). Three questions determined the order in which we would build features:

1. Is the feature so valuable that it sells itself?
2. Does the feature give customers an unfair advantage in winning contracts? (over competitors using GovWin and other systems)
3. Would the feature enable ways to expand the total available market (TAM) beyond existing government contractors? (as determined by the System for Award Management or SAM database)

Our starting point was to repurpose Englue’s existing DeepFind technology for medical research to find the reasons why some companies won contracts and why others lost. Our technology enabled us to build predictive models by simply evaluating open-source public information. In technical terms, we applied feature detection methods using natural language processing (NLP) to help identify how well a company’s capabilities match those sought in a government contract. Releases in the near future will include:

• Updating the matching algorithm based on user’s input about each new contract they review.
• Predictive models for who is most likely to win a contract based on past performance.
• Find similar contracts to past winners.
• Enable government buyers, primes and subcontractors the ability to find companies that have the right qualifications for a contract.

Our approach enables FedBizFind to get more accurate with use. That means early adopters get the benefit of a more accurate system (at a lower cost). The use of reinforcement learning (where users train the system to find what they want) is a key feature: It makes each instance of FedBizAccess act as an intelligent agent to find the best matches for each user.

Empowering IRL Relationships with Machine Learning

We believe machine learning must reinforce human strengths. So we focus on creating features that enable users to build relationships “in real life” (or IRL). People do business with people. A fundamental requirement of any machine learning application is to empower humans to do more with less. Therefore, FedBizFind must make it easier to find the right real-life relationships faster than incumbent approaches; namely the practice of hiring recently retired government workers to “walk the halls” and sell to old colleagues. FedBizFind changes the equation by shifting focus away from “old colleagues” to “who can do the best job” because now knowing who to talk to is no longer a function of experience. Moreover, FedBizFind transforms the game by enabling many new companies to enter the market. From a business development perspective, the government is a vast organization where it is easy to get lost. Many small businesses refuse to do business with the Feds because they fear wasting time and money navigating huge bureaucracies. FedBizFind cuts through the clutter by telling you exactly who to call and what they are looking for. Simple.

Talk is Cheap, Code is Expensive: Lean Customer Discovery Interviews and Trial Launches

Our interviews lasted for more than a month before we started to build any code. We call our priorities the “3 C’s” – customers, code and capital (in that order). We did two test releases before we went into full production. The June 1 release was validated core market assumptions by measuring response to an email campaign (n=10,000) followed by “listening to” responses, return visits and (of course) complaints. From June 8 to July 15 our team added a few new features and redesigned the interface and signup method. Another test email blast on July 16 (n=1,000) exceeded our expectations. So on July 28, we launched the production system to 50,000 customers. The chart below tells the story.

Lessons Learned: FedBizFind’s Killer Feature Set

Fundamental design principals led us to create a feature set to enable small businesses to build and maintain the right relationships by focusing their efforts on the opportunities that are easiest to win. Yes, our search is better. But that’s not enough. Search technologies are constantly improving. We are better simply because our team is more proficient in current technologies (and our competitors are still using methods from the 2000’s). Our big differentiators are very difficult to copy and provide increasing value over time by giving our customers “unfair informational advantages” including:

1. Prediction of success: FedBizFind provides a score and color codes the similarity of a company’s capabilities to the needs of a contract. This empowers business development executives to focus their efforts on the opportunities with the highest chances of winning. The key benefit: Stop wasting time on contracts that are already destined to be won by others.
2. Knowing who to call: We constantly data mine Federal records to identify contracting officers, small business advocates and the prime contractors most likely to win a contract. The key benefit: Stop wasting time on the wrong people. For most small businesses the fastest, easiest way to win a contract is to subcontract to a prime who is already obligated by law to give small businesses a portion of the contract. At the moment, we are the only source on the internet that provides a list of every Federal employee responsible for small business contracting, including: small business specialists (SBS), procurement center representatives (PCR) and Office of Small and Disadvantaged Business Utilization officials (OSDBU). We obtain and maintain these lists through exhaustive data mining of public records. Yes, you need to pay to see these.
3. Providing transparency into departmental spending: Also at the time of this blog posting, we believe we are the only single source for all projected spending on small businesses by every department within the government.

And more is on the way…

Releases Every Other Week

Speed wins every race. So we designed the FedBizFind architecture to enable continuous deployment. Our constraint in deploying new features is how fast we can talk to customers to validate market demand. We are much faster at testing product hypotheses now that we have over 6,000 users and a sales team to call them. At the moment it takes about a month to take a feature from idea to market validation to coding to testing to production. Several features validation cycles run concurrently, so we usually do a minor release every week and a major release every two to four weeks. The most fun part of the process is getting our technical team involved in learning what the end user really needs. Opinions continue to be a luxury that we cannot afford — and neither can our customers.

FedBizFind is free to register and provides matches at no cost. It is the only system on that gives you the power of machine learning for free. At the moment, we require users to pay to know who to call. We are constantly experimenting. And yes, we will always treat our early adopters as our favorite customers.

Original post: http://englue.com/deepfind-success-fedbizfind-captures-1-market-share/

Olin Hyde

@olinhyde