Theory of Hypothesis Testing

Check Eligibility vs. Apply Now

What is a great hypothesis?

If there was to be a Maslow’s heirarchy of hypothesis, my guess would be:

(1) Learning + Serial Impact (bonus: cross funnel) → Top of the pyramid

(2) Learning + Serial Impact - Current funnel Impact

(3) Learning + Current funnel Impact

(4) Learning + No Impact

(5) No Learning → Bottom of the pyramid

It may be fair to assume that product managers/ owners grow their way up the pyramid and consistent “top of pyramid” hypothesis generation could be a hallmark of a great product manager. Or appropriately, a self-actualized product manager

Here is an example of a “top of pyramid” hypothesis we managed at my current company, Lendingkart.com

Hypothesis

Users seek a light weighted approach toward understanding what specific loan terms to expect

This means before any activity a user can undertake in the loan process, he must be made aware of his specific eligibility. While this may seem obvious in hindsight, taking loan is equivalent to buying a shirt was not obvious i.e. user will mostly do anything to get a loan was the predominant belief

How do we possibly know this?

(1) User research: Lendingkart sees two personas of users — Type O (Orthodox) & Type T (Tech Savvy)

Type Os are married with kids and have been frequently rejected by banks and financial institutes for loans. Hence they tend to be extremely skeptical to share any data and seek eligibility before progressing to apply

Type Ts on the other hand are aspirational youngsters who freely explore multiple sources without going deep into any process. Spray & pray. Hence, they seek eligibility before progressing to apply

(2) Data: This is strongly supported by how the conversion funnel shoots up post providing his loan terms. >50% to <25%

Product Manifestation

AB: Home page “Check Eligibility” vs. “Apply Now” button

Why? If the hypothesis is true, then it is very likely that it is the most important first action the user will take upon landing on Lendingkart

Sample: Only new users to avoid surprise in experience

Statistics: Sample Size = 3000. Alpha = 99%. 1-Beta = 95%

Business Impact

The Click-through-probability went up from a sub-10% to a mid-30% and it all took 0.5 man days of tech effort

Learning & Serial Impact

Strong validation of hypothesis opened up avenues to further optimize the “Start of Journey” experience. Few small steps taken include:

(1) Replace a registration process with Check Eligibility process. Eligiblity = Reject/ select

(2) EMI calculator based onboarding and exit popup. Eligibility = Reject/ select, Estimated EMI/ IRR/ Tenure

(3) Pre-approval loan offer based on PAN only (Government ID). Eligibility = Loan offer

)

Karthik Ramakrishnan

Written by

Sr Director Product at Lendingkart, Ex-Ola, Housing, InMobi, OnMobile

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade