The Hard Thing About The Behavioral Science Of Things
Marketing is like magic, except it’s a science.
In 2014, Kevin Li (@liveink) sat me down and explained why he thought I had the qualities and potential of a good marketer.
Then, I didn’t believe a word he said. Skepticism is a mofo.
Like frfr.
“This dude didn’t know what he was talking about. I didn’t know the first thing to being an effective marketer.”
— McKenzy Germain
Long and behold, six years and many moons later, Kevin was right. I could only delay the inevitable for but so long.
The truth is that I’ve been selling and marketing my entire life. Whether it be people, skills, Christmas trees, video games, basketball cards, marvel cards, sneakers, fantasies, or products, I needed to pawn off to pay for graduate school. I’ve either sold it or attempted to sell it.
Additionally, I was secretly trained in the scientific method's dark arts in my early adolescent years. Which now makes for a great combination in my line of work. Btw, I’ll most definitely hint at how this all came about below.
Let’s turn back the hands of time for a bit and go on a lightning-speed journey of my early childhood development. During my elementary and middle school years, I spent most of my time in Brooklyn acquiring street smarts from my Uncle and cousins. During the summers, I spent what was supposed to be a vacation acquiring advanced book smarts in Hell's Kitchen. My godfather was an engineer by trade who vowed that I would someday become an engineer as well. He definitely challenged me and made me think analytically at all times. A skill that I would later combine with my street smarts.
Look, there was something that I was told as a kid which changed my entire outlook on life. I was sitting at the kitchen table one night, frustrated and crying at 2:00 am because my godfather had me working on a problem that I couldn’t solve. He pulled me aside and said:
“Didi, just because you have to undergo a lot of repetition, it doesn't mean that you are bad at it. In reality, what you’ are doing is sharpening your pencil, so that you can draw the perfect picture. So keep at it, test different options, and struggle harder.”
All of those sleepless nights I spent as a kid, correcting answers to questions that I had gotten wrong until the wee hours of the morning, helped me build true grit, resourcefulness, and the ability to survive extensive trial and error scenarios, which is the exact mindset that you need to adopt if you’re considering a career in growth marketing.
I eventually took the creative path and underwent 12 years of art school instead of an engineering track. Regardless, he supported my efforts throughout the span of my educational career and beyond.
He gave me a gift that no one else in my environment could provide; FIRST PRINCIPLES.
Assume nothing. Question everything. Learn the truth!
(The origin is unknown, but I’ll claim this quote as my own.)
Moreover, learning first principles at such a young age was a gift and a curse because I challenged everyone and everything! I mean, why not? Wouldn’t you have used this knowledge to trump adults when you were a kid?
I did! But, I digress…
Hold that thought! While writing this article, I realized that I was one of his experiments, a mere test subject. OMG!
Here’s his A/B experiment in a nutshell:
- H0 (null hypothesis): let me continue my schooling in Brooklyn.
- Ha (alternate hypothesis): provide year-round room and board in Hell’s Kitchen so that he can optimize my learning curve.
Side note: The high probability or risk of letting me go back to the hood meant that he couldn’t add direct value to my personal growth year-round. Hence, further diluting his teachings.
The result: It’s an ongoing experiment. I may not have met his expected outcome initially, but the deeply rooted seed that he planted has placed me on a path to leveraging the scientific method in my daily life.
Now, you are probably wondering if he is yet to achieve statistical significance?
IMHO, I’ve never been able to confirm without having enough data to present my learnings. “Maybe you should ask Mr. Owl.”
So, what’s the hard thing about the behavioral science of things?
The outcome is unpredictable. However, with the right team, set of tools, research, and hypothesis, you can conduct experiments that will lead you to the promised land; or not. Because even in a failure, there’s knowledge to gain.
Insert Confucius say quote here “_____________________________________.”
A rigorous research and testing process are not for the meek. Especially when you are putting out fires.
It would be best to understand testing approaches that work (and pitfalls that don’t) to get more wins and insights from optimization efforts.
running experiments | customer base studies | prioritization
Last week, I stated that “week one of CXL’s Growth Marketing Minidegree course has been nothing short of remarkable. Given an adequate amount of time and focus, I believe that anyone who completes this 12-week course will be able to write their own ticket to marketing rockstardom.”
Week two is the epitome of “how to prevent yourself from drinking out of a fire hose.” Literally!
To complete all of the coursework on a live product and get a deeper understanding of your learnings, you’d have to do the following:
- stretch the coursework out across the course of 3-4 standard college semesters
- get an opportunity where you learn on the job
- or start a side project with the intent of scaling it
In a nutshell, you are acquiring a portable master’s degree.
Make the calculation for yourself, understand how much money you can invest to solve this problem, and to have this continuous tracking of your website if its being measured or not. — Ton Wessling
Here’s a rundown of the topics that are covered in the Running Growth Experiments course.
Research and testing
- Part 1: Peep Laja talks about what to test first and how quickly. Additionally, he covers what qualifies as optimization and what doesn’t, how to know what matters to your customers, and how to watch for confirmation bias.
- Part 2: Peep discusses the 6 types of data to help you make great optimization decisions. What should you be testing, and how should you test those ideas for the most impact? Peep also overs heuristic analysis, data analytics, and user testing.
- Part 3: In this talk, Peep discusses testing the right data, sample size, false positives, and false negatives, as well as statistical significance as an inaccurate stopping rule. Peep details what to do once you’ve identified your company’s individual problems and divides those problems into 4 different types of issues: instrument, “just do it,” test/hypothesize, and investigate.
- Part 4: Peep answers audience questions about A/B tests and sample size, best testing methods for your site, and his favorite tools for optimization.
Conversion research
Peep explains what conversion research consists of and the 3 buckets that they are categorized under.
Experience-based assessment
- Site walkthroughs
- Heuristic analysis
- Usability analysis
Qualitative research
- Online surveys with recent customers
- On-site polls
- Phone interviews
- Live chat transcripts
- Customer support insight
- User testing
Quantitative research
- Web analytics analysis (e.g., Google Analytics and other quantified data tools like Adobe Analytics, KISSMetrics, MixPanel, Heap Analytics)
- Mouse tracking analysis
A/B testing mastery
Intro
- History: understanding the evolution of A/B-testing
- The value: understanding what effectiveness can bring you on top of efficiency. Understanding the position of A/B testing within the hierarchy of evidence.
- When to use it: understanding what situations A/B testing can be valuable and how it should be applied in those situations.
Pillar 1 — Planning
- Wondering if you have enough data to conduct A/B tests? Learn about the “ROAR” rule of thumb; get an extensive explanation on statistical Power; learn how to calculate whether or not there's enough data to conduct A/B tests.
- Having issues picking your north star? Learn about when to pick what kind of KPI; understand how to set up and OEC (overall evaluation criterion).
- Research to get insights for your A/B tests: understanding the 6v research model and how it can be used to generate user behavior. Also, learn why and gather insights on View, Voice, Versus, Validated, Verified, and Value.
- Hypothesis setting: understanding why it’s good to write a hypothesis before running a growth or research experiment.
- Prioritize your A/B tests: learn how to use prioritization models and transfer them to an A/B test roadmap.
Pillar 2 — Execution
- The proper way to Design, Develop, and Quality Assuring your A/B test.
- Configure your A/B test in your tool: learn proper configuration and what consequences certain options may have; pre-testing vs. post-testing; understanding the value of making an analytics connection.
- How to calculate the length of your A/B test; learn how to calculate the scheduled length of your A/B test; shorten the length of your A/B test; why you should not continue an A/B test when it’s finished.
- Monitoring your A/B test: understanding what needs to be monitored during an A/B test and when it makes sense to stop your test early.
Pillar 3 — Results
- A/B test outcomes: understanding when your A/B test is a winner; learning what to do when your A/B test is not a winner; knowing when to conduct behavioral analysis.
- Presenting your learnings: creating a presentation template and understanding what information is valuable to present to your company.
- Business case calculations: understanding why the result of your A/B-test program is not the sum of all A/B-winners; learning how to make a proper business case calculation of your A/B-test program.
Outro
- Scaling up testing: understanding the balance between quality and quantity while scaling up A/B-testing vs. understanding the most logical maturity growth path (and the big pitfall) of A/B-testing throughout your whole organization.
- Sharing and scaling insights: understanding the value of building an archive; knowing what is interesting to “tag” while filling your archive; learning how meta-studies of your A/B-tests can be used for better decision making.
Bonus: this wouldn’t be top secret if I spilled the beans. Sorry, but not sorry. To find out what it is, you’ll have to take the course.
Statistics fundamentals for testing
Overview of the statistical concepts that every marketer and optimizer should know.
- Sampling — Populations, Parameters, & Statistics
- Mean, Variance, and Confidence intervals
- What statistical significance (p-value)is and isn’t
- Statistical Power
- Sample size and how to calculate it
- Regression To The Mean & Sampling Error
- 4 Statistics Traps to Look Out For
Let me end the second installment of my 12-week series by saying Thank You to everyone for sticking with me throughout this journey.
I really appreciate you!
My post-CXL Growth marketing course to-do list:
- Learn javascript
- Grow faster with the right mentors. Complete every single CXL course that I possibly can. → Apply my learnings to a personal project.
- Does recession consideration → acquire another Master's degree in Human Factors w/ a minor in Behavioral Science? Dollar, dollar bills y’all. 🤣
- Become a Growthmonger! 😈
- Get my family to pivot away from calling me, Didi!
P.S. Hopefully, the little voice in your head is reading my content with a sarcastic tone. Because that’s how the monologue runs through my mind.
What more can I say? I’m an ENTJP. 🤗
P.P.S. You can learn more about me here.
P.P.P.S. If you haven’t done so, you can also read my last essay here.
P.P.P.P.S. If you loved my first principles quote, then this is the best shirt for you.
