How I Got Idea Validation Wrong
I recently shared how I springboard my learning (especially programming) with ChatGPT.
In the story, I described how I had used ChatGPT to build a rapid prototype for a tool to convert YouTube videos to blogs for my sister.
What I said about ChatGPT giving learning superpowers is true.
What I didn’t realise, though, was that I also inadvertently made some classic mistakes of idea validation!
Don’t make the same mistakes as me.
Early idea validation
When I first spoke with my sister, the conversation went something like this:
Me: “Hey, y’know I’m learning to code? I’m looking for a side project. Do you have any problems you need solving.”
Sister: “What kind of problems?”
Me: “I don’t know, just any problem. You’re a nutritionist right? Perhaps something to do with growing your personal brand?”
Sister: “Ok! Umm, well I’d love something can help me convert my YouTube videos to blogs for my website. The raw transcripts are not that easy to convert, and I don’t want to pay for any of the more expensive tools.”
Me: “Oh that’s a cool idea — so you would just take a link for YouTube. drop it in and use AI to generate a blog outline which you can edit before making the blog?”
Sister: “Yes that could work!”
Pause conversation there ⏸️. Off I go to build my one evening prototype. Next day:
Me: “Hey! I’ve got a mock up of that tool we discussed yesterday. Can I show you?”
Sister: “Really?! Yes of course.”
(I demo the prototype)
Sister: “That’s so cool! I’m very impressed you put that together so quickly.
Me: “I’m glad you like it! ☺️ Clearly it leaves lots to be desired in terms of design. But if you think you would use it, I’ll happily make it better?”
Sister: “Yes, I would use it. I’d like to play with it a bit more but I can definitely see it being useful.”
I spend the next few days upgrading the interface and adding some new functionality.
Still not a polished product. But check out my previous blog and you’ll see that it’s an improvement. I’m excited to show my progress.
During the demo of prototype 2.0…
Sister: “Wow it’s come a long way! Using it again now, I think I’d probably use it mainly to get to an outline and then write the blog myself. Once I’ve got a good outline, the writing is the easy bit that I enjoy.
Me: “Ok, that makes sense.”
Sister: “I had no idea AI had become so powerful. Infact, another cool use case that would be amazing is something that can summarise the academic articles I have to read. I spend HOURS sifting through these. It’s definitely the biggest pain point in my process.”
Me: “Have you seen the latest ChatGPT release? You can now upload PDF files for free, so it would be perfect for that…”
Not looking good for prototype 3.0.
What I got wrong
The Mom Test must be one of the most recommended books for validating business ideas. Especially those related to building software.
The book outlines an approach to learning rapidly about potential customer’s problems through conversation.
Side note: If you are thinking about starting a business, read this book first. It’ll save you lot’s of time and effort.
Having now read the book, and reflecting on the validation process above, I got several things wrong. Namely, I:
- Focused on the idea not the problem — I thought I was being clever by asking my sister about “problems she needed solving”. But as you can see. I quickly let the discussion slide into brainstorming about ideas for solutions.
- Built rather than listened — So eager to build something, I missed the opportunity to dig into the details. How much time does she spend on converting YouTube videos to blogs right now? How important is this relative to other obstacles in her content creation progress? And, I overlooked the clue that she had briefly looked but isn’t willing to pay for existing solutions (aka. probably not a very important problem).
- Gave weight to compliments — My sister is a wonderful, kind person. Like many, she is also naturally positive and encouraging. So no wonder she gave me encouraging compliments. She may well have been impressed by what I was building. But, as the Mom Test says:
“Compliments are the fool’s gold of customer learning: shiny, distracting, and worthless”.
Looks like I failed the Mom Test whilst speaking to my sister…
Where to go from here?
Building the prototype 2.0 gave me impetus to learn new skills in Next.js such as:
- Using a new component library (Shadcn/ui — highly recommend).
- Adding authentication and authorisation with NextAuth.js.
- Integrating a Stripe Payment Link with a web hook.
I will take these skills into my next project. So the experience has been worthwhile.
I am also glad to have learnt these lessons about idea validation early on.
AI is making it easier and easier to build solutions quickly. But beware, the unintended consequences might be that you skip the critical step before, which is learning about what is important to solve.
As always, I will continue improving my programming skills. But I am now also thinking about finding more opportunities to practice idea validation.
Before you go!
If you enjoyed this story, please burn a few finger calories and clap this post 👏🏻. It helps me know whether I am writing things that you like to read.
Follow me for more stories on using technology and building your online presence. (I’m just getting started, so you can learn along side me.)
Until next time, thanks all! 🙏🏻
