The Goldilocks Problem
This is the final part of Future Generative; read the earlier parts here and here. All views are my own and do not represent those of DCM.
In mathematics, an expression is said to be well-defined if it has a unique value or interpretation for a given input. For example, functions are well-defined since each input maps to exactly one output. Generative AI, on the other hand, is not well-defined because running the same prompt multiple times will result in different outputs. A generator can still produce desired outputs if the prompt itself is well-defined, but this often requires a level of sophistication that its intended users do not have. Conversely, sophisticated users have no need for it. I call this the “Goldilocks problem” because there are sometimes no users who are just right.
As an example, I recently argued that generative chatbots like Replika will always be poor substitutes for human companionship. Contrary to how it may have read, I think the gap is more due to user deficiency rather than technical limitations. The prompts are nominally messages, but the true prompt hiding behind those messages is “help me feel less lonely”. This is clearly not a well-defined prompt. Making it well-defined would require a degree of self-awareness that many people lack, either because of the sheer amount of reflection and time needed to get there or because it is outright unachievable (not to get philosophical). This may be the case especially for Replika users who rely heavily on the chatbot for connection. But those who are self-aware enough probably enjoy fulfilling (human) relationships and have no need for Replika.
The Goldilocks problem also came up in a question I’ve been researching: Will generative social content be driven more by curation or co-creation? I initially envisioned a future with no recommendations since the ability to interact with and shape content using generative AI means any video could become the right video for a viewer. But after trying many generative social games, I am convinced that curation will be even more important than it is today. Take AI Dungeon, the best-known of the choose your own adventure text games. First of all, the scenarios you can play are already written in a way that encourages engagement — a form of curation. But even with that head start, I found myself losing interest fast. Again, the prompts here are nominally messages, but the true prompt is “entertain me”. This is also not a well-defined prompt, and making it so would require my imagination be much more exciting that it is. I suspect most AI Dungeon players feel that the scenarios end too soon and aren’t fun after the novelty wears off for the same reason. We would be better off watching someone else’s playthrough, something the algorithm chose from the impending flood of AI-generated content.
Let me offer one final example to illustrate an interesting dynamic of the Goldilocks problem. I’ve been evaluating a class of generative AI startups that help non-technical people write SQL (e.g., AirOps) or Excel formulas (e.g., Formula God). Defensibility aside, I don’t think these startups make sense. Users don’t need them for very simple queries or formulas, but more importantly, they may not know how to describe more difficult operations. A well-defined prompt could be effectively the logic needed to execute the analysis. But as any programmer knows, coming up with the pseudocode is often more difficult than writing the actual code. In learning how to use these tools more effectively, a user can become sufficiently technical to not need them. This bypassing of the Goldilocks stage is a phenomenon I call “graduation risk”.
Because of the Goldilocks problem, I think generative AI founders will have an easier time building for professionals than for consumers or prosumers. Users who are experts already know what they need and how to craft well-defined prompts, so a founder can “simply” focus on creating a verticalized platform that meets those needs using generative AI in an opinionated way. While professional-facing startups are more likely to succeed, I don’t think investors should avoid consumer. They just need to reconsider generative AI as a temporary surrogate for people, who are better at interpreting and servicing ill-defined prompts. But prosumer always faces graduation risk by definition and should be avoided.
I’ve characterized the Goldilocks problem by user type for simplicity, but it doesn’t matter as long as a company can translate its users’ intentions into well-defined prompts somehow. If you think you’ve found an interesting way to do so, email me at jiwang[at]dcm[dot]com. I’d love to hear how you solved the Goldilocks problem. ∎
