Discover the role of an AI Product Manager
About Me
Hi, my name is Yuyu. I’ve spent the last decade exploring various corners of the data and AI landscape, from strategy development to model building and team leadership across different sectors. My roles have ranged from handling retention strategies at Citi Cards to building subprime lending products at Reach Finance, working in data science at Meta, and innovating supply chains at Walmart. Now, at Beamery, I lead AI product development.
Throughout these experiences, I’ve picked up numerous insights into AI product management that I’m keen to share, and hope to connect with others interested in this evolving field.
Introduction to AI Product Management
Unless you’ve been living under a rock, you’ve most likely noticed the public attention on AI. Gone are the days when artificial intelligence was the exclusive playground of tech aficionados and research labs. Today, AI is front and center in the tech arena, reshaping industries and daily life alike. Amidst this surge, there’s a growing spotlight not just on the technologies themselves, but on the people and functions that make the technologies possible and useful.
As AI becomes a more integral part of our digital landscape, the role of managing these products becomes both more challenging and more critical. We’re the ones tasked with translating cutting-edge technology into practical applications, ensuring that innovations not only dazzle but also deliver real value. Exciting? Absolutely yes. But what is AI product management really about? What makes for a good AI product manager and what are the pitfalls? As someone in the trenches of AI product management, I’d like to share some of my thoughts.
There are many ways to define an AI product manager, but in my opinion, an AI product manager is simply a product manager who works on AI products and systems. Furthermore, as artificial intelligence is a fairly technical domain, I consider AI product managers a subset of technical product managers.
A good AI product manager is therefore also a good product manager and a good technical product manager. Specifically:
Product Managers First:
We AI product managers are product managers first: we start with the problems and value, then the right solutions. We don’t look for nails with the AI hammer we have. We especially shouldn’t do so because we suddenly see a lot of hammers being sold.
For example, now that large language models (LLMs) have become the center of attention, and chatbots are a natural implementation to show its power, many people have started offering chatbots as their main AI functionality. But is a chatbot the best answer for the problem you are trying to solve? If you are looking for a way to showcase the power of LLMs to understand human languages and generate content like OpenAI did with chatGPT, then of course the answer is yes. If you are looking for a way to answer customer questions that may just be a little different from time to time, and save them the trouble of digging through your FAQs and piecing together disjointed answers, the answer is likely also yes. But is a chatbot the best answer to pull up repeated reports that people look at everyday? Probably making them easy to access with one click or directly on their workflows is a much better answer.
Sometimes, the optimal answers to customers’ questions cannot be solved by AI alone, and we must recognize it and advocate for the non-AI parts as well as the AI parts. When we started working on our new workforce insights products to help customers (who are large enterprises) understand what skills they need in their organizations now and in the future, pilot customers found the skills data we provided hard to make sense of, and thus doubted the accuracy of our AI skills inferences. Looking into what the customers were really saying, we identified the problem as two areas: one is that our AI skills inference did need an improvement, but the second one is that we need to think of better UI design to show the skills, so it’s easier for customers to understand and make sense of.
Within our improvement of the AI skills inferences, we realized that our inferences needed to use the context of groups of roles (such as job families) instead of the individual roles, and we need a new concept we called “capabilities” that can organize groups of skills to explain the tasks/scopes achievable with the skills. Our new combined solutions were well received by customers, leading to a 5X increase in perception of accuracy. We were able to build this new concept partly with LLMs, showcasing again that new AI capabilities can be used to solve problems in more ways than one, as long as we start with the problem and not the technology.
All in all, we are product managers, we start with the user values and problems to solve, and figure out what are the best solutions (AI or not) instead of forcing whichever AI functionality that’s on trend onto our products and customers. Because if we do that we’ll get stuck with a less-than-useful solution that now complicates instead of simplifies our workflows when the current hot trend is replaced, and it for sure will be replaced soon with the speed of AI developments today.
Technical Product Manager/System Thinkers:
As AI product managers, we need to be good at systematic thinking and consider the holistic impact of our decisions. Systematic thinking is important for technical product managers in general but is especially crucial in the intricate world of AI, where each component interacts with complex systems, and understanding these interconnections is key.
For me, it’s not just about choosing the most advanced AI technology; it’s about how this choice fits within the entire product ecosystem. Will it enhance the user experience? How does it impact data security? These are the types of questions we grapple with.
Specifically, if you are an AI product manager in a company that doesn’t solely work on AI products, you are likely to be working on AI platforms that enable other products or solutions within your company. In such cases, you also need to think about what impacts your solutions and roadmaps have on your partner teams and their end users.
In Beamery’s case, my team works on one of our flagship products, AI Talent Match, finding good candidates for jobs and good opportunities for candidates. This functionality lives in several of our product suites, serving the personas from talent acquisition teams, candidates, to talent executives/management teams.
When I think of our future vision and possibility with the functionality, as well as the things to consider in every update or new version of the algorithm, I need to think beyond the functionality itself and consider what impact this would have on the other teams’ roadmaps and the customer personas they serve.
Our role demands a broad perspective, where we consider not only the technological implications but also how they align with the product’s vision and user needs. It’s this holistic approach that enables us to create solutions that are not just technologically sound but also meaningful and seamlessly integrated into the user’s world.
AI Product Manager
However, we still need to be quite familiar with the AI technologies, the algorithms and models that are being created and can be utilized in the world, and especially at our companies. We would need to be aware of what’s possible once we know what is valuable.
We may not specifically take an AI hammer and look for nails, but we should know what the technology would allow for us to make, so when we do have a need for hammers we can know what’s the best fitting hammer, and when we see a need for pliers instead we’ll know whether that’s possible and, if not, what is the next best answer.
What are large language models? What are recommender systems? What is reinforcement learning? What are the limitations and super powers of all the AI technologies developed or being developed that could have an impact on the vision and mission of your area of work? There is always a lot to learn, so curiosity is definitely a good trait to have as an AI product manager.
Understanding both the capabilities and limitations of current AI technologies is crucial for guiding our products in the right direction and setting realistic expectations with customers. For instance, I often point out that while AI excels at providing estimates that are generally correct, it struggles with delivering the exact answers users might seek. This is due to the inherent trade-off between accuracy and precision found in AI solutions. Therefore, in scenarios where both high precision and accuracy are critical — and where the stakes are high if we get it wrong — it’s essential to integrate AI with complementary solutions that enhance its reliability.
Parting Words
Navigating the world of AI product management is a bit like solving a complex jigsaw puzzle. Every piece represents a different challenge — be it blending the latest tech with user needs, balancing cutting edge work with systemic stability, juggling various stakeholder demands, or staying ahead of the fast-moving AI curve. It’s a hands-on, minds-on kind of role, where the satisfaction comes from both the grand visions and the nitty-gritty of making things work.
Sure, it’s demanding. Balancing technical possibilities with practical realities means we’re often walking a tightrope, trying to find that sweet spot where innovation meets utility. But here’s the thing — it’s also incredibly rewarding. There’s a unique kind of fulfillment in piecing together a solution that clicks, in seeing your products make a real difference, however small, in people’s lives. It’s about holding a greater vision in sight, and celebrating those small victories, the incremental improvements that, piece by piece, build the path forward in the AI landscape.
Are you interested in joining Beamery or being part of our fantastic Engineering, Product and Design team? Visit our open roles and apply today.
