Prompt-and-Deploy: Get your AI MVP to market within days (not months)

TL;DR Prompt-based AI development is revolutionizing product management by enabling quick model creation, deployment, and iteration similar to MVP practices, but caution is needed with high-stakes decisions, and real-world, production data is the most reliable for testing due to potential data shifts over time.

Greetings, fellow product managers!

In the competitive product landscape, agility and speed are crucial. In the conventional realm of product management, we often stress getting to market quickly to collect real-world data, iterate and gain competitive advantage. But how does look when building AI products? Let’s take a look!

Traditionally, in the machine learning arena, test sets have been foundational. They are used in both academic research to benchmark algorithms and in commercial applications to measure performance and ensure accuracy before and after deployment. However, with the advent of prompt-based development, the development process has been radically transformed. Prompt-based models, such as those used in zero-shot and few-shot learning, have reduced the need for extensive data sets and are disrupting the time-to-market norms.

This emerging approach, which could be dubbed as ‘prompt and deploy’, looks like this:

1. Develop a model: Use a text or visual prompt to develop a model quickly — possibly in minutes to hours.
2. Deploy: Run the model on live data swiftly but safely, potentially in ‘shadow mode’ where the model’s inferences are stored and monitored but not used.
3. Evaluate: If the model’s performance is acceptable, let it start making real decisions.

Now, comparing this to traditional product management, you can clearly see the parallel with developing a minimum viable product (MVP), launching it quickly, gathering user feedback, and iterating based on real-world data.

However, it’s important to note the caveat that comes with this fast-tracked process. While many deployments may not pose a risk, it is vital to be extra cautious with models that may make critical decisions in sectors like healthcare, criminal justice, finance, and insurance. In such scenarios, collecting a rigorous test set and validating the model’s performance before allowing it to make consequential decisions is indispensable.

Another point worth noting is the practical challenge posed by concept drift and data drift. Real-world product data can change over time, and so can the underlying data used for AI models. In these cases, the most accurate test data is actually production data, which further highlights the advantages of fast deployment and iteration.

To conclude, the ‘prompt and deploy’ approach in AI development presents exciting possibilities for product managers, particularly in terms of speeding up the product development cycle. The lessons learnt from this can be potentially transformative, fueling us to reevaluate and reinvent our traditional approaches to product development. The “Prompt and Deploy” approach can help product managers get to market quickly with an MVP, iterate, drive competitive advantage, and allow you to take a vacation sooner. And isn’t that really the most important takeaway here?! Happy producting 🤖💡🛠