Voice This! Podcast: Episode 6

Continuous Validation with Fariya Mostafa

Testing! Testing! Testing! What better way to figure out if your ideas are valid and your prototype makes sense than to test? Today we’re joined by Fariya Mostafa, who, at the time of recording this episode, was the Group Product Manager at Winterlight Labs for their Life Sciences product. Fariya is now a Product Manager at Legacy. This is our first episode looking at the application of voice in the healthcare space.

Disclaimer: Fariya is NOT a designer, she’s a product manager and we actually love her all the more for it. In this episode, she guides us through the day-to-day of working in product management, which may include: defining a target market, learning from users, building roadmaps, and monitoring key product metrics, among many other things.

Highlights

• [6:28] Fariya’s definition of continuous validation
• [12:46] Understanding the differences in cognitive assessments between pharma and senior care
• [17:07] Duration of the validation phase
• [21:04] Validation frameworks
• [26:26] Validating products vs. conversational products
• [30:59] The importance of hands-on work

[6:28] Fariya’s definition of continuous validation

I think continuous validation is just a practice that, that you must, in my opinion, and I’m sure there are others, it’s, it’s a practice that studios are trying to build products that improve them for sure, like hands down. But you know, in more technical terms, it’s a practice. It assumes that you are trying to build and improve on a product.

In a shapeshifting market, so users’ needs are always changing and you’re trying to learn what their pain points are, and you’re trying to solve for them, and you’re trying to make the whole experience better. Whether it’s in one particular target market or a new market, and, you know, needs will change even a particular market as new capabilities develop.

Whether it’s technology, whether you know, it’s just preferences or group habits that develop in a certain demographic. Just important no matter what products you’re building. Especially if it’s technology based.

[12:46] Understanding the differences in cognitive assessments between pharma and senior care

So what we learned is that whereas in the pharma use case, they wanna do research to find out what’s wrong with this particular patient. They also wanna do this in senior care. However, the senior doesn’t want to know if they have cognitive impairment. Right. Some of them are very proactive and they wanna know, but think about it like these retirement new people who are 84 plus for the most part.

They’re like, you know what? I don’t need to notice this and terrify myself. They might wanna know, okay, if something’s wrong, what could you improve to prevent me, for example, cognitive impairment. It plays a big role in a lot of bad stuff that happens. You miss medications, you’re more likely, you fall more likely to be hospitalized. Losing like your independent abilities.…But the idea is, so what we learned from that experience earlier on was that we, we really need to prove out one market first and build that use case developer technology in that domain.

So what we did ended up doing was we worked with, we have, and we still are the pharma market, uh, working with them to build something that could be used scalably in the context of research studies or other types. Studies, and we work with academics and physicians who are interested in doing research as well.

[17:07] Duration of the validation phase

So the validation goes very fastly. Depending on the market you’re in. If you’re doing B2B, again, this is a heuristic. It doesn’t apply to everyone. In a B2B, you probably want around six reference customers. When I say reference customers, it’s just someone who would be like a huge fan of your product, or at least a fan and and who are going to be using your product, integrating it into their daily workflow, like in a B2B model. So if they’re about six of them, then you’ve got product market fit. That is a higher level of validation. However, you could have a phase of market validation where you’re just testing and learning that people are willing to buy this. And, and at that phase you haven’t built anything.

Now, the other heuristic is that how much time it takes to validate, it’s like, Potentially twice your sales cycle. For example, if you think the sales cycle potentially for selling to a particular target customer is six months a, again, I’m talking about a B2B model. Obviously consumer products are different, then you probably need 12 months to validate it properly.

Q: Is that considered quick in terms of the healthcare space? Doing it within a year.

I think yes, because there are lots of regulations involved too. It isn’t just, healthcare is a very carefully regulated space and rightfully so. Maybe sometimes a little too, too regulated. But there’s barriers to accessing the type of data you want, like healthcare data, because you, you have to be compliant with certain regulations, legislations, different areas

[21:04] Validation frameworks

Q: So is there some sort of framework or techniques that you use or something that people use when it comes to validating products or features?

Exploratory Research without a Prototype

I think, so when you’re trying to figure out a target market, whether you’re entering a new market, this is the first market you wanna have customer interviews, but not with any prototype, right? Because you’re just trying to understand if you have hypothesis like this, hospital’s going to use your technology to figure out which of the patients, let’s say older adults are at risk of various bad outcomes….So whatever target customer you have, do you wanna talk to a group of them, depending on whether it’s B2B or B2C? If it’s B2C and you want 50 or more people, probably B2B, you want some amount of, number of organizations and you talk to people, then you ask them what they do in their day-to-day life, like today, how did it take care of this patient population? Is there anything that could be improved? What is it like? You keep it very open-ended and let them say whatever, do not prime them. That’s the very, it’s a very important thing to keep in mind. Don’t be opinionated. You’re really there to learn and understand them and their problem and take notes, and then of course, building up relationships that you can reach out to them if and when you do build something, once you have validated a first pass.

Testing low-fidelity prototypes with an internal team

Then, you know when you are building something, of course there are prototypes that you work with and there are different types of testing you do. So I’m gonna start with the prototype piece, but basically you could, again, I’m no designer, but like a broad stroke, so, Low fidelity prototypes, I think should just be used internally. Like when you’re trying to brainstorm with your dev team and your designers, that’s when you’re using that just to like figure out what it is and to conceptualize a solution.

Usability testing and metrics tracking

But the moment someone starts writing code, that becomes expensive, right? From an organizational perspective, both in terms of time and money. So when you do invest a time, Let’s say to do some sort of demo that is, I would say, a high fidelity prototype and you would use that. So it could be like you had some mockups from Fig Mine. You connected to the app and you asked someone to like just use the flow. So that’s the stage. I’m sure as researchers know more about those than I do, but you sort of hand someone a demo to work with this user and you observe them because the biggest learning is an.

What they’re doing, because there’s always this main action you want them to take. Are they taking it? How fast they get into it? What are their struggles? That’s what you’re screening for in the first pass of the high fidelity prototype. Then you have hybrid prototypes too, right? Where you are manually telling them, if you answer this question, you get her score.

And you have some developer, or maybe it’s the product manager, like inputting something and giving them a score. So that’s your hybrid. You do all that stuff. When it gets more mature, of course, you also then validate it at that stage. Then you want users to actually. So all the stuff I talked about before involved an aspect of usability testing, but you wanna go deeper, and this is where maybe an MVP is helpful.

Certainly a lot of tracking of product metrics and how they’re using it is crucial at that point. But you maybe hand whatever the solution is to like 10 users and say, this is an mvp. It’s like, but I want to see like how you use it in a B2B case, for example, in your workflow. What you learn, and this is something you know, even I’ve read elsewhere, you learn a lot from when a user misbehaves like they do something that you do not predict they would do.

[26:26] Validating products vs. conversational products

A lot of things are similar because in all cases you’re trying to understand is this usable? Is this a viable thing to build? Will a lot of people use it? How does it make the customers feel like in those aspects? And, and, and not just that, but also, you know, as you’re using a mature product, how users wanted to change if they do or what they don’t like.

So all of that is similar, but I think conversational AI is different because you are technically interacting with the product at a deeper level, and almost you’re interacting with, I wouldn’t say a person, but kind of a person depending on the, so it’s very different. I think one thing that you wanna avoid, probably, and this will really depend on the demographic and the nature of the person.

You don’t want the conversational AI to be intrusive. Like, you know, you almost want to, you want your privacy to be in task, so you don’t wanna be worried about. Is this listening to me? Is it gonna just start talking to me? Also, it’s not just voice space, even in a text. Like, you know when you go into a website and there’s just like bot talking, you’re like, I don’t why so, so you know, I mean these aspects, …you want the user to have everything they might want or need, uh, their tooltip, but you don’t want to impose. The technology on them. And I think that’s something that I don’t think, because I haven’t worked with web-based, but I would think that that’s not a problem or constraint you have to think of in those other scenarios.

[30:59] The importance of hands-on work

What I’m gonna say is there’s no substitute, as with developing software, there is no substitute for doing the work.

Literally no substitute. You can read everything you’re reading here. It’s probably generic, right? Maybe like 50, 20% will apply to the field. Uh, so the broad principles look applying anywhere. However, what is it that works for your users and for your market, if something, You can only learn when you get your hands dirty.

And it doesn’t have to be necessarily in a way that you’re a product manager of the company and that’s how you are developing the skill. Maybe you are in a different role of this company. Maybe you’re starting off helping out in customer support, you’re helping out with user research. Or if you’re someone not working at a company, but you have dev skills, maybe you just built something and you, there are many ways to get to this, but, but nothing will substitute that.

Resources

[Book] The Lean Startup by Eric Ries

About Voice This! Podcast

Conversations with the people who make conversational AI 🎙️Join Millani Jayasingkam and guests as they discuss voice technology, conversation design, AI trends, and the strategy of creating effective conversational experiences. Tune in for first-hand learnings, insights, anecdotes, and sometimes jokes! Say hi and send us your questions at: voicethispod@gmail.com.

Find us wherever you get your podcasts or Listen on Spotify!