How to make your healthtech product a success, part 2: Creating a product that doctors will like

Article by Anna Klepchukova, MD, Chief Medical Officer at Flo Health Inc.

This is the second article in the series on “How to make your healthtech product a success.” Read the first article here.

To create a medically credible healthtech product, you first need to understand exactly what medical credibility looks like for different groups of people — namely, doctors vs. typical app users.

Doctors have extensive knowledge about medicine, clinical safety, and scientific evidence and pay attention to these details when deciding if a product is medically credible or not. Medical credibility is important for users too, but they judge it based on different factors. I’ll share more on that in my next article in this series.

You can think of it like shopping for a car: Compared to an automobile engineer, the average person will look at different components of the car’s design to decide if it’s high quality and worth buying.

What is medical credibility for doctors?

Doctors like me ask the following questions to decide if a healthtech product is medically credible:

1. Does it follow standards of evidence-based medicine?
2. Are medical experts involved in the product development?
3. Have there been scientific studies done to evaluate the effectiveness of the product?
4. If required, does it have regulatory approval?

I’ll go through each of these and explain how Flo and other healthtech companies follow these guidelines to convince doctors that their product is medically credible.

1. How to follow standards of evidence-based medicine when developing a healthtech product

“Evidence-based medicine” refers to the use of scientific research to guide health care decisions (1). To follow these standards and be considered credible when communicating health advice, you need to cite medical guidelines and recent studies on the topic.

In a nutshell, advising users who are trying to conceive to eat blueberries because your friend did it and got pregnant quickly is not an evidence-based approach. Citing a 2018 study published in the journal of Human Reproduction that found that women who ate 3 or more portions of fruit per day had better fertility and a lower time to pregnancy than those who ate less is (2).

• Use reliable sources and cite them

Doctors can quickly determine if an article is written based on evidence by looking at the sources. When talking about reproductive health, cite information from credible medical associations like the American College of Obstetricians and Gynecologists, not lifestyle magazines like Teen Vogue or SELF.

Medically credible sites such as WebMD list references at the end of content and refer to reliable sources in the medical field.

Source: webmd.com

Credible sources of medical information include scientific studies published in reputable peer-reviewed journals and medical guidelines from organizations like the World Health Organization and Centers for Disease Control and Prevention. For instance, Flo has a whole list of trustworthy sources that we refer to when creating content.

• Ask medical experts to perform a review

To build medical credibility, you also need to have experts review your product features and content for accuracy. Medical experts are the best people to determine where the information should come from, where to look in scientific research for said information, and add any missing info based on their clinical experience.

At Flo, medical review is at the heart of everything we do. Because we aim to improve the health and well-being of women and people who menstruate around the globe, there are many aspects of our app and website that require an evidence-based approach and collaboration with experts.

When creating new content or features, we follow three steps:

1. Develop the content/feature based on the latest medical guidelines and scientific research.
2. Send content/feature for internal review with the Flo medical doctors (team members who work at Flo full time).
3. Conduct a final check with external medical experts with specializations and long clinical practices in that particular field (like a practicing clinician who specializes in breast cancer).

• Update content as new information becomes available

Finally, following an evidence-based approach requires staying up to date on the latest medical guidelines and research in your field. Health information is constantly evolving as science advances and new discoveries are made. When facts change, your content and product features need to reflect those changes. At Flo, we conduct regular reviews of existing content to update it as new facts come out and remain open to feedback from users and the medical community if they find statements in the blog or app that are incorrect or outdated.

2. How to use medical experts when building a healthtech product

It has become standard in the healthtech industry to include doctors in product development from the start. There are a few ways to do this, depending on the type of product you’re building:

1. Create an in-house team of doctors.
2. Build a medical board of external doctors who work in the field and advise your company on a part-time basis.
3. Use both in-house doctors and an external medical board.
4. Hire doctors for virtual health care services.

• Create an in-house team of doctors

For more complex products where medical experts, engineers, developers, and designers need to communicate on a daily basis to create features like AI-based disease detection algorithms, it’s necessary to have an in-house team. You can’t call a doctor on your medical board once per month and expect them to have a deep understanding of your product’s development. Rather, hire doctors for your internal team to make certain that all aspects of the product are medically sound and evidence-based.

Companies that are building complex solutions, such as Ada Health Inc. and Babylon Health, have big in-house teams of clinicians. Clinicians also take strategic senior positions to make sure the product is effective and safe.

To see this in action, let’s take a look at how Ada Health presents its medical team. Ada has expert involvement in every aspect of the company, from the leadership to the content and product.

Source: ada.com

And here is an example of expert involvement from Babylon.

Source: www.babylonhealth.com

• Build an external medical board

Another possible route is to set up a medical board of practicing clinicians who work with your company on a part-time basis to review your materials. This is a good option for healthtech companies whose core focus is to educate users — for instance, through a website — but that don’t need expert involvement to prototype, test, and clinically evaluate product features.

BabyCenter has a team of over 40 medical advisors plus two health organizations that they work with to conduct reviews of their content and ensure that all of the information presented is accurate and safe for their readers.

Source: babycenter.com

• Build in-house and external teams of doctors

If your company is like Flo and has both complex product features and educational health content, in my opinion, you should have an internal and external team of experts. Our medical affairs team is made up of 15+ doctors and research scientists. Their job is to ensure that all of our content and features are medically accurate, up to date, and safe for our users worldwide. We recently added a director of medical accuracy and a director of science to our team to help us on this mission.

Some of our team members focus on content development. They collaborate with our content and marketing teams to decide on new ideas for articles, courses, and videos; edit content; and conduct internal reviews. Others have both a medical background and a strong understanding of product design, so they work together with our developers to create prototypes of new app features and co-develop medically sound algorithms for our cycle predictions.

Once a new feature or content piece is ready, we send it off to our external medical board for independent review. If it’s a new chat in the app about endometriosis, we involve experts in that area, not a neurosurgeon. It’s important for us to get unbiased opinions from professionals working in the field to challenge us and point out any gaps or where something may not work.

Flo’s medical board is made up of over 100 certified health experts from different specialties. We mainly work with experts from the United Kingdom and the United States who have at least five years of clinical experience and are known for great work in their fields.

Source: flo.health

We are currently expanding our Flo in-house medical and scientific affairs team, so check the open positions and apply to join us!

• Hire doctors for virtual health care services

If you are building a telehealth company, you’ll need to hire doctors to provide digital medical services. An example of this is BetterHelp, owned by Teladoc Health, which has the world’s largest network of licensed therapists to help people with a wide range of mental health conditions from the comfort of their homes.

Source: betterhelp.com

3. How to scientifically evaluate your product

• Why do you need scientific research to build credibility for the medical community?

Evaluating digital health solutions is crucial for 2 reasons:

1) To find out if the healthtech product will do what you want it to do (for example, reduce stress, help to cope with insomnia).

2) To get substantiation of marketing claims, you will need for your ads. You can’t just put “Our app helps to reduce weight in 5 days” on a promotion. You have to conduct a research study to prove its effectiveness. Otherwise, you will have problems with organizations and regulatory bodies like the Food and Drug Administration (FDA) and Federal Trade Commission.

Here is a good example of the use of scientific research to support the marketing claim “improve sleep quality” by Calm.

Source: apps.apple.com/us/app/calm/id571800810

• What types of scientific research are there, and how do you choose the best one?

There are a few different types of research to evaluate healthtech products: qualitative, descriptive, and comparative studies.

Choosing which one to use depends on several factors:

1) The type of claims you aim to generate

To get a generic claim, such as “Our weight loss app is loved by 90% of users,” you will need a simple survey (see this cross-sectional survey of Calm app users as an example) (3). To get more serious claims, e.g., “Our weight loss app helps users with diabetes lower their blood glucose levels by 30%,” you need a comparative study (preferably a randomized control trial, such as this RCT conducted on adults with a binge-eating disorder using the Noom app) (4).

2) The stage of the product development process

When developing a product, qualitative studies are particularly useful, as they allow you to deepen your understanding of how to improve your product for your target users. If your product is already on the market, you may need to prove its real-world effectiveness with a comparative study.

3) The intended audience

If your product targets vulnerable populations, such as individuals affected by a health condition, more rigorous proof of efficacy may be needed (via an RCT) for the patients to feel the product is safe and does what the developer claims it does. Proof of efficacy may also help health care providers decide whether to endorse your product or even adopt it (see this RCT on using an internet-based solution for patients with insomnia and depression) (5).

4) The presence of risks posed to users

Products that intend to treat specific health conditions, especially for more vulnerable groups (e.g., adolescents), will typically be classified as medical devices and will require extensive, rigorous research to establish their safety and efficacy (see this RCT run on an internet-based solution for adolescents with depression) (6). Conversely, products with lower risks will require less rigorous methods (see this pre–post study on a mindfulness-based tool to reduce rumination in adolescents) (7).

5) The product’s goal

If your product is an educational resource, then less rigorous studies would be needed for its evaluation (see this study on the development of an app aiming to increase awareness of the nutritional values of foods via games) (8). If you aim to deliver an intervention, however, comparative studies would be required to allow you to say whether the intervention has the intended effect (see this RCT evaluating the use of a mobile app in a cohort of pregnant women to reduce levels of gestational diabetes) (9).

To learn more about the different study designs, refer to this guide published by Public Health England (10) and this article by Sonia Ponzo, director of science at Flo Health Inc (11). Whichever study design you choose to evaluate your product, it’s essential to back up any claims you make with scientific research.

• How we evaluate the Flo app through research

At Flo, we use several tools for the scientific evaluation of our product.

We conduct simple, descriptive studies (e.g., cross-sectional surveys) that tell us what people think about the Flo app at a specific point in time, what motivates them to engage with the app, and what they feel is most helpful. Cross-sectional surveys can also help identify user needs that can provide meaningful insights for product development down the road. On top of that, we run qualitative research, including interviews, focus groups, and usability testing, to learn how users feel about our app, how they interact with it, and what matters to them.

We are now planning more rigorous research studies, including RCTs, to evaluate the effectiveness of Flo on various health outcomes (e.g., knowledge about reproductive health, well-being, productivity, and mood).

If you want to collaborate with the Flo Scientific team on a research project, don’t hesitate to apply here.

4. The role of regulatory approval for medical credibility

If our aim is to build medical credibility in the eyes of doctors, it’s important to keep in mind that medical credibility is not equal to regulatory approval.

• What is regulatory approval, and why might a healthtech company need it?

Healthtech companies bear the responsibility of dealing with the most important asset we all have: health.

Governments keep a close eye on companies engaged in and associated with the diagnosis, cure, mitigation, treatment, and/or prevention of diseases or other health conditions (products that can be classified as medical devices) to ensure that everyone receives a safe and effective product.

Each country has its own regulatory body — an institution that controls healthtech products and can impose different levels of pre- and post-market requirements on these companies.

For example, in the United States, this role is played by the FDA; in Brazil, by the National Health Surveillance Agency (ANVISA); and in the United Kingdom, mainly by the Medicines and Healthcare Products Regulatory Agency (MHRA).

In the European Union and the United Kingdom, in addition to each of the countries’ regulators (like the MHRA), companies go through a mandatory third-party conformity assessment performed by the so-called “notified bodies.” While in the United States, the FDA takes full control of device approval, in the European Union, the job is mostly done by third parties (notified bodies like BSI and TÜV Rheinland).

In the European Union, medical devices need a CE Mark certificate, which proves that this device meets the European health and safety standards under the Medical Devices Regulation (MDR).

• How do you know if your product needs regulatory approval?

As a digital health product manufacturer, the first step is to scope:

- the device’s functionality (e.g., a food tracker to log what a user has eaten)
- the intended use (e.g., to count daily calorie intake)
- what you are claiming (e.g., our app will help users to count calories and lose weight)

This scope will determine if the product is considered a medical device or not. Not all health apps are considered medical devices by law and, therefore, are not subject to most regulatory obligations.

The regulatory pathway for digital health products isn’t exactly straightforward. Here, it’s essential to have a strong regulatory strategy with a solid understanding of the peculiarities of each country’s regulations where the company plans to operate.

To achieve that, you can hire a regulatory consultant or create an in-house regulatory team.

• What is the value of regulatory approval?

When you’ve gone through the whole process, and you finally receive that stamp of regulatory approval, make sure to showcase it loud and proud on your website. That certification will go a long way toward building your medical credibility among doctors and users alike.

Take a look at how Natural Cycles, Babylon Health, and MindDoc present their certifications:

Source: naturalcycles.com/the-science

Source: babylonhealth.com/en-gb/regulatory

Source: minddoc.com/medizinprodukt

Conclusion

Getting doctors to view your healthtech product and company as medically credible boils down to four things:

1. evidence-based medicine
2. expert involvement
3. evaluation through scientific research
4. regulatory approval (if applicable)

Nail these factors, and you’ll be well on your way to making a name for yourself as a trustworthy and effective product within the medical community.

Sources:

1. Tenny, Steven and Matthew Varacallo. “Evidence Based Medicine.” StatPearls, 30 Oct. 2021, PMID: 29262040.
2. Grieger, Jessica A. et al. “Pre-pregnancy fast food and fruit intake is associated with time to pregnancy.” Human Reproduction, vol. 33, no. 6, 01 Jun. 2018, doi:10.1093/humrep/dey079.
3. Huberty, Jennifer, et al. “Can a Meditation App Help My Sleep? A Cross-Sectional Survey of Calm Users.” PLOS ONE, edited bySergio A. Useche , vol. 16, no. 10, Public Library of Science (PLoS), 22 Oct. 2021, p. e0257518. Crossref, doi:10.1371/journal.pone.0257518.
4. Hildebrandt, Tom, et al. “Randomized Controlled Trial Comparing Health Coach-Delivered Smartphone-Guided Self-Help With Standard Care for Adults With Binge Eating.” The American journal of psychiatry vol. 177,2 (2020): 134–142. doi:10.1176/appi.ajp.2019.19020184.
5. Christensen, Helen, et al. “Effectiveness of an online insomnia program (SHUTi) for prevention of depressive episodes (the GoodNight Study): a randomised controlled trial.” The lancet. Psychiatry vol. 3,4 (2016): 333–41. doi:10.1016/S2215–0366(15)00536–2.
6. Smith, Patrick, et al. “Computerised CBT for Depressed Adolescents: Randomised Controlled Trial.” Behaviour Research and Therapy, vol. 73, Elsevier BV, Oct. 2015, pp. 104–110. Crossref, doi:10.1016/j.brat.2015.07.009.
7. Hilt, Lori M., and Caroline M. Swords. “Acceptability and Preliminary Effects of a Mindfulness Mobile Application for Ruminative Adolescents.” Behavior Therapy, vol. 52, no. 6, Elsevier BV, Nov. 2021, pp. 1339–1350. Crossref, doi:10.1016/j.beth.2021.03.004.
8. Samuels, George, et al. “NuSwipe — Creating Nutrition Awareness in the Elderly through an Interactive Game App.” Pace.edu, Pace University, 4 May 2018, http://csis.pace.edu/~ctappert/srd2018/2018PDF/b2.pdf.
9. Kennelly, Maria A., et al. “Pregnancy, Exercise and Nutrition Research Study with Smart Phone App Support (Pears): Study Protocol of a Randomized Controlled Trial.” Contemporary Clinical Trials, vol. 46, Elsevier BV, Jan. 2016, pp. 92–99. Crossref, doi:10.1016/j.cct.2015.11.018.
10. UK Health Security Agency. “Evaluating digital health products.” Public Health England, 19 Oct. 2021, https://www.gov.uk/government/collections/evaluating-digital-health-products.
11. Ponzo, Sonia. “What Scientific Evaluation of an App Is and Why Do You Need It for Your Healthtech Product.” Medium, Flo Health, 6 Apr. 2022, https://medium.com/flo-health/what-scientific-evaluation-of-an-app-is-and-why-do-you-need-it-for-your-health-tech-product-cda2a7182e83.