Pharma | Biotech | AI | Hallucination | Asset valuation
AI Hallucinations in Clinical Trial Predictions, Asset Valuations and Portfolio Management
Written by
Remco Jan Geukes Foppen is an international business executive in AI and life sciences, with proven expertise in the life science and pharma industry. He led commercial and business initiatives in image analysis, data management, bioinformatics, clinical trial data analysis using machine learning, and federated learning for a variety of companies. Remco Foppen has a Ph.D. in biology and holds a master’s degree in chemistry, both at the University of Amsterdam. He can be reached on LinkedIn.
Vincenzo Gioia is business and technology executive, with a 20 year focus on the quality and precision for the commercialization of innovative tools. He specializes in artificial intelligence applied to image analysis, business-intelligence and excellence. His focus on the human element of technology applications has led to high rates of solution implementation. He holds a master degree at University of Salerno in political sciences and marketing. Please check his innovative views on the application of AI at LINK.
Intro — Developing medicine is a high-risk endeavor. Clinical trial predictions, asset valuations, and portfolio management are crucial parts for the pharma industry to ensure a sustainable business and contribute to society. Traditionally these are largely manual processes.
In traditional valuations, generic “first tier” data from therapeutic areas, patient population, market cap, market share, and total addressable market (TAM) are often used. Frequently this data is determined empirically. To give some context to this, in an ever-changing macroeconomic environment and considering the high failure rates of drug development, building business cases based on TAM alone is hard to sustain. Separately, data from therapeutic areas are not precise enough to determine discount rates (what investors expect in return for investment) and attrition rates (the rate at which a drug in clinical development does not reach marketing authorization). Both are adopted across multiple therapeutic areas and modalities to calculate the risk-adjusted net present value (rNPV is a common valuation method in the drug development industry).
AI-based scientific innovations and engagements are called upon across many medical affairs initiatives to offer enhanced levels of precision and strategic insight (How to predict a drug will transition to the next development phase of its clinical pathway? How to reduce risk of investment opportunities?).
The AI toolbox — In order to de-risk clinical trial predictions, asset valuations and portfolio management one would benefit from having a dataset to start with, even before doing any lab or clinical work. AI offers an accelerated, repeatable, and traceable decision-support process in such a complex market context. When we talk about AI we have to think about a group of technologies that can be adopted individually or in combination to do a specific task. This technology is like a toolbox that gives us the right tool at the right moment for the right problem.
As the AI toolbox expands with generative and multimodal AI, new possibilities emerge as well as risks. Where lots of time and value is lost using traditional tools to manage data, the AI toolbox is suited to iteratively and reliably source vast amounts of “next tier” data like medical, business, and other public domain data.
A heterogeneous source that has gained increasing interest from pharmaceutical, medical, and regulatory instances is Real World Evidence, for its promising impact on drug development, clinical productivity, and its potential to expand the known patient population.
“AI/ML could be used to scan the medical literature for relevant findings and predict which individuals may respond better to treatments and which are more at risk for side effects”
Patrizia Cavazzoni, M.D., Director of the Center for Drug Evaluation and Research, FDA.
Secondly, Artificial Intelligence and Machine Learning (AI/ML) systems can map a network of billions of relationships across millions of data points and thus offer incredible value in making predictions, recommendations, or decisions influencing real or virtual environments. In particular, as biomedical data is notoriously messy, it is hard to normalize multiple sources into one high-quality knowledge base to power biologically sound modeling. The use of visual network graphs opens up a world of possibilities for interactive conversations, and thus an enhanced democratized user experience.
Tides of data — The flow of data comes from more than 200,000 clinical trials, 20,000 prescription drugs, and 7,000 clinical conditions. This flows into next-tier data, where therapeutic insights can be drawn from. Furthermore, AUTM, formerly known as the Association of University Technology Managers, lists over 31,000 assets available for licensing. BioTechGate has 100,000 assets in the database of which about 43,000 are listed for partnering.
The tide can also ebb for many reasons. These reasons can even be compounded. Anyhow, a recurring reason in clinical trial predictions, asset valuations, and portfolio management is strategic selection. This is the filtering, cutting, and stratification by therapeutic area. An equally persistent ebb is the difficulty of reading large portions of unstructured data.
The total universe of assets and data is rich, and can only be guessed as most assets and data repositories are private or non-public. This wealth of assets and data is only enhanced with the associated metadata, and interconnective data points among data sets. Ultimately the universe of data is growing overwhelmingly.
What could happen during low tide? — Common belief is that AI gives the best results, the more data is available. The way the AI model works is also important. The model autonomously makes relations and clusters data. The background of the cognitive AI model can become difficult for humans to follow. On this path, an AI must be considered closer to an “Oracle” than to a process based on a cause-effect relationship.
What happens if we shrink the data universe? This question doesn’t have a single answer because the AI systems can be in three different conditions:
- I know what we are talking about and this is my response
- I don’t know what we are talking about
- I think I know what we are talking about and this is my response
AI systems that allocate the response in the third condition mentioned above, are prone to generate a response that is unexpected or incomprehensible. This can sometimes be referred to as AI hallucination. This is a response provided by the system with a high degree of confidence, but which is not supported by the data on which it was trained. This phenomenon is called “hallucination” because it is similar to what happens when a person is under the influence of drugs or alcohol. There are several reasons for AI to hallucinate, and here we will elaborate on the AI deviating from the dataset, due to lack of sufficient data, and is thus guessing. Quality of the AI outcomes is addressed separately.
AI hallucination-type symptoms in clinical development during low tide — Some clinical developments inherently have either less data to start with or less comparable data to make a prediction on. As examples, two areas of drug development come to mind when thinking of small datasets: First In Class (FIC) — and rare disease-drug development.
Best-in-class (BIC) drugs enter the market after a FIC, and provide a superior therapeutic effect in terms of safety and efficacy than a previous generation. Therefore one can imagine a baseline level of data being available for BIC. FIC uses a “new and unique mechanism of action” to treat an unmet medical need that didn’t exist before. Whereas one might expect there to be little reference data for FIC, there is still a strong basis of biomedical data: molecular biology, signaling pathways, and biological processes.
Rare diseases are diseases that affect a relatively small population. Due to this rarity, drug development faces the obstacle of a small volume of data, and more specifically, a small set of detailed epidemiological data. Major efforts are ongoing to create centralized and comprehensive knowledge repositories, where collaboration is facilitated and drug development is accelerated.
Are FIC and rare diseases the only places where AI hallucinations could occur? Probably not. Along the same lines of thought, one can also imagine, evaluating innovative clinical trials that employ strategies, which have not been used in previous tracks.
Final thoughts — Whereas clinical trial predictions, asset valuations, and portfolio management are data-rich and moving towards a data-based ecosystem, AI won’t produce high levels of precision and strategic insight overnight. Besides the new skills and competencies required from people applying AI in clinical trial predictions, asset valuations, and portfolio management, high-quality datasets within a good information architecture will need to become available. With this in place, AI models can correctly be contextualized for appropriate use (“the right tool at the right moment for the right problem”), and avoid being subjected to the situation of “I think I know what we are talking about and this is my response”.
The high tide of increasing the availability and requirement of next-tier data allows AI to superimpose next-tier data on first-tier data. The wealth of next-tier data by itself already offers additional insights into clinical trial predictions, asset valuations, and portfolio management. Take discount rates as an example: Say 18% is the empirically determined magic number for a discount rate, it should matter how you get to that number, given that a single percentage point would have quite an impact on rNPV. Similarly, this holds for other first-tier data. Therefore, the inclusion and analysis of next-tier data will provide more precision to empirically determined first-tier data. Clinical trial prediction would also benefit the drug monitoring committees, and their changing role in drug development programs. Separate insights could also include identifying key factors influencing trial success, trial design, and tracking competitors.
Should one even consider assessing the risk of AI hallucinations? In 2022, FDA’s Center for Drug Evaluation and Research (CDER) approved 37 novel drugs, of which 22 were identified as FIC and 20 were approved to treat rare or “orphan” diseases. 12 novel drugs were both FIC and rare. This shows the importance FDA gives to FIC and rare diseases for drug development and approvals. Therefore one is expected to assess the risks of AI hallucinations that come with the applications of the AI toolbox in clinical trial predictions, asset valuations, and portfolio management, where the promise of enhanced levels of precision and strategic insight is encouraging.
Reach out to us
Use the links below to check out our other content, learn more about AI and asset valuations, reach out to us about projects, or just to say hi.
Correct But Misleading: AI Hallucinations In Complex Decision-Making
New FDA Draft Guidance: Data Monitoring Committees In Clinical Trials
How would AI address Transparency and Bias in asset valuations?
Are you certain you need an AI?
Generative AI and its impact on thought generation and human creativity