Beyond the Gold Standard: How AI and Real-World Data are Transforming Medical Practice

Artificial intelligence (AI) in healthcare is often synonymous with cutting-edge innovation, but it’s not just about the future; it’s about the present — about what we can learn from the vast amounts of real-world data that already exist. While randomized clinical trials (RCTs) have long been the gold standard in medical research, real-world data offers a complementary perspective, particularly when it comes to understanding the effectiveness of treatments across diverse populations and settings.

這篇刊登在 JAMA 的文章探討了人工智慧（AI）和現實世界數據（RWD）如何在醫療場域中發揮影響力，補足所謂正統的隨機臨床試驗（RCT）的不足。雖然 RCT 仍是新治療方式的研究黃金標準，但它們絕大多數在高度受控的嚴謹狀態、條件下進行，可能無法反映真實臨床現場的多樣性。藉由分析來自電子健康記錄（EHR）和其他數據源的現實世界數據，AI 有機會幫助醫師比較不同的治療方法，與病人共同做出最符合需求的治療選擇，並確保價值鏈裡的所有人都能受益。此外，文章也強調醫師與 AI 開發者合作的重要性，確保 AI 工具的臨床相關性和數據安全性。AI 有機會成為一種「可擴展的特權」，將高質量的醫療服務延伸到更多的患者群體。AI 在醫療中的應用必須透明、公平，並以改善病人生活品質及功能為最終目標（復健科觀點）。

The Limitations of Randomized Clinical Trials: Why Real-World Data Matters

Randomized clinical trials have their strengths — they provide high levels of evidence, control for confounding factors, and establish causality. However, they also have limitations. RCTs often involve highly controlled environments and selected patient populations that may not reflect the diversity seen in everyday clinical practice. Moreover, they are expensive, time-consuming, and sometimes ethically or practically infeasible to conduct for every clinical question.

This is where real-world data comes in. By analyzing data from electronic health records (EHRs), insurance claims, and other sources, researchers can gain insights into how treatments perform in the messy, variable reality of clinical practice. As Dr. Atul Butte, a distinguished professor at UCSF and director of the Baker Computational Health Sciences Institute, puts it, “The future is to learn from the data.” This approach allows us to understand not just what works in an idealized setting, but what works in the real world — across different demographics, healthcare settings, and patient populations.

AI and Real-World Data: A Symbiotic Relationship

The synergy between AI and real-world data is undeniable. AI thrives on large datasets, and real-world data provides just that. At the University of California Health System, Dr. Butte and his team have amassed a staggering repository of data: 9.1 million patients, 1.5 billion drug prescriptions, and 50 million medical devices, all in a single central database. This dataset represents a treasure trove for AI, offering endless possibilities for analysis and discovery.

What’s particularly exciting about this approach is the concept of “comparative effectiveness at a new scale.” By comparing and contrasting different treatment pathways — whether it’s different drugs, devices, or biologics — AI can help identify which options are truly effective and for whom. This is not about setting up artificial experiments but about leveraging the natural experiments that occur in everyday practice. Each clinician, each department, and each patient presents a slightly different scenario, and AI can sift through these variations to uncover patterns and outcomes that would be impossible to discern through traditional research methods alone.

Beyond the RCT: Real-World Evidence in Action

The potential of real-world data extends beyond academic curiosity; it has practical implications for patient care. Consider a recent study published in JAMA Network Open, where Dr. Butte’s team conducted a massive comparative effectiveness analysis of secondary therapeutics for type 2 diabetes. By analyzing pair-by-pair comparisons across various second-line therapies, they were able to identify advantages and disadvantages that would be challenging, if not impossible, to discern through RCTs alone.

This approach doesn’t just answer questions about what works — it also helps us understand what doesn’t. Some drugs may have been approved based on RCTs, but real-world data can reveal whether they still provide value in today’s clinical landscape. With longitudinal data spanning over a decade, researchers can track long-term outcomes like heart attacks and strokes, offering a more comprehensive understanding of treatment efficacy.

Collaboration Between Physicians and AI Developers: A Path to Better Patient Outcomes

As AI becomes more integrated into healthcare, the collaboration between physicians and AI developers will be crucial. Physicians bring a deep understanding of clinical practice, while AI developers provide the technical expertise needed to harness the power of real-world data. Together, they can create AI tools that not only improve patient care but also enhance the efficiency and effectiveness of healthcare delivery.

However, collaboration must be approached thoughtfully. Physicians must be involved in the development process to ensure that AI tools are clinically relevant and designed with patient outcomes in mind. At the same time, developers must prioritize data security and patient privacy, adhering to regulations like HIPAA while also educating patients about how their data will be used. As Dr. Butte emphasizes, it’s our responsibility to ensure that the data is used “safely, responsibly, and respectfully.”

Is AI a Democratizing Force in Healthcare?

One of the most compelling aspects of AI in healthcare is its potential to democratize access to high-quality care. In many ways, AI can act as a “scalable privilege,” providing the benefits of expert care to broader populations, including those in underserved areas. For instance, AI-driven decision support tools, developed using data from top-tier medical centers, can be deployed in rural hospitals or low-resource settings, helping clinicians make better-informed decisions and improving patient outcomes.

This concept of “scalable privilege” is particularly relevant as healthcare systems strive to address disparities in care. While RCTs may not always reflect the diversity of the general population, real-world data can capture the full spectrum of patient experiences, from the wealthiest to the most disadvantaged. By training AI models on these diverse datasets, we can ensure that the benefits of advanced medical care are extended to everyone, not just a privileged few.

The Ethical Considerations of AI in Healthcare

While the potential benefits of AI are enormous, they come with significant ethical considerations. The issue of “hallucinations,” where AI models generate plausible-sounding but incorrect information, is a prime example. Ensuring the accuracy and reliability of AI-driven insights is critical, especially when patient care is at stake.

Moreover, transparency and fairness must be at the forefront of AI development. AI tools must be trained on diverse, representative datasets to avoid perpetuating biases that could exacerbate healthcare disparities. As we work toward scaling the privileges of expert care through AI, we must also ensure that the technology is accessible, equitable, and accountable.

Conclusion: A Future Driven by Data

The future of healthcare is undoubtedly data-driven. By harnessing the power of real-world data and AI, we can go beyond the limitations of randomized clinical trials to gain a deeper, more nuanced understanding of what works in medicine. This is not to replace RCTs but to complement them, filling in the gaps and answering the questions that traditional research methods cannot.

As physicians, researchers, and developers work together, the goal should always be to improve patient care — whether that means identifying the most effective treatments, reducing unnecessary care variations, or democratizing access to high-quality medical services. With thoughtful collaboration and a commitment to ethical practices, AI has the potential to revolutionize healthcare, turning data into actionable insights that benefit patients across the globe.

— -

📒 Compiled by — Sigrid Chen, Rehabilitation Medicine Resident Physician at Taichung Tzu Chi Hospital, Occupational Therapist, Personal Trainer of the American College of Sports Medicine.