HealthPals’ CLINT Demonstrates First Large-Scale RCT Replication with Veradigm’s EHR Data

In 2020, COVID-19 changed how we approach healthcare in the US and globally. Seemingly all of our energy and attention is now trained upon containing the pandemic, as well as treating — and now vaccinating — patients across the globe. Lost in the threat this virus has presented to our immediate vitality is the long-term impact on future health and the clinical trials that power new treatments for COVID-19 and non-COVID-19 treatments.

Key Takeaways:

• Approximately 80% of non-COVID-19 clinical trials have been suspended, stopped, or delayed due to difficulties in trial initiation and recruitment.
• Solutions to these pandemic-induced challenges can be found in Real-World Data (RWD) and Real-World Evidence (RWE).
• CLINT™ (HealthPals, Inc.) is a novel, guideline-encoded population health platform capable of generating machine learning-based insights on RWD at scale.
• To demonstrate the capabilities of CLINT™, particularly its ability to leverage RWD for the generation of ECAs, HealthPals has successfully duplicated a large randomized controlled trial (ROCKET-AF) using Electronic Health Record (EHR) data.
• This replication establishes CLINT™ as the first operational RWD analytics platform capable of replicating RCT results in EHR data.

Randomized Controlled Trials (RCTs) for drug and device development are long, expensive endeavors with a high rate of failure due to frequent design and recruitment challenges. Even prior to COVID-19, clinical trials were rapidly increasing in duration, complexity, and cost. In a 2016 publication, the Tufts Center for the Study of Drug Development (CSDD) found that, from 1990 to 2010, the total capitalized costs associated with clinical trials for pharmaceutical compounds were increasing at a real rate of 8.5% annually [1]. In a 2020 study, the CSDD found that, even with faster new drug approval phases, clinical trial times are taking longer. Data from 2014 -2018 shows that, while the mean approval phase decreased by 1.9 months, the overall trial times increased by 6.7 months [2]. Despite the massive investment in both time and money, clinical trials carry with them considerable risk of regulatory failure. A 2016 CSDD study also showed that, of drugs entering clinical testing, the probability of receiving FDA approval is only 11.83% [1].

In a 2020 article in the Lancet, Aaron van Dorn describes how the COVID-19 pandemic has severely affected the ability to conduct trials in safe and effective ways [3]. Thousands of trials, approximately 80% of non-COVID-19 trials, have been suspended, stopped, or delayed due to difficulties in trial initiation and recruitment. Meanwhile there has been a dramatic reorientation in clinical trials research towards COVID-19. This combination of factors ensures that the impact of COVID-19 on clinical trials will be long-lasting, and overcoming the inertia to recover a previously bustling industry may prove daunting.

Research firm GlobalData tracks the impact of COVID-19 on the number of trials that have been disrupted, delayed, or had slow or suspended enrollment. They have found that, while these numbers rose dramatically as a result of COVID-19, adjustments in clinical trial design strategies made by contract service providers and sponsors are enabling some trials to resume.

Figure 1: A timeline of clinical trials disrupted due to Covid-19. Source: GlobalData

To summarize, COVID-19 has stymied RCTs in nearly all non-COVID-19 arenas. Once we do overcome this global pandemic, we will be faced with enormous voids in other areas of clinical research that will require unique and creative use of real world medical data to help jumpstart — and finance — medical research.

A Solution to the COVID-induced RCT dilemma: RWD/RWE and External Control Arms

Real-World Data (RWD) and Real-World Evidence (RWE) have demonstrated their potential value in the R&D, clinical trial and regulatory spaces in a number of major ways. The FDA has used RWD and RWE to monitor post-market safety and adverse events and to make regulatory decisions while medical product developers have used RWD and RWE to support clinical trial designs (e.g., large simple trials, pragmatic clinical trials) and generate observational studies to produce innovative, new treatment approaches.

EXTERNAL CONTROL ARMS

External control arms (ECAs) are a specific use case for RWD/RWE in which patient cohorts are derived from external, real world data to provide a comparison control arm for an experimental arm in a clinical trial. ECAs are matched to experimental arms in such a way as to simulate the effects of randomization by:

• Reducing bias associated with confounding factors by distributing those factors equally across experimental and control groups.
• Facilitating causal inference.
• Providing the basis for statistical inference.

ECAs can be used to reduce the necessary sample size for the study control arm and thereby reduce the duration and cost of trial associated with patient recruitment. In addition, ECA’s can be used to supplement submission to regulatory bodies and help to mitigate the risks associated with regulatory approvals.

“We are in desperate need of a platform that can both accelerate time-to-market and lower R&D and RCT costs…RCT cohort and clinical trial optimization and next-generation real-world control arms that have scalable and regulatory-grade results.”

Despite the obvious value of RWD/RWE, the development of robust, high throughput, RWD-powered R&D and RCT support tools has eluded the industry. We are in desperate need of a platform that can both accelerate time-to-market and lower R&D and RCT costs via intelligent RWD analysis and insights, RCT cohort and clinical trial optimization and next-generation real-world control arms that have scalable and regulatory-grade results.

ECA Technology: Duplicating the Rocket-AF Trial

To demonstrate the ability of CLINT™ to leverage RWD, generate ECAs and unlock the potential of clinical RWD (and specifically EHR RWD), HealthPals, in collaboration with Veradigm, a division of Allscripts and owner of the largest source of de-identified ambulatory longitudinal patient EHR records available (over 150M patients), duplicated the ROCKET-AF study, a large, landmark, cardiovascular clinical trial using EHR data [5]. For full details on the methodology and results, download the white paper here. RCT replication was accomplished using the HealthPals CLINT™ platform to:

• Operationalize ROCKET-AF inclusion and exclusion criteria (operationalization is the process we use to map EHR data elements to clinical concepts).
• Identify eligible experimental and control patients.
• Match control patients to the experimental cohort using methods capable of balancing the cohorts with respect to confounding variables.

These capabilities can be applied to the generation of a clinical trial ECA.

Figure 2 below illustrates the process by which experimental and control arms were generated. 644,877 patients in the Veradigm were identified as having atrial fibrillation (AF). Applying the ROCKET-AF inclusion and exclusion criteria to the data set yielded 324,634 patients, and restricting to those with adequate wash-in and follow-up time who also had either warfarin or rivaroxaban prescriptions resulted in 102,055 patients. Of these, 65,815 were on warfarin at the time of eligibility and 36,240 were on rivaroxaban. The warfarin patients comprised the control group while rivaroxaban patients comprised the experimental group.

Figure 2: Flowchart cohort diagram detailing the critical filtering steps and the resulting number of patients at each stage of the process.

The experimental and control arm cohorts were then balanced with respect to prognostic variables using standardized mortality ratio weighting (SMRW), the most effective of the several cohort-balancing methodologies considered.

The SMRW approach was able to effectively balance both binary and continuous features across the experimental and control arms. Figure 3 below shows the SMRW-weighted distributions of continuous prognostic variables. The similarity of these distributions illustrates that the arms are well-matched and that prognostic variables are balanced between groups.

“This is a powerful result that enables the direct inclusion of real world data in clinical trial analysis.”

Figure 3 Experimental (Rivaroxaban) and Control (Warfarin) Arm Cohort Comparisons

It is evident that the methods employed are capable of matching real-world control arm patients to an experimental arm, as well as balancing prognostic variables, to simulate the effects of randomization. In doing so, a control arm is created from RWD that can be directly compared with an experimental arm and used to infer statistical relationships. This is a powerful result that enables the direct inclusion of RWD in clinical trial analysis.

“In doing so a control arm is created from real world data that can be directly compared with an experimental arm and used to infer statistical relationships”

With real-world experimental and control arms identified and balanced, we analyzed the longitudinal patient data to calculate the main composite outcomes measured in the ROCKET-AF trial: the primary efficacy endpoint, a combination of stroke and systemic embolism, and the principal safety endpoint, a combination of major and clinically relevant non-major (CRNM) bleeding. The rates of outcomes per 100 patient years were calculated for RWD control and experimental arms and were compared with outcome rates found in the ROCKET-AF trial. It was observed that stroke and systemic embolism outcome rates were similar between RWD and clinical trial patients, with both control (warfarin) arms showing increased rates of stroke and embolism (Table 1). While bleeding event rates were lower in RWD, the results were directionally consistent with ROCKET-AF results.

Table 1: Events Per 100 Patient Years

Figure 4: Cumulative stroke and embolism events in RWD (left) compared with cumulative events from the ROCKET-AF study (right) [5].

Cumulative event rate curves for the current study also display a high level of agreement with those from ROCKET-AF. Notably, the current RWD effort followed patients for more than 2,000 days after the index date, while the ROCKET-AF study reported only 840 days of outcomes after the randomization date. In the HealthPals results, the protective effect of rivaroxaban was demonstrated to be a full 4 years longer than what was reported in the original RCT. Clinically, this is important information for managing patients confidently over a long period of time, as many drugs exhibit regression to the mean after an initial benefit, whereas this anticoagulant does not. Similarly, this strength of RWD analysis allows for the evaluation of the longevity of the treatment effect of any drug, provided you can track the relevant outcomes for this determination.

These results demonstrate that, by applying our processes of concept operationalization, cohort selection, and cohort balancing to the Veradigm data set, we were able to faithfully replicate the methodology of the ROCKET-AF clinical trial in RWD and produce results very similar to those found in the study.

The Future of RWD/RWE and ECAs — and HealthPals’ Role

In the COVID-19 era and beyond, we must extract maximum value from available real-world datasets to help address the staggering costs and risks of RCT conduction, especially when resources and time windows will be severely restricted due to the pandemic. Currently, there is a paucity of analytic engines powerful enough to extract this value across disease states in a scalable, operational platform. The HealthPals’ CLINT™ platform is capable of generating highly detailed RWD analysis at scale. We present the duplication of ROCKET-AF, a large, landmark cardiovascular clinical trial as a demonstration of the power of this class-leading technology.

Operationalization is a difficult process by which elements in the EHR data set are mapped to clinical concepts. Typically this is done using codes alone. CLINT™ uses all available data to identify clinical concepts, providing for a more thorough and accurate capture of clinical variables. This technology was developed by the very close collaboration between clinicians and data scientists on the HealthPals team. The operationalization effort ultimately requires that the key elements are present in the data set. For this reason, it is important to work with a large, longitudinal data set like that possessed by Veradigm.

“This work represents the first example of an operational platform replicating RCT results using Electronic Health Records.”

In addition to supporting operationalization, HealthPals’ deeply integrated clinical and data science expertise enabled more efficient and effective mapping of inclusion and exclusion criteria and calculation of outcomes. In broader efforts, this deep integration of data science and clinical expertise enables more effective collaboration with clients, design of studies, development of analytical tools including machine learning models, and interpretation of results.

A critical element of the success of ECA work and RCT replication is the use of EHR versus claims data. Recently, the RCT Duplicate team published a study “emulating” several published RCTs using insurance claims data and showed good correlation with results [4]. The authors noted, however, the limitations of claims data in defining the clinical state of patients and that certain applications may be limited by use of claims data alone. We submit that EHR data is uniquely able to capture the clinical state of any patient. A major contributor to the high degree of concordance between this study and ROCKET-AF was the capture of continuous clinical variables (i.e. blood pressure, platelet, or hemoglobin values), the values of which could never be captured by claims data.

This exercise demonstrated the ability of CLINT™ to operationalize complex clinical concepts from EHR data, identify eligible patients in RWD, balance cohorts with regard to prognostic variables and follow those patients longitudinally to calculate outcome rates. The exercise also demonstrated the quality of Veradigm data in that replication of the methodology of the ROCKET-AF clinical trial in RWD yielded similar results to those found in the study.

These results show that HealthPals, using CLINT™, has the ability to conduct studies in RWD that produce insights very similar to those that ordinarily required a large RCT to uncover. Insights such as these can be used to support R&D efforts, clinical trial planning and design, post-market surveillance and the discovery of alternate uses for existing therapies. These same capabilities also enable the generation of ECA’s which can be used to reduce clinical trial size, cost, duration and risk by providing support for regulatory submissions.

This work represents the first example of an operational platform replicating RCT results using Electronic Health Records. We submit that EHR data is distinctly advantageous to using claims records alone for this endeavor. Future projects will utilize linked EHR and claims data which will enable an even more efficient and accurate mapping of clinical concepts resulting in first in class operationalization and analysis.

HealthPals is a Silicon Valley-based company that developed CLINT™, a precision insights platform capable of understanding clinical RWD from the patient to the medical system. The platform was built with scale in mind, has been run on over a Billion patient-life-years of data, and has been used by major life sciences companies and medical payers to profile user-defined patient cohorts and model their clinical outcomes and disease progression. CLINT™ Cohort Optimization and Synthetic Control Arm offerings support the next-generation of clinical trial design and can work in conjunction to enable significant time and cost savings for its partners.

If you are interested in learning more about HealthPals and our CLINT™ platform, contact us at hello@healthpalsinc.com.

Veradigm is an integrated data platform and services business unit of Allscripts that combines data-driven clinical insights with actionable tools to help healthcare stakeholders improve the quality, efficiency, and value of healthcare delivery. The company’s Life Sciences organization has unique data assets supporting life sciences researchers, including the largest source of de-identified ambulatory EHR data, as well as PINNACLE Cardiovascular and Diabetes Collaborative registries, operated in collaboration with the American College of Cardiology.

SOURCES

1. DiMasi JA, Grabowski HG, Hansen RW. Innovation in the pharmaceutical industry: New estimates of R&D costs. J Health Econ. 2016;47: 20–33.
2. Faster New Drug Approval Phases Are More Than Offset by Longer Clinical Times in U.S. Tufts Center for the Study of Drug Development Impact Report. 16 Jul 2020.
3. van Dorn A. COVID-19 and readjusting clinical trials. Lancet. 2020;396: 523–524.
4. Franklin JM, Patorno E, Desai RJ, Glynn RJ, Martin D, Quinto K, et al. Emulating Randomized Clinical Trials with Nonrandomized Real-World Evidence Studies: First Results from the RCT DUPLICATE Initiative. Circulation. 2020. doi:10.1161/CIRCULATIONAHA.120.051718
5. Patel MR, Mahaffey KW, Garg J, Pan G, Singer DE, Hacke W, et al. Rivaroxaban versus warfarin in nonvalvular atrial fibrillation. N Engl J Med. 2011;365: 883–891.