Exact inference, under the confidence distribution framework, to synthesize clinical trials of rare events.
[Under Review]
Data consisting of forty eight(48) comparative studies of Rosiglitazone against another treatment treated as control, are provided on the risk of Myocardial Infarctions (MI), and on Cardiovascular death (Death), from cardiovascular causes(Tian et al. 2009). We load them here, then copy them in a txt.file and export them in an excel.file(but save the file as a csv.file, preferable for R manipulation) in which we add headers: study ID, name of the study, n.TRT, MI.TRT, Death.TRT, n.CTRL, MI.CTRL, Death.CTRL.
The first 6 and the last 6 observations are:
We use the function metabin() from the library “meta” to fit a fixed effects model FEM and a random effects model REM to the binary data on the outcome MI only.
You can also specify sm = “RD” for risk difference(difference in risks), or “RR” for risk ratio(ratio of both risks, with the risk of the control group always at the denominator; similar for odds). Those are common measures of effect sizes between groups of interventions that have reported a binary specific outcome.
We print the summary of the object “MI.OR”:
Out of the 48 studies, we confirm from the code below that, ten of them(studies number 20, 31, 33, 38, 43, 44, 45, 46, 47, and 48) reported zero MIs for both treatment groups:
datMI<-datMI[,c(“ID”, “MI.TRT”, “MI.CTRL”, “n.TRT”, “n.CTRL”)]
for (i in 1:length(datMI$ID)){
if (datMI$MI.TRT[i]==0 & datMI$MI.CTRL[i]==0){print(datMI$ID[i])}
}
[1] 20
[1] 31
[1] 33
[1] 38
[1] 43
[1] 44
[1] 45
[1] 46
[1] 47
[1] 48
Fixed effects and random effects yield the same overall estimate of 1.29 with 95% CI of [.89; 1.85], and a p-value of .17, from the combined 38 remaining studies. The test for heterogeneity is of course not significant with p-value = .89 and tau²=0. So we do not have enough evidence to claim heterogeneity of the studies involved.
There is no statistically significant difference between the two groups and in terms of the odds,
the odds for a patient to experience the MI event in the rosiglitazone group is greater than the one in the control group, but we are not certain about it, because additionally, the CI [.89; 1.85] suggests that the OR could be 1 or less than 1.
The zero cells prevent R from computing the standard error for the estimate of the OR, so by default, R generate a value zero for it. This makes inference about the OR, effect size of interest, unreliable. Let’s add a small frequency value to these zero-cells, say 0.3, and see what happens:
What about letting R do a treatment arm continuity correction (TACC) instead (Sweeting et al., 2004; Diamond et al., 2007)?
A third scenario we can also look at is when the element “incr” is not specified. In this case R employs 0.5 as default frequency value for the zero cells.
Although Sweeting et al. (2004) have convincing evidence that arbitrary values to “incl” would yield different conclusions, in our case, there is no improvement in results, with or without continuity correction.
An alternative approach has been suggested by Liu et al. (2012), which does not require such correction. The method is termed as an “exact method” and is specified in the “gmeta” — the generalized meta-analysis approach — function as methods = “exact1”. It synthesizes evidence from a 2x2 table, under a framework of combining confidence distributions (CDs). We implement it in R the following way:
Arrange columns in the data set “datMI” such that, the first and third column are number of events in Rosiglitazone and control group respectively(MI.TRT, MI.CTRL), the second and fourth column are the total number of patients in Rosiglitazone and control group respectively(n.TRT, n.CTRL). From the output below,
> colnames(datMI)
[1] “MI.TRT” “MI.CTRL” “n.TRT” “n.CTRL”
the data we shall use in the gmeta function(gmeta is a package at the same time) is stored in the object “datMI4exact”
datMI4exact <- datMI[, c(1, 3, 2, 4)] #just switching the 2nd and 3rd cols
Finally, the “exact1” meta-analysis on log-odd-ratio (LOR) based on Liu et al (2012), is derived from the object “MI.exactlOR” below
MI.exactlOR <- gmeta(datMI4exact, gmi.type=”2x2", method = “exact1”,
ci.level=.95, mc.iteration = 10000)
By the way, a 2 by 2 table looks like these:
The summary output looks like this
The structure of the object tells how to reach elements in the summary above
So we extract the “exact”estimates, which results from the combined confidence distributions of the 48 studies involved like this
We transform back to the original OR scale, from the log scale
These results(OR=1.39, 95% CI [.97; 2]) still indicate that, there is no statistically significant difference between both groups, but the one remarkable thing with this method is that, we can explicitly get the value of the variance( which is the square of the standard deviation stddev) of the parameter of interest(the effect size in OR).
Some plots
We contrast confidence distributions of some studies(studies number 2, 8, 26, 33, and 45) with the overall confidence distribution for the MI condition, on a single graph.
> plot(MI.exactlOR, studies = c(2,8,26,33,45), option=T, xlim = c(-5,5), xlab = “Exact log(OR) for MI”)
The confidence distribution of study 33 cannot be plot because it contains zero-zero events.
The confidence distribution of study 45 cannot be plot because it contains zero-zero events.
The output of the plot() function, is not only a set of graphs, but additionally we have a message telling us why the other two plots were not generated, which is not a surprise. A similar graph is generated for the “Death” outcome:
> plot(Death.exactlOR, studies = c(2,8,26,33,45), option=T, xlim = c(-5,5), xlab = “Exact log(OR) for Death”)
The confidence distribution of study 2 cannot be plot because it contains zero-zero events.
The confidence distribution of study 45 cannot be plot because it contains zero-zero events.
Exercise: Employ the “exact2” method to generate the estimate of the “RD” of both groups.
Conclusion
Fixed effects and random effect models are not appropriate for rare events. The confidence distribution use exact distributions to combine results from studies. It has been implemented in the R package “gmeta” for easy use. This overview on how to handle rare events in meta analysis, is actually extracted from Chap 8 of the book on “Biopharmaceutical Applied Statistics Symposium: Volume 2 Biostatistical Analysis of Clinical Trials” by Karl E. Peace et al. 2018.
Further reading
- Lane, P, W. (2012). Meta analysis of incidence of rare events. Statistical methods in medical research, 2012 Jan 4 Epub.
- Sutton, A. J., Cooper, N. J., et al. (2002). Meta analysis of rare and adverse events data. Expert review of Pharmacoeconomics and outcomes research, 2(4), 367–379.
- Sweeting, M. J, Sutton, A. J. et al. (2004): What to add to nothing? use and avoidance of continuity corrections in meta analysis of sparse data. Statistics in medicine, 23, 1351–1375.
- Tian, L. Cai, T., Pfeffer, M., (2009). Exact and efficient inference procedure for meta analysis and its application to the analysis of independent 2x2 tables with all available data but without artificial continuity correction. Biostatistics, 10(2). 275–281.
