Response to “Interpretation of TWAS and its vulnerabilities”


Recently, we released a pre-print discussing some vulnerabilities of transcriptome-wide association studies (TWAS). Dr. Sasha Gusev, the developer of one of the original TWAS methods, Fusion, responded to the pre-print in a blog post.

We appreciate Dr. Gusev’s candid discussion of the strengths and vulnerabilities of TWAS, and we are glad that we agree on the broad strokes: TWAS is valid as a weighted burden test for association and is often useful for gene-based prioritization, but does not have any guarantees for causal gene identification.

We are also very appreciative of the constructive dialogue with Dr. Gusev and the broader community since the release of the pre-print. At the end of the day, we all believe in the value of TWAS and related GWAS-eQTL methods: we started working on TWAS because we were eager to apply and extend the methods, and only in the course of doing so realized the existence and extent of the vulnerabilities we describe in the pre-print. Indeed, many aspects of the pre-print, such as our discussion of fine-mapping, are inspired by our own successes and failures as end users of TWAS methods.

We wanted to clarify a few aspects of the pre-print in light of the blog post:

  • We agree with Dr. Gusev that it is a misinterpretation to say that TWAS is a causal inference test instead of a test for association. Nonetheless, despite the care taken by TWAS methods developers to avoid this misinterpretation, it is prevalent within the community, as noted by several researchers:

Some members of the community may have been confused by statements in the original TWAS papers that could be interpreted as suggesting causality, such as the PrediXcan paper’s statement that “our method is likely to identify causal genes” and the Fusion paper’s statement that Fusion “directly implicate[s] the gene-based mechanisms underlying complex traits”. Nonetheless, we want to emphasize that, as we say in the pre-print, “both papers are careful not to claim causality”. In the pre-print, we simply reinforce the point that TWAS should not be misinterpreted as a causal gene test.

  • What matters is not just black-and-white whether TWAS is a causality test, but also what shade of gray it is: how well TWAS prioritizes causal genes in practice. The original TWAS papers note that pleiotropy could affect TWAS results in theory, and a more recent pre-print notes that “we frequently observed hotspots of multiple TWAS-associated genes in the same locus”. Our paper extends these observations by showing that cis co-regulation underlies pleiotropic TWAS associations and crucially, unlike in GWAS, that the set of associations at a locus may not even contain the causal gene.
  • Dr. Gusev mentioned that “One nice aspect of the [Wainberg et al.] analysis is that they tried to find such genes in the literature and then asked whether they show up as TWAS hits. In fact, most of the known genes DO came up as significant TWAS associations when evaluated in the expected tissue”. We actually performed the reverse analysis, starting with genes that were TWAS hits in multi-hit clusters in the LDL/liver and Crohn’s/whole blood TWAS and then looking for literature evidence of causality. So, by definition, all of our candidate causal genes are TWAS hits. This means that our results are not representative of whether or not “TWAS is actually pretty good at identifying association for causal genes”, since we did not also look at how many candidate causal genes are not hits.
  • TWAS fine-mapping does not only make the assumption that the causal effect is observed in the study, but also that the TWAS z scores and gene-based covariance matrix (however these are defined) are unbiased. However, as we discuss in the pre-print, several factors may cause bias, such as:
  1. Not using expression from a mechanistically related tissue;
  2. Even if using a mechanistically related tissue, some genes acting solely through another tissue or having a causal effect through multiple tissues;
  3. Only having a single causal cell type within a tissue;
  4. Presence of e.g. blood/immune cells even in “pure” tissue samples;
  5. Genetically- or environmentally-driven differences in cell type proportions across individuals in the expression panel;
  6. Technical biases in expression quantification;
  7. Coding variants.

Crucially, unlike variance due to the finite size of the expression panel, the effects of these factors cannot necessarily be quantified or corrected via resampling or Bayesian methods. On the other hand, we are cautiously optimistic that including other data types besides GWAS and bulk expression data may aid in fine-mapping TWAS associations by mitigating some of these biases.

  • We agree with Dr. Gusev that “a TWAS association is no more unstable than an eQTL that is significant in some tissues and not others”. However, eQTL analysis and TWAS have very different primary goals: the primary goal of eQTL analysis is to find tissue-specific expression-variant associations, while the primary goal of TWAS is to prioritize candidate causal genes at GWAS loci regardless of tissue. In other words, variability across tissues is a desirable biological effect to capture in eQTL studies, but undesirable in TWAS. To the extent that eQTL studies also have the goal of prioritizing candidate causal genes at GWAS loci, variability across tissues becomes just as much of a problem as for TWAS, since it can lead to causal genes failing to show association when evaluated in a mechanistically unrelated tissue.

As a departing thought, Ewan Birney offers some general advice that is relevant when thinking about TWAS, as well as many other genomics “big data” activities such as fine-mapping GWAS variants:

Finally, we wanted to emphasize again how grateful we are for the forthright and constructive discussion with the TWAS method developers and the community. Let’s keep it up!