COVID Fact Flips
Why do scientists seem to get it wrong almost every single time?
Months have passed since the first identified case of COVID-19 was reported. And yet scientists seem nowhere near the final solution. With each new scientific ‘breakthrough’ seemingly contradicting the previous one, one wonders — is research headed in the right direction? What is going wrong here?
The initial days of the pandemic were rife with optimistic predictions. Some suggested a possible respite in warmer months, citing a lower rate of transmission in regions with relatively higher temperature and humidity. A few even went to the extent of saying that it would simply “go away” in the summer. It took yet another barrage of reports and real-time experience to convince the public that this ‘COVID-climate correlation’ was “modest” at best. This was back in May. In July there was another study that asserted that “temperature is closely related to COVID-19 and its spread” — a ‘stark’ warning for the upcoming months, for Winter Is Coming.
This is not the first (or possibly the last) of these ‘COVIDian’ fact flips. Six months into the pandemic, it seems that the research community still hasn’t reached a consensus. Is the coronavirus air-borne or not? Is hydroxychloroquine effective in the treatment of COVID-19 or not? How much is the so-called safe distance — 1 m or 2 m? Each new study seems to contradict the previous one, or conclude with a neither-here-nor-there statement. Caught in an avalanche of caveats, a layperson with a reasonably sound scientific bent of mind is left hovering between opposing poles. But isn’t Science the Absolute Truth? Then why do scientists seem to go wrong so often?
The Data Gap
One of the golden maxims of research is ‘garbage in, garbage out’. So for scientists, availability of reliable real-time data in ample quantities is crucial. It helps them prepare robust models and validate them; analyse trends and reach conclusions with a fair degree of accuracy. COVID-19 poses a unique challenge here because of:
1. The Novelty Factor — The severe acute respiratory syndrome coronavirus 2, SARS-CoV-2, (or the novel coronavirus, 2019-nCoV, as it was previously known) exploded into the public consciousness in January 2020. The first set of COVID-19 infections due to the then-unknown virus was reported in December 2019 from the city of Wuhan, China.
The data in existence today is limited to these few months, a (relatively) short span of time in scientific terms.
2. Data Collection or Accessibility Issues — Though in-depth, reliable and consistently-documented data may be available in pockets representing certain areas or communities, it is not the case across the globe. The reasons are many.
The initial few months of COVID-19 response were riddled with problems — even detection of cases was difficult. Almost all nations were (and some still are) facing a severe crunch in testing capacity. Faulty kits were a major bug in many countries including the US, the UK, Spain, the Czech Republic, Turkey, India and the Philippines. Later, when health care systems started buckling under the COVID impact, it became increasingly difficult to maintain a proper database regarding patients and their medical history. The latest of such disturbing instances is in South Africa.
Even as testing, reporting and treatment protocols continue to evolve, many countries still do not have adequate resources or proper frameworks in place to capture and process good quality real-time data. This is further complicated by the legal, financial or administrative hurdles in accessing them. Due to the inadvertent occurrence of such blind spots, the available data may not be truly representative for any given nation or community.
Statistics on testing, transmission, infection, recovery and mortality; numbers regarding availability of health care facilities and medicines; success rates of treatment protocols and vaccination trials are all key to driving medical research. They can even be used to drive economic and policy decisions. But they are tangible indicators of the relative success (or failure) of a nation in its battle against the pandemic as well. So it comes as no surprise that the issue of COVID-related data is getting increasingly politicized, and suspicions are being raised regarding transparency of the figures reported from a host of nations including (but not limited to) Brazil, Turkmenistan, China, Myanmar, Spain, Russia, India and Iran.
3. Non-standardised Data Management — Everything related to COVID-19 is new and evolving: so is the modus operandi for data management. From collection to compilation, the protocols that are being followed by most medical facilities, administrative wings, etc. as of now tend to be area- or organization-specific. Therefore, clubbing or comparing data is taxing on researchers. They have to individually assess the data protocol followed by their sources and verify whether (for any given parameter) all data sets were measured using similar yard sticks before proceeding. Else the conclusions drawn may be faulty.
For instance, in June 2020, a study published in the Annals of Internal Medicine and the subsequent interpretations of its findings suggested that asymptomatic transmission (transmission by people who never show any symptoms, before and even after getting a positive result) played a significant role in the spread of this contagion. However, some in the scientific community soon questioned this deduction, one of their reasons being that not all cases cited in the study were truly asymptomatic. Instead, mildly-symptomatic (those who exhibit only mild symptoms) and pre-symptomatic people (those who have no symptoms at the time of getting a positive result, but develop them later on) were also wrongly lumped into this category, thus presenting a very different version of reality (regarding disease transmission).
Another example pertains to England’s puzzlingly high COVID-related death toll compared to its sister nations. Apparently, England had reported all patients’ deaths as COVID-related irrespective of the time gap between their COVID-positive result and death, while Wales, Scotland and Northern Ireland had counted only those that occurred within a 28-day window. The lack of an “agreed method of counting deaths from COVID-19” among the four UK nations and the blind comparison without recognising the difference caused this confusion.
4. Questionable Data — The time pressure on generating research outcomes under ever-changing circumstances generated a sudden and pressing worldwide need for real-time data. This input vacuum combined with the availability of social media platforms led to a virtual burgeoning of ventures in the data aggregation and analytics sector. While this was a largely helpful development, there were some issues as well, created by the presence of inexperienced players and certain shady entities looking to make quick buck out of the chaos.
The Surgisphere debacle is a prime example of the havoc wreaked by the use of data from questionable sources. It started with the publication of studies on COVID-19 treatments in the Lancet and the New England Journal of Medicine, NEJM, both highly reputed journals in the medical field. The results from the Lancet study even led to the halting of hydroxychloroquine drug trials by many health organizations (including the WHO) all over the world. But later, it came to light that the database used for both studies (from a relatively obscure company called Surgisphere) was unreliable and possibly suspect. This opened up a whole can of worms. The much-publicised and embarrassing retraction of the said papers and the scandal surrounding them did much damage to the credibility of the scientific community.
5. Ethical Conundrums — In some cases, ethical reasons prevented the collection of data in a certain manner. An example is the question of using of human challenge trials for COVID-19 vaccine studies. Conventional tests to study a disease (or its treatment or vaccine) involve scientists waiting for the volunteers to come in contact with the disease-causing microbe over the natural course of time. But in challenge trials, participants are intentionally exposed to the pathogen and way their bodies respond is studied. Thus it is the fastest way to gather data and conclude a study.
But this method has so far not been in use for conducting COVID-19 vaccine studies due to serious ethical concerns surrounding it. Unlike other diseases (that had vaccines tested this way), COVID-19 does not have a cure; nor do we fully comprehend the disease and its effects on the human body. This makes many experts question the ethicality of deliberately infecting volunteers. Considering the need for fast-tracking vaccine trials, many scientists and ethicists (WHO ethics criteria, WHO draft report, advocacy group 1Day Sooner, a publication in The Journal of Infectious Diseases) endorse these trials (albeit with adequate precautions) for vaccine studies. However, experts are yet to reach a consensus on the matter and conventional trials are in progress.
Time Considerations
Time is an essential component in a scientific study. However, when a pandemic is raging, the research outcomes are crucial to policy-makers for taking decisions and running public health awareness campaigns, and to the health care personnel for administering life-saving treatment and vaccines. Therefore scientists cannot afford the luxury of time. As a result, costly mistakes may be caused due to compromises made on:
1. Verification of the Data Source — Generally scientific studies are guided not only by data, but also intuition, honed by years of experience. But in the field of research, ends do not justify the means, and the conclusion, however logical, gets discredited if the data used is proven flawed.
This is what happened initially to the issue of transmission of COVID-19 by patients who showed no symptoms. Back in January 2020, a letter to the editor published in the NEJM titled “Transmission of 2019-nCoV Infection from an Asymptomatic Contact in Germany” sparked panic. Soon it was proven that the study was flawed — the contact in question was actually mildly-symptomatic, not ‘asymptomatic’ as the letter had claimed. This ‘error’ crept in because in the hurry to publish this finding, the data was not thoroughly vetted (the authors did not directly speak to the ‘contact’; instead they relied solely on the testimonies of the patients who had contracted COVID-19 from her). Though later studies reaffirmed that seemingly healthy people (pre-symptomatic, mildly-symptomatic and maybe even asymptomatic people) can also transmit COVID-19, news of the NEJM study being proven wrong was used by many as an argument against it.
2. Peer Review Process — Generally an academic study is considered official only once it appears in a journal, a report or the proceedings of a conference. But before publication, it is usually sent to a handful of experts in the field for review. An impartial perspective from a different angle helps catch flaws or gaps in the research (which can then be rectified by the authors), and this goes a long way in improving the quality of the work. But such rigour needs time — anywhere from a few months to a year. But given the unique circumstances surrounding COVID-19 research, it has been compressed into a few days or weeks by some journals. This has allegedly put pressure on the process, and led to the publication of flawed papers that had to be eventually retracted.
The peer review process can also be unknowingly affected by reviewer bias. In the case of the controversial study published in the Proceedings of the National Academy of Sciences (PNAS), which states that masks are “the most effective means to prevent inter-human transmission,” the reviewers were handpicked by the authors (in a process that is not typical to journals). Scientists who opposed the paper pointed to oversimplification of the observed trends and issues in methodology adopted to arrive at the conclusion — things they allege could have been resolved through a regular peer review process.
The Role of Information Outlets
A majority of the public rely mainly on news agencies and medical wings of international and governmental organizations to share reliable, easy-to-comprehend and up-to-date summaries of scientific studies, especially in the face of a pandemic. But sometimes due to inherent biases, errors of judgement or lapses, the results of research get miscommunicated by these agencies. Unfortunately in these cases, the public is either unable to verify the information or it associates such mistakes with the research work quoted and the scientific community. Instances of such improper reporting (using examples of how the results of COVID-related research on drugs hydroxychloroquine, dexamethasone and remdesivir were communicated to the public) are elucidated in an article that appeared in the Journal of the American Medical Association, JAMA (a reputed medical journal).
In this context, pre-prints (papers released online before a formal peer review) get unwittingly coerced as a major source of misinformation. The chief purpose behind releasing pre-prints is to garner suggestions and comments from a larger scientific community which would speed up the review and correction process. So they are just in-progress versions of papers and their conclusions on no account qualify as verified information. But news outlets, in a hurry to disseminate the latest information to the public, often give these caveats a toss. For example, when a pre-print on the use of ivermectin for treatment of COVID-19 appeared in the Social Science Research Network (SSRN), a few South American countries added this drug to their COVID treatment protocol. The said pre-print was later deleted by its authors. But it still continued to be cited (unlike a published paper, there is no formal framework to officially retract a pre-print) and the drug remains wildly popular in Latin America despite the fact that the few clinical trials in progress are yet to be concluded and the drug’s efficacy on COVID-19 patients is yet to be proven (as per Pan American Health Organization, PAHO).
The Research Race
The desperate need for information in the face of this pandemic has led to an overhaul in the way research was done till date. It accelerated the pace of scientific progress, and created better collaboration and cooperation in the fields of medicine, research and analytics. But it also spurred a race — to report data, publish papers, discover medicines and develop vaccines. Why is this an issue?
1. Water, Water Everywhere — No self-respecting research is a stand-alone piece. It is built on the solid foundations of published (meaning, credible) studies that went before it, duly acknowledged through citations. But this requires researchers to stay updated in their respective fields. The COVID-19 situation has given rise to an unprecedented surge in publications, especially pre-prints. A report in the Science that came out in May speaks of an estimated tally of “more than 23,000 papers” with the number “doubling every 20 days”. An online collection on coronavirus literature called CORD-19 is presently reported to contain more than 130,000 articles. Caught in a deluge of information (and misinformation), scientists themselves find it difficult to assess the merit of many of these studies or use them accordingly for their works.
2. Performance Pressure — Be it from a private or a public source, funding makes research possible. With the whole world eagerly awaiting a breakthrough, the race to discover the silver bullet for COVID-19 is on another level altogether and the prospects of gaining fame, money and power may persuade funding agencies to push for immediate results. In a mad scramble to lead a race complicated by financial or political pressures, accuracy may get compromised. Unfortunately, it is this very suspicion that casts a cloud on Russia’s recent Sputnik-V vaccine announcement.
Under the intense scrutiny brought on by the pandemic, genuine efforts are being made for the betterment of research and the way it is reported to the masses. Recognising the need for good quality input, pioneers in official, crowd-sourced and private agencies are trying to bridge the data gap and filter out misinformation with the help of innovative solutions using smart-device technology, big data and Artificial Intelligence (AI). To source reliable data and obtain up-to-date information, open-access databases have also been developed. But for the accuracy of research outcomes to improve drastically — one, research needs to be consistently nurtured with adequate resources; and two, the interests of science should be placed above political or partisan interests. To that extent, COVID-19 is a lesson for the long run; one that the world will not be forgetting soon.
These are unprecedented times and the scientific community is also struggling with the challenges. It is understandably frustrating for a layperson to stay updated on the ever-changing sequence of information. But it is important to keep in mind that everything about this situation is new and continuously evolving. And when all is said and done, every fresh scientific study on COVID-19 brings humanity one step closer to defeating this pandemic. At this stage, mistakes are only natural; questioning and correcting the status quo are a part of scientific evolution.
As Alfred [Batman Begins, 2005] would say:
“Why do we fall, sir? So that we can learn to pick ourselves up.”
