Feature Article — No. This Study Does Not Prove What You Think It Does: Part III
Published on
Parents PACKPublished on
Parents PACKIn the March 2022 issue of Parents PACK, we discussed some common misconceptions about science and scientists, and in the April 2022 issue, we discussed some surprising findings that were initially not believed but were ultimately proven to change our previously established understanding of science.
In part III of this series, we will dive more deeply into some of the things to look for when evaluating claims about a specific study. We’ll do this by highlighting a few papers that caused fear, generated confusion, or spread misinformation by looking at the interpretation, what it was based on, and why it was not correct.
Ricke DO. Two different antibody-dependent enhancement (ADE) risks for SARS-CoV-2 antibodies. Front Immunol. 2021 Feb 24;12:640093.
This paper suggested that COVID-19 vaccines that cause antibody production may cause a condition known as antibody-dependent enhancement (ADE). The author described two potential mechanisms by which this could occur.
ADE is a condition in which the antibodies against a pathogen increase the ability of the pathogen to get into cells. The antibody binds to the pathogen, but instead of blocking it from entering cells, it facilitates entry of the pathogen into cells. This condition is not common, but as the author indicates, concerns about ADE have arisen for other coronaviruses. Therefore, most scientists were critically monitoring data for any potential issues as COVID-19 vaccines were developed
The misconception related to this paper was that people thought it proved that ADE could occur after receipt of the COVID-19 vaccine. The paper does not provide such evidence. Here’s why:
Misclassification of article
Lyons-Weiler J. Pathogenic priming likely contributes to serious and critical illness and mortality in COVID-19 via autoimmunity. J Transl Autoimmun. 2020 Apr 9;3:100051.
This paper compared protein sequences from the SARS-CoV-2 virus to protein sequences in humans and described where the latter are found in the body. With this information, the author suggested that the quantity of overlap in some sequences could cause the immune system to attack one’s own proteins, called autoimmunity, leading to more severe disease.
As with the Ricke paper, this report presents a theory but not evidence of something happening. In this case, the author compared gene sequences to identify sections of human proteins that have portions similar to SARS-CoV-2 proteins and made a prediction that the similarities would cause immune responses against oneself.
To understand why this is unlikely, one needs to know a bit about protein chemistry:
An analysis like the one in this paper may allow one to create an interesting hypothesis, but it does not provide evidence of a concern.
Three other points about this paper are worth noting:
Misclassification of study and logical fallacies (hasty generalization and appeal to ignorance)
Alden M, Falla FO, Yang D, et al. Intracellular reverse transcription of Pfizer BioNTech COVID-19 mRNA vaccine BNT162b2 in vitro in human liver. Current Issues in Molecular Biology. 2022;44(3):1115-26.
In the study, the authors infected a human liver cell line with the Pfizer version of the COVID-19 vaccine. They used different concentrations of the vaccine and multiple timepoints to measure:
Some pointed to this paper as proof that COVID-19 vaccines alter DNA. However, this is not an appropriate conclusion from this paper for several reasons:
Logical fallacy called hasty generalization (It is important to note in this case, that the misconception was not because of the quality of the science or the messaging of the authors, but rather because others took the findings out of context.)
Chia PY, Ong SWX, Chiew CJ, et al. Virological and serological kinetics of SARS-CoV-2 Delta variant vaccine breakthrough infections: a multicentre cohort study. Clin Microbiol Infect. 2022 Apr;28(4):612.e1-612.e7.
This study of 218 people infected with COVID-19 compared their vaccination status, severity of illness and blood samples. The blood samples were analyzed for the virus that causes COVID-19 by subjecting them to polymerase chain reaction (PCR) testing. Antibody levels were also measured in separate tests.
At the heart of the misconceptions about this paper is figure 1, which shows viral shedding among vaccinated (green) and unvaccinated (red) individuals. Each dot on the table represents a study subject. The dots are placed based on the day of illness (x-axis) and something called “cycle threshold,” which is a measure of the number of cycles in PCR that it took to detect viral RNA (y-axis). The more virus in a sample, the fewer cycles it would take to detect RNA. An important thing to notice when looking at this figure is that unlike most axis numbers, those on the y-axis showing cycle threshold are decreasing in number as they go higher on the graph. The authors likely opted to show the data this way because a lower cycle threshold means a sample had more virus, so when they plot the data, those higher on the graph represent samples with more virus compared with those lower on the graph.
Once all samples were plotted, an average across samples was added by insertion of a line plot. On the figure, the two lines (comparing vaccinated and unvaccinated patient data) are almost in the same place for the first four or five days of illness, but then they separate, and the vaccinated group more quickly requires higher numbers of PCR cycles to detect viral RNA, meaning there was less virus in those samples.
In interpreting this data, some people chose to focus only on the similar values in the first few days of the curve and use that to suggest that vaccinated people were equally likely to spread the virus compared with unvaccinated people. And, while this may be the case early during infection, there are a few additional considerations when thinking about these data:
Cherry-picking data
Hopefully these examples demonstrated some of the ways that a paper should be examined. Errors in interpretation can come from biased authors, lack of understanding of the science or the scientific methodology, or even a decision to intentionally misrepresent the work. As such, consumers of information should always consider messages with a questioning attitude. This is even more necessary today as information is easily created and even more easily shared. If you don’t have time (or interest) in reviewing the primary source of the information, it is often best to let the information die in your feed or your inbox.
This three-part series was aimed at describing several important aspects of science. In part I, we focused on the big picture. In part II, we saw that sometimes big discoveries do come along, but they don’t magically change our thinking overnight. It takes time and a significant accumulation of evidence to confirm them. And, this month, we saw a few of the many ways that science can be misinterpreted and misused. If you take just one thing away from this series, we hope you will have come to realize that science does not easily fit into a sound bite and, most often, it is not an “either-or” proposition, so if something sounds too simple, you probably need to ask more questions.
These tools for evaluating information may be useful:
Categories: Parents PACK May 2022, Feature Article
Materials in this section are updated as new information and vaccines become available. The Vaccine Education Center staff regularly reviews materials for accuracy.
You should not consider the information in this site to be specific, professional medical advice for your personal health or for your family's personal health. You should not use it to replace any relationship with a physician or other qualified healthcare professional. For medical concerns, including decisions about vaccinations, medications and other treatments, you should always consult your physician or, in serious cases, seek immediate assistance from emergency personnel.