Login
There has been little attention paid to Cochrane’s review of COVID‐19 vaccines, in stark contrast to the worldwide media attention paid to its review of face masks.
Since Cochrane Reviews have been lauded as “rigorous” and “trustworthy,” it is prudent to apply equal scrutiny to the latest update of the “Efficacy and safety of COVID‐19 vaccines.”
Findings of the Cochrane Review?
Published in December 2022, the review analyses 41 randomized controlled trials of 12 different vaccines, involving more than 400,000 individuals without a previous SARS-CoV-2 infection.
The review indicates that most trials were no longer than two months in duration and conducted before the emergence of variants of concern such as Omicron (up to November 2021).
Compared to the placebo (which was mostly saline), the authors conclude with “high certainty” that most of the available vaccines could reduce symptomatic COVID-19, and in some trials, they could reduce severe or critical disease.
It also concludes that there was “probably little or no difference between most vaccines and placebo for serious adverse events.”
Due to trial exclusions, the authors did acknowledge that the results could not be generalized to pregnant women, people who’d already been infected with SARS-CoV-2 or immunocompromised people.
A flawed review?
A critique of the Cochrane Review has recently been published by researchers Peter Doshi, Joseph Fraiman, Juan Erviti, Mark Jones, Sander Greenland, Patrick Whelan and Robert M. Kaplan.
The critique raises some serious concerns about the conclusions reached by the Cochrane authors.
Doshi et al. are the same researchers who reanalyzed the pivotal mRNA trials and found one additional serious adverse event (SAE) occurred for every 800 people vaccinated with an mRNA vaccine.
This contradicts Cochrane’s review which concludes that there is “little or no difference in SAEs compared to placebo.”
Doshi et al. point to significant flaws in the Cochrane Review that could explain the contradictory findings.
For example, in Moderna’s trial reports, SAE tables included efficacy data on individuals suffering from serious COVID-19, which were almost entirely in the placebo arm.
Cochrane did not remove the efficacy data before presenting its analysis of Moderna vaccine SAEs, thus tabulating results that dilute the true rate of harm.
The Cochrane Review also shows the absolute difference in SAEs between most COVID-19 vaccines and placebo “was fewer than 5/1000 participants.”
That’s equivalent to 1 in every 200 people — which is not rare — and incompatible with Cochrane’s stated conclusion that there is “little or no difference.”
According to Doshi et al., another flaw in Cochrane’s review is the “composite endpoint” used to categorize “severe or critical COVID-19” cases.
Doshi’s team points out that the endpoint includes many participants who were not “severe or critical.”
For example, in the Moderna trial, 21 out of 30 cases were not hospitalized (see table below).
There was also a problem with how “COVID-19 cases” were counted in the trials, and this problem carried over into the Cochrane Review when the evidence was being synthesized (garbage in = garbage out).
The trial sponsors waited one week (Pfizer) or two weeks (Moderna) after dose 2 before counting COVID-19 cases.
Put another way, COVID-19 cases were not counted until four or six weeks after dose 1 was administered (see table below).
Excluding all cases in the 4-6 weeks after dose 1 is “particularly concerning” to Doshi et al. because there was very little follow-up time before the manufacturers decided to unblind the trials and offer the vaccine to the placebo group (median follow-up was two months after dose 2).
Another issue highlighted by Doshi et al. is the fact that the mRNA vaccines were very “reactogenic” and likely to “unblind” the participants and bias the trial outcomes.
People in the vaccine group, for example, took more medications to lower their fever following treatment than those who took the placebo, making it easier to guess which treatment they’d received.
Nonetheless, the Cochrane reviewers judged the risk of bias related to blinding in the trial as “low.”
The Cochrane Review also states that it’s “unclear if and how vaccine protection wanes over time” and that the evidence in their review is “up to date to November 2021.”
However, Doshi et al. say that this is incorrect. The Cochrane Review includes two papers (1 and 2) where Pfizer, in fact, did show that efficacy against COVID-19 declines over time — vaccine efficacy had waned to 84% more than four months after dose 2.
Finally, the trials analyzed in the Cochrane Review mainly included healthy people who’d never been exposed to the virus, which means that the efficacy outcomes are no longer applicable to most people who have recovered from single or multiple infections.
In the same way, the trials are not applicable to pregnant women, or immunocompromised people. This was not emphasized by the Cochrane reviewers despite being a major limitation.
Cochrane’s response?
A media inquiry with a list of questions was sent to the lead author of the Cochrane Review, Isabelle Boutron, a professor of Epidemiology at Université Paris Cité and director of Cochrane France.
Boutron confirmed her team had received the comments by Doshi et al. but did not respond to further questions, except to write in an email:
“We are working on the responses, which will take some time to check and respon[d] adequately to each point. We will, of course, inform you when our responses have been finalised.”
There is no timeframe in which Cochrane authors are required to respond to criticisms of their reviews — some have taken several years. A critique of Cochrane’s review of HPV vaccines submitted by Doshi et al. in 2018 still has not received a response.
If Cochrane accepts the criticisms laid out by Doshi et al. and revises its review accordingly, it’s sure to arrive at a very different conclusion than what currently stands.
Originally published by Brownstone Institute.
