When the U.S. Food and Drug Administration (FDA) convened a panel of outside experts Oct. 22 to advise it on potential coronavirus vaccines, members of the advisory committee expressed several serious concerns about the testing and approval process.

One medical doctor warned of the potential for “a total disaster.” Another complained of “hugely missed opportunities.”

What the FDA heard — or chose to hear — is another matter.

At the end of the daylong meeting, an FDA official summarized the discussion in a way that seemed to minimize, mischaracterize, or gloss over some of the outside experts’ warnings.

“I heard that the general … principles and the standards that we are applying are right on the money, and that there is really buy-in for that,” the FDA official, Marion F. Gruber, said in part.

The acting chairman of the advisory committee — whose past relationships with companies developing coronavirus vaccines the Project On Government Oversight (POGO) documented in a recent report — endorsed the FDA official’s description.

“I fully agree with your summary,” Dr. Arnold Monto said.

The conclusion of the meeting left this reporter wondering: Did we just listen to the same discussion?

Other observers have reacted similarly.

“If FDA officials like Gruber walk away from adcomms [advisory committees] just deciding in their own minds what they heard and not what was actually said, what’s the point? And what damage did her closing remarks just do?” Donna Young, who covers pharmaceuticals and biotech for S&P Global News, tweeted.

“What planet was Gruber on today?” Young added.

“Many of us who watched the Oct. 22 [meeting] were scratching our heads at FDA’s end-of-day summary that agency guidance was ‘right on the money,’” Jessica Adams, who works in pharmaceutical regulatory affairs, later tweeted.

The roster the FDA posted of experts participating in the meeting named a long list of doctors. It didn’t say anything about spin doctors.

Both the FDA official and the acting chairman of the advisory committee offered a bit more nuance in their closing remarks on Oct. 22. But, as they framed what the FDA would or should take away from the meeting, neither seemed to do justice to feedback and advice committee members provided.

The public meeting, live-streamed on YouTube, came as the FDA was trying to restore public trust after appearing to bow to political pressure by issuing so-called Emergency Use Authorizations (EUA) for unproven coronavirus treatments.

At the meeting, some speakers presented research showing that members of the public are deeply skeptical about and disinclined to use potential vaccines. Other speakers warned that if the FDA approves a coronavirus vaccine that turns out to be dangerous, it could shatter faith in coronavirus vaccines generally and undermine efforts to fight the pandemic.

In an interview with POGO, advisory committee member Dr. H. Cody Meissner said he thought Gruber’s intention was to express appreciation to people who participated in the meeting.

“I don’t have any problem with her concluding remarks,” said Meissner, who is chief of the division of pediatric infectious disease at Tufts Children’s Hospital in Boston.

“I think it was a very robust, productive discussion that hopefully the FDA found useful,” he said. “I think they got a variety of opinions.”

During the meeting, members of the committee explained several reasons to worry about the clinical trials now underway to test potential coronavirus vaccines and how the FDA review process could unfold.

Some of the concerns related to the potential for the FDA to issue emergency use authorizations for experimental vaccines. Such orders would allow distribution of vaccines before they qualify for formal approval.

The FDA has said it would consider granting an EUA for a vaccine if the human test subjects in a clinical trial of the vaccine have been followed for a median of at least two months after completing the dosing regimen. The median is the midpoint in a set of data, meaning that, by the FDA’s standard, when an authorization is issued, almost half of the test subjects could have been followed for less than two months.

Two months is more than the White House wanted when it was hoping to get a vaccine authorized before election day. But members of the advisory committee expressed concern that two months of follow-up might not be enough to determine if the vaccine causes harmful side-effects or if whatever protection the vaccine provides wears off quickly.

Members also expressed concern that an EUA could undermine the ongoing clinical trial meant to gather critical data about the experimental vaccine over a longer period. In the vaccine trials, some test subjects are given the experimental vaccine and others are given a placebo — a harmless fake. To make the experiment as objective as possible, neither the test subjects nor the medical personnel interacting with them are supposed to be told who is in which group. In other words, they are supposed to be “blinded.”

One vaccine developer, Pfizer, has stated that if it received an EUA, it would have an ethical obligation to tell test subjects about it and allow those receiving the placebo to switch to the vaccine. That could drain the clinical trial of placebo subjects and unblind all who remain involved in the study.

Some members said they opposed issuing EUAs, at least under certain scenarios.

Committee members also challenged the way vaccine trials are measuring success — where they are setting goal lines, which are known as “endpoints.” Most of the trials are designed to declare a coronavirus vaccine effective if it reduces the number of mild cases of COVID-19, whether or not it reduces severe or fatal cases.

Some committee members said it is more important to determine if a vaccine spares people from the worst effects of the virus. One criticized key endpoints in company-sponsored trials as flawed because they reflect patients’ subjective impressions of symptoms rather than items that are objectively measurable. Another member called for greater focus on whether vaccines prevent people from getting infected in the first place, because preventing infection is essential to preventing the spread of infection. Even people who show no symptoms can infect others.

Committee members said they were wary of using coronavirus vaccines on children. They essentially argued that risks could outweigh benefits because COVID-19 is generally much less severe in children than in adults, and they called for strong evidence that a vaccine is safe before administering it to children. At least one expressed concern about inferring or extrapolating a vaccine’s effects on children from data on adults, as the FDA has contemplated.

Members urged the FDA to make sure minority populations such as Black Americans are well represented in clinical trials and warned of tragic consequences if they are not.

In this YouTube video, Dr. Luigi Notarangelo, a committee member and a chief researcher at the National Institutes of Health, said: “I mean, we’re relying basically upon reporting from the subjects without any objective validation of what they’re reporting.”

Dr. Luigi Notarangelo, a committee member who is a chief researcher at the National Institutes of Health, minced no words as he articulated several of the critiques.

Notarangelo said measures of vaccine effectiveness included in an FDA document the committee was asked to review have two problems.

“First of all, they really are biased — skewed towards mild disease,” he said. “Mild disease may not mean very much.”

“The other problem with those efficacy measures is that most of them are really subjective,” he said. “And I think that’s a major concern. I mean, we’re relying basically upon reporting from the subjects without any objective validation of what they’re reporting.”

“I think children at this point should not be considered for use of this vaccine until there is sufficient evidence” that it’s safe for them, “and what we’ve been presented today does not provide that,” Notarangelo said.

Without sufficient racial minority representation in the clinical trials, “the net effect will be that perhaps the white population might be protected, and we will only see cases of severe COVID among the Black [population], which would be a total disaster,” he added.

When an FDA official said the agency did not feel it had the power to dictate a primary endpoint that focused on severe disease, Dr. James Hildreth, president of Meharry Medical College, pushed back.

In this YouTube video, Dr. James Hildreth, president of Meharry Medical College, said: “Since severe disease and death occur primarily among minorities with this virus, if we put a vaccine out there that does not address that issue it’s just going to perpetuate the perception that that population or that segment of our population does not matter much in … this town.”

“Since severe disease and death occur primarily among minorities with this virus, if we put a vaccine out there that does not address that issue it’s just going to perpetuate the perception that that population or that segment of our population does not matter much in … this town,” Hildreth said.

“You said you cannot mandate what the drug companies might set as their primary endpoint. If I’m not mistaken, the taxpayers of the United States of America are paying the tab for this, so maybe you might have more authority to mandate than you might think,” Hildreth added.

The FDA recently appointed Hildreth to the advisory committee on a temporary basis, saying his expertise in virology, immunology, and health disparities affecting minority communities would make him indispensable.

Dr. Hayley Altman-Gans, a professor of pediatrics at Stanford University Medical Center, painted a picture of the clinical trials as deeply flawed. She spoke of “hugely missed opportunities” and said, “We really need to be thinking about this differently.”

“I really feel like they haven’t gone far enough in terms of the safety outline” and “efficacy as well,” she said. What’s more, “nobody’s collecting as far as I can tell” certain important data.

“One of the things I have not heard much about during this conversation is infection,” said committee member Kathryn Holmes, a professor emerita at the University of Colorado School of Medicine. “I’d like to see how we could actually be measuring infection rather than just mild disease. … We should be looking to see what can prevent infection because that is the rubric which would prevent spread through the community most effectively and that is what would protect our elderly as well.”

Dr. Michael Kurilla, an expert on infectious diseases and a director of clinical innovation at the National Institutes of Health, said a standard the FDA has set for coronavirus vaccine authorization — 50% efficacy — doesn’t make sense in all situations. For health care workers and staff in long-term care facilities, a vaccine “that would take them merely from a mild infection to potentially an asymptomatic infection where they still might be infectious doesn’t seem like it’s something worthy” of an EUA, he said.

“The minimum has to be much, much higher in terms of having a general overall public health impact,” he said.

Under the proposed FDA standard, “we could have a vaccine that seems to do well, meets the 50% test, and it’s effective at avoiding mild cases but actually does very little to address what we really care about, which is serious disease and death,” said Sheldon Toubman, a legal assistance attorney in New Haven, Connecticut, and the advisory committee’s consumer representative.

Toubman also challenged the FDA’s stated openness to granting an EUA based on two months of observations.

“There could be adverse effects we don’t know about, and so isn’t two months a little short?” he asked.

Toubman recommended that the FDA not grant EUA for coronavirus vaccines.

FDA officials rebutted some of the committee members’ concerns — saying, for example, that focusing on more severe cases would require enrolling an unrealistically large number of people in the clinical trials.

On some topics, FDA officials shared committee members’ concerns. On the issue of EUAs causing people to drop out of clinical trials, one official said, “I don’t have any specific remedies to offer at this time.”

POGO emailed the committee’s acting chairman requesting comment. We did not receive a response.

POGO also emailed Gruber asking how her summary of the meeting squared with committee members’ remarks. We received an email from an FDA spokesperson that largely repeated Gruber’s summary and added, “As Dr. Gruber stated, the FDA appreciates the perspectives heard from the committee members and are carefully and thoughtfully considering them as we move forward.”

During the part of the meeting set aside for members of the public to comment, a series of speakers with backgrounds in medicine or health policy made some of the same points as committee members, sometimes in sharper and more forceful terms.

In her summary at the end of the meeting, Gruber, director of the FDA’s Office of Vaccines Research and Review, did not attempt to summarize the public comments. She made no mention of them.

She acknowledged committee members’ points about minority populations and, more obliquely, children.

“I hear there were some concerns and suggestions made for some of the details, the importance of making sure that minorities are included in clinical studies. We had some discussions on endpoints,” Gruber said. She indicated it might be too late for the FDA to act on the advice for vaccines already in advanced or “Phase III” clinical trials.

“We can take this forward as we have, you know, new vaccines entering clinical studies. It may be a little bit difficult for those who are already in Phase III,” Gruber said.

“What I want to know from you, the two months, the median two months follow-up that we said on the EUA, I’ve heard people expressing some concerns with that being maybe not short enough, but you know if it then cannot be longer by no means should it be shorter than two months of median follow up,” she said. “That’s what I heard.”

That seemed to distort committee members’ comments. (In her statement to POGO recapping Gruber’s remarks, the FDA spokesperson did not repeat Gruber’s observation that some people thought a median of two months of follow-up might not be short enough.)

Addressing the threat that an EUA would disrupt a clinical trial by unblinding it, Gruber seemed to assume committee members accepted that as inevitable. She falsely implied that committee members all bought into the idea of issuing EUAs.

“And in terms of the blind … even though you have an EUA, you said, even though we all would like for this to continue, but we have to realize that at some point we can’t really maintain the blind. But do I hear you saying and do I hear the committee saying that the blind should be maintained for as long as feasible and it should not necessarily be an automatic crossover of the placebo recipients … getting the vaccine?”

“I think that is very clearly what you heard. I don’t think there’s been any doubt about that one,” Monto, the committee’s acting chairman, said.

“I think there may be some questions about the two months and also some of the outcomes that are being used,” Monto said, apparently referring to measures companies are using to judge whether experimental vaccines are effective.

Monto proceeded to inject his own views into a summary of the committee members’ comments.

“And as somebody who’s worked with flu vaccines for a long time, what you are using as the outcome is standard for most respiratory vaccines,” he said. “So I fully agree with your summary,” he concluded.

Then, before other committee members could weigh in, he adjourned the meeting.

Published with permission from Project On Government Oversight. The Project On Government Oversight (POGO) is a nonpartisan independent watchdog that investigates and exposes waste, corruption, abuse of power, and when the government fails to serve the public or silences those who report wrongdoing.