Middlebrow science



When I studied economics at Chicago in the 1970s, I was taught that the FDA was a highly flawed institution, which was structured in such a way as to avoid errors of commission at all costs and not worry so much about errors of omission, even if 100 times more consequential. That’s the system, and we’ve known it for 50 years. The current vaccine fiasco was entirely predictable. This isn’t “Monday morning quarterbacking”.

I presume that FDA officials are decent people, but they are embedded in a bureaucracy that operates with a really bad set of incentives. The solution is not to replace one FDA bureaucrat with another, it’s to take government out the the process of deciding what vaccines are acceptable, who gets them first, and at what price.

Alex Tabarrok recently posted some excellent comments on the vaccine fiasco. Here’s Sure:

The likelihood ratios for RCTs and approval mechanisms are powerful. But we are talking thousands of deaths per day. The odds that these tests will remotely alter management decisions is nil. It is malpractice to delay life saving treatment on tests exceedingly unlikely to change management decisions.

And remember the UK is not seeing horrid outcomes for doing this for a while now. A lot of theoretical failure mechanisms are now off the table.

Science is wholly about building a reliable model that accurately predicts future outcomes of current actions. While doing the actual experiment is the gold standard for knowledge acquisition, it is not the only option and in cases like this pandemic is not sufficiently better than past data to merit waiting.

As far as the regulators. I work with some of them directly. They are not overburdened to anywhere near the degree that the frontline clinicians have been hit. When I ask them to explain their cost benefit calculations, they have none. Not I cannot follow them. Not I disagree with them. They have done not an iota of math to justify their course of action.

Sorry, but I believe in evidence based medicine, not eminence based medicine. If you as a regulator cannot explain to me in technical terms the math behind your decision process, even if only back of the envelope, you are not worth putting in charge.

Approve all the vaccines, FDF, fractional dosing trials, and first dose followed by variolation trials should all be done now.

Here’s Tom Meadowcroft:

Scientific researchers search for the truth. Medical clinicians use limited data balance cost and benefits in the face of uncertainty to save the most lives.

When searching for the truth, it is important to have high standards of statistical significance, integrity, and patience, because credibility and a reputation for integrity is everything. Every academic knows that a retracted paper or an accusation of playing fast and loose with statistics can be the death knell for a career. As a result it is prudent to be very certain before publishing. Public health officials, particularly those in charge of approving vaccines, dread the possibility that a vaccine that will be given to millions of healthy people, often children, to prevent diseases where death is rare, which could harbor some flaw that causes a hundred avoidable deaths; they seek the highest standards of proof of safety and efficacy before approving such a vaccine.

But a pandemic is not a search for truth, and a COVID vaccine administered in the midst of a pandemic is very different than a measles vaccine administered to 2-year-olds. The pandemic makes these decisions for FDF or for vaccine approvals into clinical decisions, where health professionals should be balancing the certain benefit of reducing the thousands of daily deaths against the uncertain cost of the possibilities of harmful side-effects and uncertain details of efficacy (when does immunity kick in, how long does it last, how valuable is a booster) that additional months of testing and trials would reveal more clearly.

Public health researchers, academics for the most part, lack the ability (and courage) to make the sort of cost/benefit analysis with necessarily limited data that clinical physicians make every day in examination rooms. Any good clinician, faced with the citizenry of a country as their patient, would have opted for FDF, the AZ vaccine, and quite likely reduced doses by the start of the year. Because we are stuck with academics and administrators as our decision makes, unable to see beyond their usual routine of searching for the truth and protecting their reputations, thousands more will die.

PS. For those of you who think we are doing fine and that Israel is an outlier, you need to add the Navajo Nation to the list of outliers:

The COVID vaccine rollout in the Navajo Nation has been highly successful, already surpassing its original goal to have administered 100,000 shots by the end of February. It’s an impressive number, given that there are an estimated 175,000 people living in the Navajo Nation. . .

As of Feb. 11, the IHS says, a total of 113,345 COVID vaccinations had been administered in the Navajo area.