Home >Opinion >Views >The many trials and tribulations of covid-19 vaccines

The world is betting on a vaccine for covid-19. Even naysayers and anti-vaccinistas have been waiting for a miracle. And it seems they may have got it, with a flurry of recent announcements of vaccine efficacies of 70-90%. That said, one of the trials inadvertently administered a partial dosage which, interestingly, boosted efficacy. So, what do all these claims mean and how can we evaluate them to make informed decisions in the weeks and months to come?

The point of a vaccine is clear: it should result in much less harm than exposure to the virus. And this is what a vaccine trial attempts to measure—the counterfactual.

Since we can never know what would have happened sans the vaccine, we have to estimate the difference between those who have been treated or vaccinated and a control group that has not been vaccinated but is as close to the vaccinated group as possible. In the absence of a crystal ball, the control group acts as our counterfactual. But how do we know that this difference reflects the true difference and is not biased or distorted?

The most critical bias is invariably a selection one, as it can undermine the entire validity of a trial. This occurs when certain participants are systematically selected for or rejected from the treated or control groups, resulting in differences in characteristics across the two groups and rendering them incomparable. To avoid this bias, participants are randomly allocated into treated and control groups, thus leading to the much-vaunted randomized control trial. Equally vital is concealing the allocation sequence, so that researchers are not aware of the next treatment allocation and cannot influence selection.

Another threat to validity arises from performance bias, in which the treated and control groups are treated differently after enrolment. Double blinding is an expected feature of most high-quality experimental trials, as it helps mitigate this bias.

Attrition is another serious source of bias in a trial. If participants drop out of a study and there is a systematic reason for those drop-outs, that may bias the result. Experimental trials may also suffer from a reporting bias, in which researchers selectively report those outcomes that show the results they desire.

So, how do we assess the current trials against these criteria? The world’s best vaccine trials are typically double-blinded, allocation-concealed randomized control trials with a low risk of selection and performance bias. There are, however, three possible concerns. As it turns out, the accidental partial dosage was only administered to participants under the age of 55, creating selection bias concerns. It’s not surprising therefore that the manufacturers will be validating the results across a more representative population to ascertain whether the reported efficacy holds up.

Another relates to attrition. If participants who suffered severe side effects or fell sick dropped out disproportionately, the results may be overstated.

The third concern involves reporting. The endpoint of vaccine efficacy reported so far in some trials refers to a reduction in confirmed symptomatic cases only. This does not mean the vaccines have necessarily reduced asymptomatic infection from covid-19, as that was not an endpoint that was measured. The risk here is that vaccine recipients may not fall sick themselves, but can still infect others who may be more vulnerable or have not taken the vaccine. To be sure, the vaccines may well protect against asymptomatic infection, but until further data is available, this has huge implications for vaccine decisions, roll-outs and transmission models.

Aside from vaccine efficacy, the world is waiting expectantly for information on critical outcomes related to side effects, long-term safety, protection against various strains, and the perseverance of immunogenicity.

What can we say about the external validity or generalizability of the trials? First, recipients of the vaccine differ not only by age and immunity, but also in terms of their exposure, risk proclivities, genetics, geographies and medical care. We need to know the heterogeneous impacts of the vaccine on different groups to help us understand who the vaccine works for. We can then decide whether to vaccinate widely and build herd immunity (as for the measles vaccine) or to protect the most vulnerable (as in a seasonal flu shot).

Second, the trial was conducted on recipients who did not have antibodies to covid-19. The impact of vaccinating those who already have immunity is unknown, and given the scarcity of the vaccine in the short term, it is possible that mass vaccination will require mass testing. This will have implications on sequencing, prioritization and the speed of the vaccine roll-out.

The meteoric race to create and test multiple vaccines in less than a year has been remarkable and unprecedented in the history of the world. Furthermore, the partial dosage results may turn out to be yet another serendipitous discovery, like that of the smallpox and chicken cholera vaccines hazarded upon by Edward Jenner and Louis Pasteur, the fathers of vaccinology.

But the sooner vaccine researchers and independent regulators can share detailed transparent data on the results, the quicker the public health fraternity can come up with a strategy to use these vaccines to best protect humanity from covid-19. In the words of Pasteur, “Le hasard ne favorise que les esprits préparés." Fortune only favours the prepared mind.

Shonar Lala is an evaluator of development effectiveness who has worked at the World Bank’s Independent Evaluation Group and at 3ie.

Subscribe to Mint Newsletters
* Enter a valid email
* Thank you for subscribing to our newsletter.

Never miss a story! Stay connected and informed with Mint. Download our App Now!!

Edit Profile
My ReadsRedeem a Gift CardLogout