As long as the majority of the population is not immune to SARS-COV2, the health care system must be protected from collapse due to overload with COVID patients. Therefore, for the past year, we have been testing measures ranging from increased hand washing to total lockdown with some success. Measures are introduced, tightened, relaxed, or abolished, only to be reintroduced, … and so it goes. Politicians justify their actions with incidence values, utilization of hospitals, model calculations and the advice of experts (see also here in my blog). Undeniably, many of these (anti)Corona measures have enormous plausibility. It is also trivial to realize that a total lockdown can severely limit the spread of a virus. However, this cannot be sustained forever. Therefore, the question of which of the measures from the black box lockdown have an effect, and for which the harm outweighs the benefit, is immensely relevant. With this knowledge, one might put together an evidence-based package of Corona measures that is less drastic than a lockdown, but just as effective. And perhaps in this way persuade some skeptics to participate. This is why the question of which evidence is available for the effectiveness of individual measures is so important. But beware.

Questioning the evidence, or pointing to lack of evidence for certain corona measures has become quite dangerous. For one runs the risk of being immediately placed in the camp of antivaxxers, contrarians (which actually used to be more of a compliment), and right-wing radicals. But lets ignore all the warnings, and draw our attention to Botswana.

We have a spate of studies that use statistical modeling to examine the effectiveness of corona measures. And not entirely surprisingly they come to very different conclusions. After all, minor changes in model parameters often lead to very different predictions. The modelers also criticize each other’s models. And who among us would dare to assess their quality, validity and predictivity? There is also a plethora of observational studies that focus on effects of Corona measures. But such observational studies provide only weak evidence and do not allow causal conclusions. What we need, therefore, are randomized controlled trials (RCTs) interventionally testing specific corona measures. RCTs are, after all, the gold standard for testing therapeutic interventions in medicine, and therefore incidentally the basis for approval of Corona vaccines. Now guess how many RCTs examining the effectiveness of social distancing interventions have been done?

I could only find 3, worldwide! One in Norway, in which participants were randomized to be allowed or prohibited from using gyms. Then the Danish mask study. There, the wearing of face masks in public was studied. This was at a time when it was not yet mandatory. The participants were randomly divided into 2 groups, one wore masks, the other did not. The endpoint in both Scandinavian studies, which recruited several thousand participants each, was the incidence of SARS-COV2 infections. In addition, a very interesting intervention, you guessed it, was conducted in Botswana! There too, schools were closed because of Corona, and a comparison was made to see if learning progress could still be achieved using ‘low tech’ interventions. To find out, students were randomized into 3 groups – no classes, daily contact with teachers via SMS, or phone call. In Botswana students do not have smartphones or laptops.

I don’t want to say anything about the results, the quality, or even the generalizability of these 3 studies to Germany or other countries. But it is a scandal that so far only Botswana has managed to conduct a randomized intervention that examines the effects of one of the most hotly debated measures, which so far already affects 1.6 billion students worldwide. And this one year after the onset of a global pandemic for which there is still no specific therapy, after more than 100 million confirmed infections and 2.3 million associated deaths, and a cacophony of fluctuating, sometimes drastic, measures of social distancing.

Instead of randomized controlled trials, our modelers and policymakers rely on observational data, including those collected during the Spanish flu of 1918/19. Wouldn’t it be time to investigate whether a total lockdown of nursing homes is more effective than the combination strategy of negative viral detection (PCR), rapid tests at the entrance, and FFP2 mask? Or whether school lockdowns are better than the combination of masks, testing, and alternate teaching? I am sure you can think of some interesting questions as well.

Now you will probably say: You sofa epidemiologist! It’s easy to ask for such studie – but controlled interventions in social distancing, that’s not possible at all! But are you so sure? Has anyone tried it? And failed, so that we would know how to do it better with a modified approach?  Unfortunately, it looks like that is not the case.

We could have learned a lot from the Botswana study, and the two Scandinavian studies, as well as a planned but never implemented Norwegian school closure study. First of all, that it is feasible in principle. The methods for such studies are basically in place, coming from clinical-epidemiological study routine. But also from randomized controlled interventions, which are now also carried out in education, economics and social sciences. Such studies can be done on the level of individuals as well as on the level of groups. The latter, for example, in so-called cluster-randomised trials, which are also frequently used in clinical questions.

Of course, this is not easy, especially under the conditions of a pandemic. To set up a useful protocol for such a study, a lot of thought has to be put into it. Using school closures as an example: What ‘dose’ and timing should the intervention have? Does one randomize school closure versus alternate instruction and reduced class size? Which grade levels to study, how large should the classes be, how often will they be ventilated by window opening? What primary outcome does one choose? SARS-COV2 infections, sure. But in which collective? In the county, in the vicinity of the school, only in parents and students? Besides, one also wants to know something about other, non-intended effects. For example, do such closures lead to poorer educational levels and grades later on? But after what time, and how should this be measured? In the case of school closures, this cannot be done at the level of individuals. You can’t randomize individual students from a class into study groups. So you have to randomize schools, or school districts. Would parents consent to something like this? To what extent is their consent even necessary? The state does not ask parents before opening or closing schools. Ethically, this is not problematic if one does not know which measure is better, if the potential benefits and risks of intervention and control are equally distributed. Ethicists call this equipoise. In the case of school closures, this is clearly the case. But even if an ethics committee agrees, what school board would go along with something like that? Would there then be an uproar from unwilling parents? Questions upon questions. This example shows that it is not at all easy to plan and carry out such interventions. But we could start with simpler and equally important questions, such as which hygiene measures are effective in restaurants. If you think that this is not possible anyway, because we are already in the middle of a lockdown, you are wrong. You would only have to reverse the time sequence and test not the introduction of a measure, but its relaxation. Moreover, we are constantly switching from more stringent to less stringent measures and back again, actually ideal conditions for causal studies.

My point is: until you get out there and try to overcome obstacles, develop new study designs and methods, the argument that you can’t crack the black box lockdown is false and dangerous. And even if one could not get answers now, which would put political decisions on a rational basis in this pandemic: The next pandemic is bound to come. Maybe we are already in the middle of it, with any of the mutants. Especially if the current vaccines are no longer effective against one of them. It’s groundhog day – Lockdown, relaxation, Lockdown etc..

But why am I so upset about the lack of randomized controlled trials? After all, we don’t even have observational studies and data collection that are easy to conduct and extremely meaningful. Do we know whether nurses, package delivery drivers, or supermarket cashiers are more often SARS-COV2 positive, and more often symptomatic? This would be quite straightforward: one would only have to report the occupational groups, together with the virus test results. And why don’t we know the true number of infected persons, i.e. those who were not tested but were nevertheless infected by the virus? This is not only important for calculating infectious mortality, but also for the question of how far we have come towards herd immunity. How about randomly selected samples, which are repeatedly tested in representative regions (city, (federal) state), as was done in the first wave in Munich? Why is there no nationwide, systematic molecular genetic monitoring of the virus genomes? Once you have spent billions on fighting the pandemic, this neglect can no longer be justified, at least not economically.

Instead, are mesmerized by the daily announcement of how many new infections were confirmed, and how many (or few) people were vaccinated. Four percent of the world’s research output in 2020 went into Corona. Pubmed already lists more than 100,000 articles on the topic. More than 4000 Corona clinical trials are registered at Clinicaltrials.gov, several hundred of which have tested the effects of chloroquine. Studies that try to find out what is good and what is bad about social distancing by means of a randomized and controlled intervention can be counted on one hand.

In Botswana, by the way, it was found that both text messaging and phone calls from teachers to parents and students significantly improved their interaction, increasing the students’ math skills over those in the control group. That’s why all families with school children now get text messages and phone calls.

 

A German version of this post has been published earlier as part of my monthly column in the Laborjournal: http://www.laborjournal-archiv.de/epaper/LJ_21_03/18/index.html

 

References:

Angrist, N; Bergman, P;  Brewster, C; Matsheng, M. Stemming Learning Loss During the Pandemic: A Rapid Randomized Trial of a Low-Tech Intervention in Botswana. CSAE Working Paper Series 2020-13, Centre for the Study of African Economies, University of Oxford. (July 9,2020).   https://ssrn.com/abstract=3663098   or http://dx.doi.org/10.2139/ssrn.3663098

Bundgaard H. et al. Effectiveness of Adding a Mask Recommendation to Other Public Health Measures to Prevent SARS-CoV-2 Infection in Danish Mask Wearers – A Randomized Controlled Trial. Ann Intern Med. 2020 Nov 18:M20-6817. https://doi.org/10.7326/M20-6817

Helsingen LM et al. A Randomised Trial of Covid-19 Transmission in Training Facilities. MedRxiv preprint https://doi.org/10.1101/2020.06.24.20138768

Fretheim A, Flatø M, Steens A, et al. COVID-19: we need randomised trials of school closures. J Epidemiol Community Health 2020;74:1078–1079. https://doi.org/10.1136/jech-2020-214262

Elgendy IY;  Nimri N; et al. A systematic bias assessment of top-cited full-length original clinical investigations related to COVID-19 Citation Data; European Journal of Internal Medicine, ISSN: 0953-620 https://DOI.org/10.1016/j.ejim.2021.01.018

 

Cristea IA, Naudet F, Ioannidis JPA (2020). Preserving equipoise and performing randomised trials for COVID-19 social distancing interventions. Epidemiology and Psychiatric Sciences 29, e184, 1–7. https://doi.org/10.1017/S2045796020000992

 

Haber NA et al. Problems with Evidence Assessment in COVID-19 Health Policy Impact Evaluation (PEACHPIE): A systematic strength of methods review. medRxiv preprint https://doi.org/10.1101/2021.01.21.21250243

 

Else, H. How a torrent of COVID science changed research publishing — in seven charts, Nature 2020;  588: 553-554 https://www.nature.com/articles/d41586-020-03564-y