Flawed experiment sends misleading message about corona-safe gyms

An experiment recently conducted in Oslo compared a group of people allowed to work out in gyms with a comparable group who did not visit the training facilities. Strict measures against virus transmission were provided in the gyms. Since no difference in corona infection was observed between the two groups, the researchers claimed it safe to re-open training facilities in Norway.

As pointed out in an earlier comment, however, the experimental result does not support this conclusion, since no corona virus was observed in the groups (one participant who had not used the training facilities tested positive, but this is not relevant for the study’s conclusion). Without the presence of coronavirus, the researchers’ hypothesis, that the risk of SARS-CoV-2 transmission in training facilities with good hygiene and social distancing measures would be low, cannot be tested. Thus, to conclude that the experiment shows that it is safe to use the gyms is unwarranted.

Flawed reasoning

To see the flawed reasoning in the training study, consider the following analogy. A researcher wants to test whether it is corona-safe to cough, sneeze and hug in an overcrowded room. Participants are randomly selected to either spend a week coughing, sneezing and hugging in a small room, or to stay isolated. At the end of the study period, the researcher finds that none of the participants tested positive for covid-19.

Would that qualify as evidence for the statement that it is safe to cough, sneeze and hug in overcrowded rooms? Of course not. If none of the participants were infected to begin with, no virus transmission could have taken place—regardless of how the groups were instructed to behave.

There is an obvious reason why people who work out, visit restaurants or stay together in small rooms remain unaffected by covid-19: Lack of viral particles. During the trial period of the training study, a total of 105 new covid-19 cases were confirmed in Oslo. That amounts to less than 0.02 percent of the city’s population, which implies that on average one would need 7000 persons to find a single covid-19 case.

This study had less than 4000 participants, split into two groups, which means that it was dramatically underpowered. Such lack of statistical power makes the study vulnerable for so-called Type II errors, i.e., to conclude that working out in gyms is safe, when in reality it is not.

The researchers assumed that 1 per cent of participants would test positive for covid-19. This is a crucial assumption, and power calculations show that this incidence rate would be required to answer the research questions posed. However, the 1 percent assumption turned out to be 50 times higher than reality. Therefore, in the given situation the study design was far from sufficient to provide valid scientific answers. The low de facto rates of covid-19 in the trial period corresponded to the rates in the weeks just before the trial, and should not have been a surprise to anyone.

Unwarranted conclusion

Rather than attributing the lack of infection observed in the experiment to social distancing and good hygiene at the training facilities, a more reasonable conclusion is that the similarity between the two groups is due to the low rate of corona infection in Oslo when the study took place. No experiments are needed to reach that conclusion.

The coronavirus pandemic is dramatic. So far over 740 000 deaths have been registered globally. In the USA, more people died from covid-19 in the spring of 2020 than during the entire Vietnam War. In the face of this serious pandemic, scientific knowledge about infection risks at public transportation, work places and gyms is highly needed. Therefore, it is worrying that researchers actively present an unwarranted conclusion from a flawed study to the media.

———

Read Joar Vittersøs first comment on this study:

Norwegian coronavirus “experiment” was misleading. People who train in gyms can infect each other with COVID-19.

Read the response from the researchers behind the study:

Coronavirus and training in gyms: Misunderstandings about Randomized Training Trial