Multiversal Methods in Observational Studies: The Case of COVID-19

In the present study, 13 covariates have been selected as potentially associated with 3 metrics of the spread of COVID-19 in 20 European countries. Robustness of the linear correlations between 10 of the 13 covariates as main regressors and the 3 COVID-19 metrics as dependent variables have been tested through a methodology for sensitivity analysis that falls under the name of “Multiverse”. Under this methodology, thousands of alternative estimates are generated by a single hypothesis of regression. The capacity of identification of a robust causal claim for the 10 variables has been measured through 3 indicators over a Janus Confusion Matrix, which is a confusion matrix that assumes the likelihood to observe a True claim as the ratio between the absolute difference of estimates with a different sign and the total of estimates. This methodology provides the opportunity to evaluate the outcomes of a shift from the common level of significance to the alternative. According to the results of the study, in the dataset the benefits of the shifts come at a very high cost in terms of false negatives.