There are many operational definitions and many indicators of interdisciplinary research. As a consequence, claims about its scientific impact may suffer from high model uncertainty and low credibility, since they can be based on specific results highly dependent on implicit choices of the modelling process. This study addresses the issue with Multiverse Analysis, a protocol for multi-model inference that specifies the modelling factors that can generate variability in the results. Combinations of these factors are fit and analysed simultaneously, so that inference cannot be selective in reporting results. 1,344 regression models are fit to a sample of 5,828 articles from journals of Business Studies. The Multiverse Analysis does not find support for the claim that interdisciplinary articles of Business are more cited. Even though a minority of the models reach statistical significance, the median estimates for the effect size are close to zero. Through an analysis of variance it is established that the choice of the indicator is the most influential, reproducing 45% of the total variance. This result confirms that claims on the scientific impact of IDR are highly dependent on the operational definition. The choice of the sources of metadata on articles, a factor previously overlooked in literature, reproduce 10% of the total variance. Findings suggest caution in accepting generic claims about the scientific impact of interdisciplinary research in social sciences.
Model uncertainty in the evaluation of the impact of interdisciplinary research in Business Studies: A Multiverse Analysis
Cantone G. G.;
2025-01-01
Abstract
There are many operational definitions and many indicators of interdisciplinary research. As a consequence, claims about its scientific impact may suffer from high model uncertainty and low credibility, since they can be based on specific results highly dependent on implicit choices of the modelling process. This study addresses the issue with Multiverse Analysis, a protocol for multi-model inference that specifies the modelling factors that can generate variability in the results. Combinations of these factors are fit and analysed simultaneously, so that inference cannot be selective in reporting results. 1,344 regression models are fit to a sample of 5,828 articles from journals of Business Studies. The Multiverse Analysis does not find support for the claim that interdisciplinary articles of Business are more cited. Even though a minority of the models reach statistical significance, the median estimates for the effect size are close to zero. Through an analysis of variance it is established that the choice of the indicator is the most influential, reproducing 45% of the total variance. This result confirms that claims on the scientific impact of IDR are highly dependent on the operational definition. The choice of the sources of metadata on articles, a factor previously overlooked in literature, reproduce 10% of the total variance. Findings suggest caution in accepting generic claims about the scientific impact of interdisciplinary research in social sciences.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
