Carpentras and Quayle (2023) discuss the limitations and challenges associated with using questionnaire-derived ordinal data to parameterize Agent-Based Models. The authors draw some of their claims from the theory of levels of measurement, which consider arithmetic operations on ordinal data inadmissible. They postulate that there is no guarantee that ordinal values will be treated by respondents as if equally spaced because these can have a distorted perception of the scale even in the presence of standard designs such as the Likert. This commentary scrutinises two particularly unrealistic features of the characterisation that make their argument unfit to hold as a general position: their results depend on opting for an ungrounded uniform distribution as benchmark, and they assume a data-generating process where all respondents hold exactly the same distortions, negating variability in the human experience. Binomial models, Item Response Theory, and Mixture models are considered to understand the issue of respondents distorting the scale. A model-based approach is advanced, suggesting to treat incoherence in scales derived from partial sums of batteries of items as a source to quantify the uncertainty of a model, instead of treating them exclusively as an element of concern for the ABM.
Reflections on the house-of-mirrors: a commentary on Carpentras and Quayle (2023)
Cantone G. G.
2025-01-01
Abstract
Carpentras and Quayle (2023) discuss the limitations and challenges associated with using questionnaire-derived ordinal data to parameterize Agent-Based Models. The authors draw some of their claims from the theory of levels of measurement, which consider arithmetic operations on ordinal data inadmissible. They postulate that there is no guarantee that ordinal values will be treated by respondents as if equally spaced because these can have a distorted perception of the scale even in the presence of standard designs such as the Likert. This commentary scrutinises two particularly unrealistic features of the characterisation that make their argument unfit to hold as a general position: their results depend on opting for an ungrounded uniform distribution as benchmark, and they assume a data-generating process where all respondents hold exactly the same distortions, negating variability in the human experience. Binomial models, Item Response Theory, and Mixture models are considered to understand the issue of respondents distorting the scale. A model-based approach is advanced, suggesting to treat incoherence in scales derived from partial sums of batteries of items as a source to quantify the uncertainty of a model, instead of treating them exclusively as an element of concern for the ABM.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
