Widespread yet Unreliable: A Systematic Analysis of the Use of Presence Questionnaires

Abstract

Presence, as a psychological state, is typically assessed using questionnaires. While many researchers in this field assume that these self-report instruments are standardized, the reliability of such questionnaires remains uncertain. This knowledge gap challenges the accuracy and validity of data derived from studies assessing presence. Ensuring reliable and precise data collection and reporting is essential for the credibility of findings in presence research, because inaccuracies may cause errors in conclusions, which affects theoretical understandings, methodological approaches and practical applications. To address this issue, we conducted a systematic analysis of 397 empirical quantitative studies on presence. We investigated the use of presence scales, including applications, modifications, a variety of measures and reporting practices. We found that the majority of the presence studies modify questionnaires, do not re-validate them and improperly report their methods. Based on these findings, we propose solutions to enhance transparency and validation of the presence measurements.