Psychometric properties of the general conspiracy belief scale using item response theory

Sumin Sumin; Khairawati Khairawati; Mohd. Shahrul Kamaruddin

doi:10.21831/pep.v29i1.84659

Psychometric properties of the general conspiracy belief scale using item response theory

Sumin Sumin, Institut Agama Islam Negeri Pontianak, Indonesia
Khairawati Khairawati, Institut Agama Islam Negeri Pontianak, Indonesia
Mohd. Shahrul Kamaruddin, Universiti Malaysia Sarawak, Malaysia

10.21831/pep.v29i1.84659

Abstract

Evaluation of the psychometric properties of conspiracy theory belief instruments has been dominated by classical approaches with limitations, especially in dependence on sample size and inaccuracies in item-level analysis. This study aims to fill this gap by applying a polytomous Item Response Theory (IRT) approach to reanalyze the General Conspiracy Belief Scale (GCBS). This study aims to re-examine the psychometric properties of the GCBS with an IRT approach to produce measurements that are more precise and independent of sample characteristics. The research design used was a quantitative replication utilizing secondary data from 2,495 students at the college level. The instrument used consisted of 15 items on a five-category Likert scale. The analysis was conducted using three polynomial IRT models, namely the Graded Response Model (GRM), Partial Credit Model (PCM), and Generalized Partial Credit Model (GPCM), with the help of R software. The results showed that the GRM model was the model that best fit the data, with most items showing high distinctiveness and providing maximum information on respondents with low to moderate levels of conspiratorial belief. Empirical marginal reliability coefficients were high, indicating that the instrument's internal consistency was perfect. This study contributes to the field by offering a more robust and nuanced psychometric evaluation of the GCBS through IRT, providing researchers with a validated framework for assessing conspiracy beliefs with higher accuracy and scale precision. However, the limitation of this study lies in the use of secondary data sourced from one particular population group, so the generalizability of the findings still needs to be further examined in a more diverse context.

Keywords

GCB; IRT; conspiracy; psychometrics; scales

References

Baker, F. B. (2001). The basics of item response theory. ERIC.

Brotherton, R., French, C. C., & Pickering, A. D. (2013a). Measuring Belief in Conspiracy Theories: The Generic Conspiracist Beliefs Scale. Frontiers in Psychology, 4. https://doi.org/10.3389/fpsyg.2013.00279

Brotherton, R., French, C. C., & Pickering, A. D. (2013b). Measuring Belief in Conspiracy Theories: The Generic Conspiracist Beliefs Scale. Frontiers in Psychology, 4. https://doi.org/10.3389/fpsyg.2013.00279

Cangur, S., & Ercan, I. (2015). Comparison of Model Fit Indices Used in Structural Equation Modeling Under Multivariate Normality. Journal of Modern Applied Statistical Methods, 14(1), 152–167. https://doi.org/10.22237/jmasm/1430453580

Chen, W.-H., & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22(3), 265–289.

Creswell, J. W., & Creswell, J. D. (2017). Research design: Qualitative, quantitative, and mixed methods approaches. Sage publications.

Dai, S., Vo, T. T., Kehinde, O. J., He, H., Xue, Y., Demir, C., & Wang, X. (2021). Performance of Polytomous IRT Models With Rating Scale Data: An Investigation Over Sample Size, Instrument Length, and Missing Data. Frontiers in Education, 6. https://doi.org/10.3389/feduc.2021.721963

Desjardins, C. D., & Bulut, O. (2018). Handbook of educational measurement and psychometrics using R. CRC Press.

Drinkwater, K. G., Dagnall, N., Denovan, A., & Neave, N. (2020a). Psychometric assessment of the Generic Conspiracist Beliefs Scale. PLOS ONE, 15(3), e0230365. https://doi.org/10.1371/journal.pone.0230365

Drinkwater, K. G., Dagnall, N., Denovan, A., & Neave, N. (2020b). Psychometric assessment of the Generic Conspiracist Beliefs Scale. PLOS ONE, 15(3), e0230365. https://doi.org/10.1371/journal.pone.0230365

Embretson, S. E., & Reise, S. P. (2000a). Item response theory for psychologists. Lawrence Erlbaum Associates, Publishers.

Embretson, S. E., & Reise, S. P. (2000b). Item response theory for psychologists. Lawrence Erlbaum Associates, Publishers.

Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Nijhof’f Publishing.

Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory (Vol. 2). Sage.

Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/https://doi.org/10.1080/10705519909540118

Kaplan, C. (2024). Relationships Between Pseudoscience, Conspiracies, Paranormal Ideas, and Personality Factors.

Kaplan, R. M., & Saccuzzo, D. P. (2017). Psychological testing: Principles, applications, and issues. Cengage Learning.

Kim, J. (2018). Extensions and applications of item explanatory models to polytomous data in item response theory. University of California, Berkeley.

Lantian, A., Muller, D., Nurra, C., & Douglas, K. M. (2016). Measuring belief in conspiracy theories: Validation of a French and English single-item scale. International Review of Social Psychology, 29(1), 1–14.

Masters, G. N. (2016). Partial credit model. In Handbook of Item Response Theory, Volume One (pp. 137–154). Chapman and Hall/CRC.

Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16(2), 159–176. https://doi.org/10.1177/014662169201600206

Muraki, E. (1993). Information functions of the generalized partial credit model. ETS Research Report Series, 1993(1), i–12.

Ostini, R., & Nering, M. L. (2006). Polytomous item response theory models (Issue 144). Sage.

Pennycook, G., Cannon, T. D., & Rand, D. G. (2018). Prior exposure increases perceived accuracy of fake news. Journal of Experimental Psychology: General, 147(12), 1865.

Retnawati, H. (2014). Teori respons butir dan penerapannya: Untuk peneliti, praktisi pengukuran dan pengujian, mahasiswa pascasarjana. Nuha Medika.

Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement.

Samejima, F. (1997). Graded response model. In Handbook of modern item response theory (pp. 85–100). Springer.

Shapiro, G. K., Holding, A., Perez, S., Amsel, R., & Rosberger, Z. (2016). Validation of the vaccine conspiracy beliefs scale. Papillomavirus Research, 2, 167–172. https://doi.org/10.1016/j.pvr.2016.09.001

Sözer, E., & Kahraman, N. (2021). Investigation of Psychometric Properties of Likert Items with Same Categories Using Polytomous Item Response Theory Models. Journal of Measurement and Evaluation in Education and Psychology, 12(2), 129–146.

Team, R. C. (2013). R: A language and environment for statistical computing.

Van Prooijen, J.-W., & Douglas, K. M. (2017). Conspiracy theories as part of history: The role of societal crisis situations. Memory Studies, 10(3), 323–333.

Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8(2), 125–145.

DOI: https://doi.org/10.21831/pep.v29i1.84659