The accuracy of the cheating detection methods in large-scale tests: mathematics national examination

Thomas Mbenu Nulangi; Djemari Mardapi

doi:10.21831/pep.v22i2.14930

The accuracy of the cheating detection methods in large-scale tests: mathematics national examination

Thomas Mbenu Nulangi, Universitas Negeri Yogyakarta, Indonesia
Djemari Mardapi, Universitas Negeri Yogyakarta, Indonesia

10.21831/pep.v22i2.14930

Abstract

This study aimed to describe (1) the characteristics of items based on the Item Response Theory, (2) the cheating level in the implementation of the national examinartion based on Angoffs B-Index method, Pair 1 method, Pair 2 method, Modified Error Similarity Analysis (MESA) method, and G2 method, (3) the most accurate method to detect the cheating in the mathematics national examination at the senior secondary school level in the academic year of 2015/2016 in East Nusa Tenggara Province. The result of the item response theory analysis showed that 17 (42.5%) items of the mathematics national examination fit with the 3-PL model, with the maximum information function of 58.0128 at 1.6, and the measurement error of 0.1313. The number of pairs detected to be cheating by Angoff’s B-Index method was 63 pairs, that by the Pair 1 method was 52 pairs, that by the Pair 2 method was 141 pairs, that by MESA method was 67 pairs, and that by the G2 method was 183 pairs. The methods which could detect most pairs doing cheating were the G2 method, the Pair 2 method, the MESA method, Angoff’s B-Index method, and the Pair 1 method successively. The methods which could accurately detect cheating based on the computation of the standard error were Angoff’s B-Index method, the G2 method, the MESA method, the Pair 1 method, and the Pair 2 method successively.

Keywords

national examination; item characteristics; cheating detection methods

Full Text:

Fulltext PDF

References

Anderman, E.,& Murdock, T. (2007). Psychology of academic cheating. Cambridge: Academic Press.

Angoff, H. W. (1972). The development of statistical indices. New Jersey, Berkeley: Educational Testing Service Priceton.

Badan Standar Nasional Pendidikan. (2015). Buku saku tanya jawab pelaksanaan UN tahun 2016. Jakarta: Badan Standar Nasional Pendidikan.

Baird, J. S. (1980). Current trends in college cheating. Psychology in the Schools, 17(4), 515–522. https://doi.org/10.1002/1520-6807(198010)17:4<515::AID-PITS2310170417>3.0.CO;2-3

Bernardi, R. A., Baca, A. V., Landers, K. S., & Witek, M. B. (2008). Methods of cheating and deterrents to classroom cheating: an international study. Ethics & Behavior, 18(4), 373–391. https://doi.org/10.1080/10508420701713030

Bernardi, R. A., Banzhoff, C. A., Martino, A. M., & Savasta, K. J. (2012). Challenges to academic integrity: identifying the factors associated with the cheating chain. Accounting Education, 21(3), 247–263. https://doi.org/10.1080/09639284.2011.598719

Biro Komunikasi dan Layanan Masyarakat Kementrian Pendidikan dan Kebudayaan. (2016). Indeks integritas ujian nasional (IIUN) SMA 2016 meningkat. Jakarta: Kementrian Pendidikan dan Kebudayaan.

Bopp, M., Gleason, P., & Misicka, S. (2001). Reducing incidents of cheating in Adolescence. Master’s thesis. Saint Xavier University.

Brimble, M., & Clarke, P. S. (2005). Perceptions of the prevalence and seriousness of academic dishonesty in Australian universities. Australian Educational Researcher, 32, 19–44.

Calabrese, R. L., & Cochran, J. T. (1990). The relationship of alienation to cheating among a sample of American adolescents. Journal of Research & Development in Education, 23(2), 65–72.

Cizek, G. J. (2001). An overview of issues concerning cheating on largescale tests. In The Annual Meeting of the National Council on Measurement in Education. Seattle, WA.

Davis, S. F., Drinan, P. F., & Gallant, T. B. (2009). Cheating in school; what we know and what we can do. United Kingdom: Wiley-Blackwell.

Davis, S. F., Grover, C. A., Becker, A. H., & McGregor, L. N. (1992). Academic dishonesty: prevalence, determinants, techniques, and punishments. Teaching of Psychology, 19(1), 16–20. https://doi.org/10.1207/s15328023top1901_3

DeMars, C. (2010). Item response theory; understanding statistics measurement. New York: Oxford University Press.

Ferry, B. R., Tidman, M. T., & Wats, M. T. (1977). Indices of cheating on multiple-choice tests. Journal of Educational Statistics, 2(4), 235–256.

Hanson, A. B., Haris, J. D., & Brenan, L. R. (1987). A comparison of several statistical methods for examining allegations of copying. Iowa: American College Testing.

Hughes, J. M. C., & McCabe, D. L. (2006). Academic misconduct within higher education in Canada. Canadian Journal of Higher Education, 36(2), 1–21.

Jensen, A. L., Arnett, J. J., Feldman, S. S., & Cauffman, E. (2002). It’s wrong, but everybody does it: Academic dishonesty among high school and college students. Contemporary Educational Psychology, 27, 209–228.

Lin, C. S., & Wen, L. M. (2007). Academic dishonesty in higher education: A nationwidestudy in Taiwan. Higher Education, 54, 85–97.

Manoppo, Y., & Mardapi, D. (2014). Analisis metode cheating pada tes berskala besar. Jurnal Penelitian Dan Evaluasi Pendidikan, 18(1). Retrieved from https://journal.uny.ac.id/index.php/jpep/article/view/2128

Mardapi, D. (2000). Evaluasi penyelenggaraan ebtanas. Jurnal Kependidikan, XXX(2).

Michaels, J. W., & Miethe, T. D. (1989). Applying theories of deviance to academic cheating. Social Science Quarterly, 70(4).

Mulyati, B., & Kartowagiran, B. (2013). Analisis hasil ujian nasional mata pelajaran ekonomi SMA di kota Serang. Jurnal Evaluasi Pendidikan, 1(1). Retrieved from http://journal.student.uny.ac.id/ojs/index.php/jep/article/view/45

Naghdipour, B., & Emeagwali, O. L. (2013). Students’ justifications for academic dishonesty: call for action. Procedia -Social and Behavioral Sciences, 83, 261–265. https://doi.org/10.1016/j.sbspro.2013.06.051

Newstead, S. E., Franklyn-Stokes, A., & Armstead, P. (1996). Individual differences in student cheating. Journal of Educational Psychology, 88(2), 229–241. https://doi.org/10.1037/0022-0663.88.2.229

Presiden Republik Indonesia. Peraturan Pemerintah RI Nomor 13, Tahun 2015, tentang Perubahan Kedua atas Peraturan Pemerintah Nomor 19 Tahun 2005 tentang Standar Nasional Pendidikan (2015).

Rakovski, C. C., & Levy, E. S. (2007). Academic dishonesty: Perceptions of business students. College Student Journal, 41, 466–481.

Stogner, J. M., Miller, B. L., & Marcum, C. D. (2013). Learning to E-Cheat: a criminological test of internet facilitated academic cheating. Journal of Criminal Justice Education, 24(2), 175–199. https://doi.org/10.1080/10511253.2012.693516

Vandehey, M. A., Diekhoff, G. M., & LaBeff, E. E. (2007). College cheating: A twenty-year follow up and the addition of an honor code. Journal of College Student Development, 48, 468–480.

Whitley, B. E., Nelson, A. B., & Jones, C. J. (1999). Gender differences in cheating attitudes and classroom cheating behavior: A meta-analysis. Sex Roles, 41(9), 657–680.

Zastrow, C. (1970). Cheating among college graduate students. The Journal of Educational Research, 64(4), 157–160. Retrieved from http://www.jstor.org/stable/27536093

DOI: https://doi.org/10.21831/pep.v22i2.14930