Content validity analysis of literacy assessment instruments

Dameis Surya Anggara, Universitas Pamulang, Indonesia
Candra Abdillah, Universitas Pamulang, Indonesia

 10.21831/cp.v42i2.55900

Abstract


This research is part of the development of teacher assessment literacy instruments. The research objective at this stage was to determine the content validity of the instrument. The instruments developed were 25 multiple-choice items constructed from Assessment Objectives, Measurement Theory, Assessment Process, and Assessment Fairness. Then the instrument is assessed qualitatively through a Focus Group Discussion to collect various suggestions and input from various points of view. After that the instrument was assessed quantitatively by 3 experts with 6 aspects of assessment, namely 1) the relevance of the item to the assessment literacy dimension, 2) relevance of items with indicators, 3) clarity of the main items, 4) logical answer options, 5) standardization of language, and 6) the functioning of descriptions/ cases/ pictures/ tables on items. The collected data were analyzed according to the type of data. Qualitative data analyzed using a qualitative approach, while quantitative data were analyzed using Conten Validity Index (CVI). The results showed that of the 25 items, each had a CVI value of more than 0.60 with an average of 0.96. So it can be concluded that the items and instruments of assessment literacy have valid criteria based on content.

Keywords


literacy assessment; content validity; instrument

Full Text:

PDF

References


Anggara, D. S., & Haribowo, H. (2021, June). Content validity analysis on mathematical quantitative literacy instruments in elementary schools. In Journal of Physics: Conference Series (Vol. 1918, No. 4, p. 042091). IOP Publishing. doi:10.1088/1742-6596/1918/4/042091

Ashraf, H., & Zolfaghari, S. (2018). EFL Teachers' Assessment Literacy and Their Reflective Teaching. International Journal of Instruction, 11(1), 425-436.

Borg, W.R. dan Gall, M.D. 1984. Educational Research: An Introduction. New York: Longman.

Boyle-Baise, M. (2005). Preparing community-oriented teachers: Reflections from a multicultural service-learning project. Journal of Teacher Education, 56(5), 446-458. https://doi.org/10.1177/0022487105282113

Brookhart, S. (2011). Educational assessment knowledge and skills for teachers. Educational Measurement: Issues and Practice, 30, 3–12. https://doi.org/10.1111/j.1745-3992.2010.00195.x

Chadha, N. K. (2009). Applied psychometry. SAGE Publications India. https://doi.org/10.4135/9788132108221

Cohen dan Swerdlik. 2010. Psychological Testing and Assessment: An Introduction to Tests and Measurement 7th Edition. USA: McGraw-Hill Primis.

DeLuca, C., LaPointe-McEwan, D., & Luhanga, U. (2016). Approaches to classroom assessment inventory: A new instrument to support teacher assessment literacy. Educational Assessment, 21(4), 248-266. http://dx.doi.org/10.1080/10627197.2016.1236677

DeLuca, C., LaPointe-McEwan, D., & Luhanga, U. (2016). Teacher assessment literacy: A review of international standards and measures. Educational Assessment, Evaluation and Accountability, 28, 251–272.

Goodwin, L.D., dan Leech, N.L. 2003. “The Meaning of Validity in the New Standards for Educational and Psychological Testing”. Measurement and Evaluation in Counseling and Development, 36:3, 181-191. https://doi.org/10.1080/07481756.2003.11909741

Gotch, C. M. (2012). An investigation of teacher educational measurement literacy (PhD dissertation). Washington State University, Department of Educational Leadership and Counseling Psychology

Irwing, P., Booth, T., & Hughes, D. J. (Eds.). (2018). The Wiley handbook of psychometric testing: A multidisciplinary reference on survey, scale and test development. John Wiley & Sons. https://doi.org/10.1002/9781118489772

Johnston, P., & Costello, P. (2005). Principles for literacy assessment. Reading research quarterly, 40(2), 256-267. https://doi.org/10.1598/RRQ.40.2.6

Koh, K., Burke, L. E. C., Luke, A., Gong, W., & Tan, C. (2017). Developing the assessment literacy of teachers in Chinese language classrooms: A focus on assessment task design. Language Teaching Research. Retrieved from https://doi.org/10.1177/1362168816684366

Levi, T., & Inbar-Lourie, O. (2020). Assessment literacy or language assessment literacy: Learning from the teachers. Language Assessment Quarterly, 17(2), 168-182. https://doi.org/10.1080/15434303.2019.1692347

Lynn, M.R. 1986. “Determination and Quantification of Content Validity”. Nursing Research, 35, 382– 385.

Mardapi, Djemari. 2008. Teknik Penyusunan Instrumen Tes dan Non Tes. Yogyakarta: Mitra Cendekia.

Polit, D.F., dan Beck, C.T. 2006. “The Content Validity Index: Are You Sure You Know What’s Being Reported ? Critique and Recommendations”. Research in Nursing & Health, 2006, 29, 489–497 . https://doi.org/10.1002/nur.20147

O'Loughlin, K. (2013). Developing the assessment literacy of university proficiency test users. Language Testing, 30(3), 363-380.

https://doi.org/10.1177/0265532213480336

Popham, W. J. (2013). Classroom assessment: What teachers need to know (7th ed.). Boston: Pearson

Rempusheski, V.F., dan O’Hara, C.T. 2005. “Psychometric properties of the Grandparent Perceptions of Family Scale”. Nursing Research, 54, 419– 427

Richey, R.C. dan Klein. 2007. Design and Development Research. London: Lawrence Erlbaum Associates. Inc

Shams, J. A., & Iqbal, M. Z. (2019). Development of Classroom Assessment Literacy Training Program for University Teachers in Punjab. Bulletin of Education and Research, 41(2), 41-52.

Stoynoff, S., Chapelle, C. A., & Viii, P. (2005). ESOL tests and testing. Alexandria, VA: TESOL.

Stiggins, R. J. (1999). Assessment, student confidence, and school success. The Phi Delta Kappan, 81(3), 191-198.

Stiggins, R. (2004). New assessment beliefs for a new school mission. Phi Delta Kappan, 86, 22–27.

Sumintono, B., dan Widhiarso, W. 2015. Aplikasi Model Rasch Pada Assessment Neill, J. 2011. “Explratory Factor Analysis Survey Research and Design Psikologi”. Lecture Material. Centre or Applied Psychology University of Canberra.

Neill, J. 2011. “Explratory Factor Analysis Survey Research and Design Psikologi”. Lecture Material. Centre or Applied Psychology University of Canberra.

Nitko, A.J. 1996. Educational Assessment of Student (2nd edition). New Jersey: Merril.

Taylor, L. (2009). Developing assessment literacy. Annual Review of Applied Linguistics, 29, 21-36. https://doi.org/10.1017/S0267190509090035

Thiagarajan, S., Semmel, D. S., dan Semmel, M. I. 1974. Instructional Development for Training Teachers of Expectional Children. Indiana: Indiana University Bloomington

Van de Grift, W. (2007). Quality of teaching in four European countries: A review of the literature and application of an assessment instrument. Educational research, 49(2), 127-152. https://doi.org/10.1080/00131880701369651

Yamtim, V., & Wongwanich, S. (2014). A study of classroom assessment literacy of primary school teachers. Procedia - Social and Behavioral Sciences, 116, 2998- 3004. https://doi:10.1016/j.sbspro.2014.01.696

Zolfaghari, F., & Ahmadi, A. (2016). Assessment literacy components across subject matters. Cogent Education, 3(1), 1252561. https://doi.org/10.1080/2331186X.2016.1252561




DOI: https://doi.org/10.21831/cp.v42i2.55900

Refbacks

  • There are currently no refbacks.


Bookmark and Share


 

Social Media:

     

 

 Creative Commons License
Jurnal Cakrawala Pendidikan by Lembaga Pengembangan dan Penjaminan Mutu Pendidikan UNY is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Based on a work at https://journal.uny.ac.id/index.php/cp/index.

Translator
 
 web analytics
View Our Stats