Developing an observation instrument for assessing the effectiveness of English teaching at vocational secondary schools

Estu Widodo; Franciscus Xaverius Sudarsono

doi:10.21831/reid.v2i2.8648

Developing an observation instrument for assessing the effectiveness of English teaching at vocational secondary schools

Estu Widodo, Universitas Muhammadiyah Malang, Indonesia
Franciscus Xaverius Sudarsono, Universitas Negeri Yogyakarta, Indonesia

10.21831/reid.v2i2.8648

Abstract

This study was aimed at: (1) developing an observation instrument for assessing English teaching at vocational secondary schools (VSSs), (2) examining the validity of the developed instrument, (3) examining the inter-rater reliability, and (4) assessing the effectiveness of the developed instrument. The subjects of the three-phase field-tests were English teachers and students of four VSSs. Observation sheets, questionnaires, and interviewes were employed to collect the data. The instruments were field-tested in three phases. Expert judgment, Kappa Coefficient, and Kane’s (2006) argument-based validation and Bachman & Palmer’s intepretive and use argument were used to determine the instrument validity and reliability. The field-test which involved stakeholders prove that the observation instrument could be implemented effectively to assess the teaching of English at VSSs.

Keywords

assessment instruments; effective teaching; English; hotel accommodation

Full Text:

PDF

References

Ackerman, D., Gross, B.L., & Vigneron, F. (2009). Peer observation reports and student evaluations of teaching: Who are the experts?. The Alberta Journal of Educational Research, 55, 18-39.

Adamson, L.S. (2008). Development and evaluation of an instrument to assess data informed instructional practice (Doctoral dissertation, John Hopkins University, 2008). Ann Arbor, MI: ProQuest/UMI. (Publication No. AAT 3309798).

Alkin, M.C. (Ed). (1992). Encyclopedia of educational research (6th ed.). New York, NY: Macmillan.

Arthur, W., Tubre, T., Paul, D.S., & Edens, P.S. (2003). Teaching effectiveness: The relationship between reaction and learning evaluation criteria. Educational Psychology, 23, 275-285.

Bachman, L.F. (1990). Fundamental consider-ations in language testing. Oxford: Oxford University Press.

Bachman, L. & Palmer, A. (2010). Language assessment in practice. Oxford: Oxford University Press.

Borich, G.D. (2007). Effective teaching methods (6th ed.). Upper Saddle River, NJ: Prentice Hall.

Borg, W.R. & Gall, M.D. (1989). Educational research: An introduction (4th ed.). New York, NY: Longman.

Brown, H.D. (2007). Principles of language learning and teaching (5th ed.). White Plains, NY: Pearson Education.

Burhanuddin. (2008). Mengeliminasi kesulitan siswa menjawab soal ujian nasional bahasa Inggris melalui pembelajaran. Pendidikan Network. Retrieved from: http://research-engines.com/burhan-udin20708.html.

Carruth, P.J. & Carruth, A.K. (2003). Evaluation of college faculty: What do accounting students really think?. Journal of College Teaching and Learning, 9, 1-8.

Cashin, W.E. (1990). Student ratings of teaching: Recommendations for use. Center for Faculty Evaluation and Development. Kansas, KS: Kansas State University.

Creemers, B.P.M. & Kyriakides, L. (2008). The dynamics of educational effectiveness: A contribution to policy, practice and theory in contemporary schools. London: Routledge.

Curtis, A. & Cheng, L. (2007). TAPping into teaching effectiveness: A collaborative approach to performance appraisal of ESL teachers in a Canadian context. In Coombee, C., M. Al-Hamly, Davidson, P. et al. (Eds.). Evaluating teacher effectiveness in ESL/EFL contexts. Ann Arbor: The University of Michigan Press, 57-72.

Djojonegoro, W. (1998). Pengembangan sumber daya manusia melalui SMK. Jakarta: Depdikbud.

Elliott, A.C. & Woodward, W.A. (2007). Statistical analysis with spss examples quick reference guidebook. Thousand Oaks, CA: Sage.

Emery, C.R., Kramer, T.R., & Tian, R.G. (2003). Return to academic standards: A critique of student evaluations of teaching effectiveness. Quality Assurance in Education, 11, 37-46.

Field, A. (2009). Discovering statistics using SPSS. (3rd ed.). London: SAGE.

Gower, R., Philips, D., & Walters, S. (1995) Teaching practice handbook (New ed.). Oxford: Macmillan Education.

Kaluge, L. (2003). Inequity progress of pupil groups of upper primary schools. Jurnal Ilmu Pendidikan, 10 (3), 237-253. http://journal.um.ac.id/index.php/jip/article/view/242

Kaluge, L. & Hadiwijaya, A.S. (2007). Good practices among current educational programs for basic education in Indonesia. Jurnal Ilmu Pendidikan, 14 (3), 144-153. Retrieved from http://journal.um.ac.id/index.php/jip/article/view/34/310

Kane, M. (2006). Validation. In Brennan (Ed.). Educational measurement (4th ed.). Westport, CT: American Council on Education and Praeger, 17-64.

Kane, M.T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50, 1-73.

Kaplan, R.M. & Saccusso, D.P. (1982). Psychological testing: Principles, application, and issues. Monterey, CA: Brooks/Cole.

Kyriakides, L. & Creemers, B.P.M. (2009). The effects of teacher factors on different outcomes: Two studies testing the validity of the dynamic model. Effective Education, 1 (1), 61-85.

Littlewood, W. (2011). Communicative language teaching. In Hinkel, E. (Ed.). Handbook of research in second language teaching and learning volume II (pp.541-557). New York, NY: Routledge.

McCaffrey, D.F., Koretz, D., Lockwood, J.R., & Hamilton, L.S. (2003). Evaluating value-added models for teacher accountability. Santa Monica, CA: RAND.

Ministry of Education and Culture. (2015). Indonesia education strategic plan 2015-2020. Jakarta.

Mitzel, H.E. (Ed). (1982). Encyclopedia of educational research (5th ed., Vol. 4). New York, NY: The Free Press.

Mohanty, G., Gretes, J., Flowers, C., Algozzine, B., & Spooner, F. (2005). Multi-method evaluation of instruction in engineering classes. Journal of Personnel Evaluation in Education, 18, 139-151.

Musthafa, B. (2001). Communicative language teaching in Indonesia: Issues of theoretical assumptions and challenges in the classroom practice. TEFLIN Journal, 12, 184-193.

Nougaret, A.A., Scruggs, T.E., & Mastropieri, M.A. (2005). Does teacher education produce better special education teachers?. Exceptional Children, 71, 3, 217-229.

Oliveri, M.E., Lawless, R., & Young, J.W. (2015). A validity framework for the use and development of exported assessments. Princeton, NJ: ETS.

Ornstein, A.C. (1991). Teacher effectiveness research: Theoretical considerations. In Waxman, H.C. & Walberg, H.J. (Eds.). Effective teaching. Berkeley, CA: McCutchan, 63-80.

Ornstein, A.C. & Lasley II, T.J. (2004). Strategies for effective teaching. (4th ed.). Boston, MA: McGraw-Hill.

DOI: https://doi.org/10.21831/reid.v2i2.8648