EVALUATING THE IMPLEMENTATION OF ENGLISH COMMUNICATION THERAPY (ECT): AN OBJECTIVE STRUCTURED CLINICAL ASSESSMENT (OSCA) APPROACH

This study is aimed at evaluating the implementation of the English Communication Therapy (ECT) instructional program oriented to the Objective Structured Clinical Assessment (OSCA) for the students of the D-3 Nursing Study Program at STIKES Al-Irsyad Al-Islamiyyah, Cilacap. Three research problems are put forward concerning (1) the fit of the program in terms of the variables context, input, process, and product; (2) obstacles faced by students; and (3) attainment of the ECT learning objectives. The study is evaluation research using the qualitative and quantitative CIPP model. The research subjects are 168 students of the Nursing Study Program of STIKES Al-Irsyad Al-Islamiyyah. Data were collected using interviews, questionnaires, and documentation. The findings show that (1) evaluation of the four variables of context, input, process, and product shows that ECT can be stated as in the good or fit category; (2) main obstacles faced by students are lack of vocabulary and low level of self-confidence; and (3) students are able to achieve the passing grade of 74 at each station on the competency of English speaking about nursing matters.


Introduction
The Health Institute of Al-Irsyad Al-Islamiyyah is a health-based educational institute in the city of Cilacap, Indonesia, that has seven study programs. The institute offers two education levels of Diploma 3 (D-3) and Bachelor level (also known as Strata-1 [S-1]). There are four study programs in Diploma 3 level, there are Nursing, Midwifery, Pharmacy, and Physiotherapy study programs. In the Bachelor level, there are Medical Aid, Pharmacy, and Nursing study programs.
The vision and mission of the Institute are becoming Islamic, and superior and advanced in the global era with goals of producing graduates who are qualified and competitive. The institute expects the students to have hard and soft skills to support their competencies in facing the world of work. The D-3 program in Nursing is the most preferred study program that has most work in producing graduates that are competitive not only in the country but also overseas.
Competency assessment is an evaluation of nurses' competencies, which is, nowadays, is conducted by both the Computer-Based Testing (CBT) and Paper-Based Testing (PBT), using the method of Objective Structured Clinical Assessment (OSCA). Development of the preparation of nurses to take the competency assessment is done by the OSCA stream since this method has the best way to measure nurses' competencies up to the phase when examinees perform their competencies (the show how phase) (Masfuri et al., 2016). The OSCA method includes the evaluation of knowledge, communicating skills, physical examination skills, diagnostic analysis, and interpretation skills, knowledge, diagnostic skills, and interpersonal relation skills (Oermann & Gaberson, 2009).
The OSCA method of assessment is given in the curriculum with the expectation that the students are prepared for the assessment. The syllabus includes cognitive, affective, and also psycho-motoric skills of nurses to be able to carry out therapeutic communication with the patients (Yanti & Pertiwi, 2008).
The OSCA is a fair assessment model since every examinee receives the same exam materials. It is structured seeing that specific clinical skills are tested using specific evaluation sheets. Each station is constructed in a way that resembles the real clinical conditions. For each station, specific time allocations are determined. The examinees go through the stations consecutively. In each station, the examinees are given questions or tasks to answer or demonstrate, graded by the assessors. Subjects that are included in the OSCA examination are Medical Surgery Nursing, Child Nursing, Psychic Nursing, Maternity Nursing, Community Nursing, Emergency Unit, Islamic Nursing, Nursing Basics, and Physiology Anatomy.
In 2013, the D-3 program of Nursing, STIKES Al-Irsyad Al-Islamiyyah Cilacap, added one subject matter, namely English. According to Sahraini and Madya (2015), development of English education in many forms and places has motivated the D-3 Nursing study program to require students to be able to speak not only in the daily language but also in a foreign language, in this case, English. Competency in English speaking will help graduates develop their career when working abroad. In the English syllabus, the students receive English for Specific Purposes (ESP), where students are trained to communicate in English about nursing. The study program expects that graduates will be able to work not only in the domestic regions but also in international places. The ESP subject in the D3 Nursing study program is called English Communication Therapy (EST). It is aimed at helping students and graduates conduct two-way communication events with foreign patients in all situations, especially during clinical checks.
Education is a conscious and planned effort to provide learning situations and processes so that learners will be able to develop their potentials actively to acquire spiritual, religious competencies, self-management, personalities, intellectuality, noble 89 behaviours, and practical skills needed by themselves, the society, the nation, and the country (Law of Republic of Indonesia No. 20 of 2003). One of the behavioral changes is that in language proficiency. Acquisition of language, however, is attained by steps (Batang, 2014), influenced by the interaction between the learners and their external environment. In language learning, the learners will be expected to master aspects of the language (pronunciation, grammar, and vocabulary) and the four language skills reading, writing, listening, and speaking (Cambridge Assessment English, 2014).
According to Sahraini and Madya (2015), the learning of teaching of English in the field of language learning has passed through vast development and changes. Brown (2007, p. 7) states that learning can be defined as "showing or helping someone to learn how to do something, giving instruction, guiding in the study of something, providing with knowledge, causing to know or understand". Concerning program evaluation, Irambona and Kumaidi (2015) stress that "it is understood that program evaluation is the use of different social research methods to check the effectiveness of social programs, find out if the programs have been implemented as planned by the society, organization, or by the government". In this relation, the learning and teaching processes include showing and helping the learner how to do something, give instructions, make an analysis, prepare something, and understand the effectiveness of learning management and evaluation.
According to Yanti and Pertiwi (2008), the components needed to prepare for conducting OSCA are (a) station, post or place for the testing of examinees' knowledge and skills; (b) assessor team, examinees who work in the station to observe and evaluate examinees' demonstrations using check lists, (c) simulation patient, somebody who is trained well to play a role of patients on designed scenarios, specifically and realistically, usually in the form of cases; (d) fasilitator, person or unit who prepares materials and tools for demonstrations and provides things needed by the examinees; (e) timer, person who is in charge of moving the examinees from station to station using a bell mking sure that examinees movement is on time; (f) time allocation, amount of time given to examinees to do the demonstrations of each case, considering the examiner's judgement and level of difficulty of the task; (g) cleaning service, person similar to facilitator, who is in charge of keeping the place clean and conducive during the examination; and (h) grader, person who calculate the scores according to the standard guide provided by the assessing body to produce the decision whether or not an examinee passes the exam in each station. Passing is decided on not in the form of a total score, but on the score of each station.
The OSCA method is regarded to be more valid, reliable, and objective, compared to the conventional aural case examination in assessing clinical competencies, communication skills, and personalities and behaviors. The OSCA reliability level is dependent upon the number of stations. With six stations, in 90 minutes each, the reliability coefficient ranges only between 0.5 and 0.6. On the other hand, with 40 stations, in a total of 4 hours, the coefficient ranges up to 0.8 (Yanti & Pertiwi, 2008).
OSCA could be used as a valid and reliable instrument for assessing the nurses' skills if it used for selection or training of nurses with standard patients (McWilliam & Botwinski, 2012). A number of weaknesses of the OSCA method, however, are the small number of the stations in order to obtain adequate, reliable information about performances, the limited time allocation, the medium-level checklists that are not too easy nor too difficult, patients that are not standard, examiners who give scores not using the provided standard guide, and administering problems such as the noisy rooms and unorganized staff aids. The OSCA scenario may not be able to simulate an actual and ideal clinical situation because OSCA is quite expensive, needs a great number of logistics, and takes much time.
Therefore, this study is aimed at describing the English Communication Therapy (ECT) instructional processes using the Objective Structured Clinical Assessment (OSCA) method for students of the D-3 Nursing program, Al-Irsyad Al-Islamiyyah Cilacap. The study is an evaluation research using the CIPP (Context, Input, Process, Product) model. The study is expected to be able to give an evaluation to the ECT in terms of: (1) degree of fit between the instruction program and the context, input, process, and product (CIPP) system; (2) obstacles faced by students in attending the program; and (3) attainment of the objectives of the program.

Research Method
The study is evaluation research using the descriptive, qualitative, and quantitative approach. The research design refers to the CIPP evaluation model which was developed by Stufflebeam. This model was chosen for the reason that it is more comprehensive than other evaluation models. The four components to be used to evaluate the ECT program include context, background, and objectives of the ECT program; input, students' preparation and materials of the ECT program; proses, implementation of the ECT program; and product, results or achievement of the instructional process.
The subjects of the study were all of the 168 students of the D-3 Nursing Study Program. Data were collected by interviews, questionnaires, and documentation. The re-search instruments were developed based on operational definitions of the indicators.
Content validity was computed on the Aiken's V formula (Azwar, 2012, p. 113) with a criterion for a valid item of ≥ 0.70 (Sireci & Geisinger, 1995, pp. 246-247). The results of the Aiken computation with the Explanatory Factor Analysis (EFA) on SPSS software showed a figure of 0.869. with a Determinant of the correlation matrix of 0.003. The following analysis using the KMO-MSA gave a score of 0.748 and Barlett's Test of Sphericity of 926.668 at the degree of significance 0.000. Based on these statistics, it was decided that data analyses could be done. Further, it was found that, from the communality measures, all the variable had a score of > 50%. It was concluded that all variable explained the factors and, from the total variance, a total of 8 constructed gave a contribution to the validity at the figure of 67.083%.
Reliability was estimated by Intraclass Correlation Coefficients (ICC) with a bottom line of 0.4 (Fleiss, 1999, p. 7). Results of the ICC analysis showed a figure of a single measure of 0. 607, meaning that the reliability level was in the "good enough" category.
For the questionnaire, data were analyzed using the qualitative analysis technique. The first step was categorizing the data into the level of tendencies. Reference for score interpretation can be seen as in Table 1 (Mardapi, 2012, p. 162

Findings
Results of the analyses on the students' achievement of the ECT can be seen in the following presentations.
Frequency distribution of the scores of the input component variable can be seen in Table 7. For this component variable, it is clearly presented that a total of 66 student respondents (39.3%) state "highly fit", 71 students (42.3%) state "fit", 25 students (14.9%) state "fit enough", and 6 students (3.6%) state "not fit".

Product Evaluation
The results of the product evaluation show that, of the 168 student respondents, the highest score is 609, and the lowest score is 502. The ideal mean is then 1 2 ⁄ (609 + 502) = 555.5, and the ideal standard deviation is 1 6 ⁄ (609 − 502) = 17.8 (see Table 8). Frequency distribution of the scores of the product component variable can be seen in Table 9. It can be seen that h 24 student respondents (14.3%) state "highly fit", 82 students (48.8%) state "fit", 37 students (22%) state "fit enough", 16 students (9.5%) state "not fit", and 9 students (5.4%) state "highly not fit".   Based on the preceding presentation of the results of the study, a discussion can be made as follows. (1) The context evaluation gives a situation that can be regarded as good or fit. In the interviews, students state that there is relevance between the vision and missions of the Institute and the learning-teaching process of the ECT. It is seen that one objective of the D-3 program is creating a campus atmosphere that supports learning processes that global-oriented. (2) The input evaluation is also categorized as being good or fit. In this component evaluation, the objects of evaluation include students' preparedness, plan for learning material/cases and learning support services. Students are active and aware of the existence of ECT in the OSCA as a support to the improvement of students' competencies in aural communication. The instructional materials and cases in the ECT are developed on the basis of the curriculum and designed in accordance with the students' needs to improve their competencies. The availability of learning aids and facilities has supported the instructional processes of the ECT. (3) The process evaluation also shows results that are good or fit. It can be seen from the fact that students are prepared, instructional materials are developed and presented clearly, time management is in accordance with the amount of the materials/cases, and the learning and teaching processes follow the guides given in OSCA. It can be stated that the implementation of ECT runs in a conducive situation. Students' understanding of the roles helps in supporting the learning processes such that obstacles can be minimized and supports can be maximized. (4) The product evaluation is in the good or fit category. It is seen from the score mean of 562.1, which is higher than the success category of 555.5. For graduation, students are declared as graduating if they obtain the passing grade of 74 at every exam station. It also applies to ECT. Students are regarded as competent if they can communicate in English as a nurse.
The obstacles in the implementation of ECT are uncovered from the interviews with students. Among others, obstacles that are often mentioned by students are: (1) over-nervousness experienced by students, (2) low motivation caused by inability to converse in English, (3) pessimism in their thought that ECT is not relevant to nursing, (4) students' low level of vocabulary mastery, and (5) low level of students' self-confidence to communicate in English.
From the questionnaires and students' evaluation scores, it is shown that students acquire scores that are the same or higher than the passing grade. It can be concluded that students do well in the ECT program. Students' competencies to communicate in English can be regarded as quite good as they can communicate in English with other people in clinical matters that they master. From the point of view of the OSCA, all students can fulfill the demand of the passing grade determined by the study program. Students' scores are consistent with their performances in the evaluation sheets which carry weighted points. It can be concluded that students can fulfill the requirements of the ECT program.

Conclusion
Based on the research problem, theoretical reviews, data analyses, and research findings, four items of conclusion can be drawn as follows: (1) implementation of the ECT for the students of the D-3 Nursing Study Program at STIKES Al-Irsyad Al-Islamiyyah Cilacap can be categorized as good or fit for the four components of context, input, process, and product; (2) the most prominent obstacles is students' lack of vocabulary inventory so that they are not confident enough to speak English; (3) attainment of the ECT learning objectives can be seen from students' abilities to achieve the passing grade of 74. Students' competencies are in the form of their ability to communicate in English about nursing matters; (4) implementation of ECT with the foci of context, input, process, and product carries in an implication for students in the form of support to students to be able to communicate not only in their mother tongue but also in a foreign language.