THE EVALUATION OF ONLINE SCHOOL EXAMINATION IMPLEMENTATION USING CIPP MODEL

This study aims to evaluate the implementation of online school exam conducted at SMK PGRI 1 Salatiga in 2019/2020 Academic Year. This study is descriptive evaluation research using CIPP (Context, Input, Process, Product) model. The research focuses on conducting evaluation activities through four-component aspects (Context, Input, Process, and Product evaluation). Data were collected through (1) interviews with the Principal, Deputy Principal for Curriculum, School Examination Committee, and (2) documentation study, conducted by collecting data information related to the implementation of the online school exams at SMK PGRI 1 Salatiga. This study involved 194 subjects consisting of 181 school exam participants and 13 school exam committees at SMK PGRI 1 Salatiga. This evaluation research results can be seen through the four aspects of evaluation component, namely (1) Context in the implementation of online School Exams has been based on a Circular from the Minister of Education and Culture and the Office of Education and Culture of Central Java Province. (2) In-puts in the implementation of online School Exams are in accordance with the planning crite-ria in implementing school exams, including online application preparation, school exam planning and schemes, network preparation, Standard Operating Procedures (SOP) as committee test organizers. (3) The process of organizing online school exams goes well, but there are technical problems experienced by school exam participants related to internet network. (4) The product in organizing online school exams is in a quite good category, indicated by the results of online school exams carried out in a relatively short time.


INTRODUCTION
The government policy regarding the determination of the emergency period of the COVID-19 spread has a large-scale impact on various aspects of life, one of which is in the field of education. The policy significantly changes the mechanism of implementing learning activities in schools. Learning activities that were initially carried out face to face turned into online learning activities. That change also affects the implementation of school exams. School exams, which were originally carried out on a paper-based basis, were changed to be electronic-based or online. This policy directly provides meaning that school exams whose implementations usually have alternatives in the form of the online, full paper, and practice tests, turns into one form, online. Decisions on the implementation of online school exams were issued by the Minister of Education and Culture and strengthened through a circular from the Education and Culture Office number 443.2/08997 of 2020, regarding the Implementation of Distance Teaching and Learning Activities in Vocational High Schools (sekolah menengah kejuruan or SMK), High Schools (sekolah menengah atas or SMA), and Special Schools (sekolah luar biasa or SLB) in Central Java Province. It implies that the implementation of school exams in high school, vocational high school, and special schools is carried out online.
From this policy, the government provides a legal basis and authority for education units to conduct online school exam activities. Online school exams are learning evaluation activities carried out to determine the graduation of twelfth-grade students at a high school/ vocational level of education at the same level through an application program conducted online. Online school exam activity is a learning evaluation activity that aims to measure students' ability by considering existing conditions. The evaluation concept is interpreted as an effort to collect, analyze, and obtain information on a program (Wirawan, 2012, p. 17) School exams are part of the education program to measure the extent to which students' learning outcomes are achieved, which in their implementation are carried out by the school. Various efforts continue to be made by the central and regional governments in the context of conducting online school examinations so that they run well and smoothly. Organizing an online exam could provide several benefits for schools as educational providers because online-based exams would be able to replace the role of paper-based exams (Yulianto, et al., 2016, p. 93). Implementation of online school exams would undoubtedly have a different impact compared to the offline or written exam. The same thing was also delivered (Santi & Prajana, 2018, p. 80) that there would be a significant difference for schools between the implementation of online examinations and paper-based written examinations.
The policy in conducting online school examinations carried out by education units has its considerations from each school's internal side. Basically, two factors influence the implementation of online school exams, including internal factors and external factors, which can be a problem and a separate obstacle for the school administering the exam. It was also experienced by SMK PGRI 1 Salatiga. As one of the first online school exam administrators in the academic year of 2019/2020, SMK PGRI 1 Salatiga also experienced obstacles in its preparation and implementation. The policy for the test implementation was based on a Circular issued by the Ministry of Education and Culture, as well as the Office of Education and Culture of Central Java Province, which was issued through an Official Memorandum of Head of the Education Office Branch, Region V, Number: 422.1/0484/Cabdin V/2020 on March 19, 2020, concerning the implementation of online school exams.
Referring to the explanation, that SMK PGRI 1 Salatiga has been carrying out written school examination activities, but with the issuance of an Education Office note regarding the implementation of online school exams, of course it becomes a problem in itself for SMK PGRI 1 Salatiga. Students who have been carrying out exam activities at school are forced to carry out examination activities at the residence of each student online. The existence of different conditions of living areas and internet networks that are less supportive and adequate for students, is a gap for students. On the one hand, the location of the residence is less supportive and the internet network is inadequate, but on the other hand, students must take the online exam from their place of residence. Based on the gaps in these conditions, it is necessary to implement online school exams at SMK PGRI 1 for evaluation. One evaluation model that can be used is the CIPP (Context, Input, Process, and Product). This study aims to determine how the online school exams at SMK PGRI 1 in terms of Context, Input, Process, and Product. The results of this study are expected to be used by stakeholders as a reference for policy making in the implementation of the next online exam and other relevant decisions.
A similar study was previously conducted (Putra et al., 2019, p. 47) at SMK 1 Jakarta. The results showed that in the context aspect covering school policy, the online exam was implemented well. For input aspects, the online exam at SMK Negeri 1 Jakarta, which includes facilities and infrastructure, software and hardware, test questions, and implementation man-agement, has been partially adequate and in accordance with the success criteria. In the aspect of the process, the implementation of an android-based online exam, including supervision, the consequences of violations, the jockey system, and the Google block system, was still lacking, namely in the Google block system, so that the results of student grades were guaranteed purity. Finally, in the product aspect, the online exam could streamline time in the value input process. Besides, the same research was also carried out (Handayani & Krisnadi, 2019, p. 7) on the Evaluation of the Web-Based Student Online Examination Program at SMK Negeri 2 Kota Tangerang Selatan. This study also used the CIPP evaluation model.

RESEARCH METHOD
The approach used in this research was descriptive qualitative. Qualitative research is research that pursues to see the phenomenon of events more comprehensively and deeply (Sugiyono, 2013, p. 20). The purpose of this research was to evaluate the implementation of online school exams by identifying data, facts, and events. The evaluation model used to collect data and information in this study was the CIPP model that sought to assess a program through four components: context, input, process, and product (Stufflebeam & Shinkfield, 2007, p. 98) . The CIPP evaluation model seeks to carry out evaluation activities through four evaluation components: context, input, process, and product (Wirawan, 2012, p. 73).
Subjects in this study consisted of committees and participants of the online school exam at SMK PGRI 1 Salatiga. It involved 194 subjects, consisting of 181 school exam participants and 13 school exam committees. The implementation time for online school examinations was from March 23 to March 27, 2020. The place of implementation was at SMK PGRI 1 Salatiga as a question operator and each student's residence as online school exam participants. Data collection techniques were carried out through (1) interviews conducted with the Principal, Deputy Principal for Curriculum, School Examination Committee, and (2) documentation study, conducted by collecting data information related to the implementation of the online school exams at SMK PGRI 1 Salatiga. Meanwhile, the data analysis techniques included (1) data collection, (2) data reduction, (3) data display, and (4) verification/conclusion of data, as elaborated by Miles and Huberman in Sugiyono (2013, p. 334).

Findings
The CIPP model evaluation concept was done through four stages: context, input, process, and product evaluation. The stages that would be outlined in the context evaluation were the things that underlay the need for online school examination. The online school exam was held based on government policies in the field of education related to the prevention and handling of the COVID-19 case. Circular of the Minister of Education and Culture No. 4 of 2020 about the Implementation of Education Policy during the Emergency Spreading of COVID-19 became one of the main bases for the implementation of online school examinations at SMK PGRI 1 Salatiga. Through this circular, all teaching and learning activities held at school were changed through online learning programs. It was conducted for around four months, from March to June 2020. Thus, the implementation of online teaching and learning activities at SMK PGRI 1 Salatiga completely changed the mechanism of the exam implementation, which was initially carried out on a paper-based basis into electronic-based or online.
The emergence of the COVID-19 pandemic forced schools to carry out online learning and school exams. It has become a demand for SMK PGRI 1 Salatiga to utilize all human resources and learning support facilities to realize online school exams. This online school exam reinforces government policy that education units are prohibited from bringing students to school for carrying out school exam activities. In fact, most SMK PGRI 1 Salatiga students came from and domiciled in areas outside the city of Salatiga, shown in Table 1. Based on the data in Table 1, the most students coming from the city of Salatiga was 50 people or 12.43%, Semarang Regency was 230 people or 57.21%, Magelang Regency was 98 people or 24.37%, and Boyolali Regency was 24 people or 5.97%. It indicates that the majority of the residence of SMK PGRI 1 Salatiga students came from outside Salatiga city around 344 people or 87.55%, spread across three districts. Therefore, amid the COVID-19 pandemic, the school was very unlikely to hold learning activities and exam activities. Based on the data description, in terms of context evaluation, the implementation of online-based school examinations has been based on government policies regarding the prohibition of presenting students to school in the pandemic outbreak of COVID-19.
Then, the input evaluation of implementing online school exam activities included planning concepts and supporting facilities. The policy on organizing online school exams at SMK PGRI 1 Salatiga was one possible alternative for conductint school exam activities amid a pandemic of COVID-19. The planning concept in organizing online school exams began with the preparation of Standard Operating Procedures (SOP) and schemes. The SOP preparation and scheme were made to organize the technical and clear flow in conducting online school examinations. At SMK PGRI 1 Salatiga, the online school exam implementation employed Google Form application.
The online school exam was conducted using a remote system, where students worked on the exam questions via an android smartphone, laptop, or PC in their home as long as they were connected to an internet connection. Whereas, the operator of the manuscript or exam questions was controlled directly by the exam committee at school to ensure that the test was running well and smoothly. The mechanism regarding the online exam implementation was specially arranged through the test implementation scheme, which accommodated the flow of exam implementation from the beginning to the end. Technical matters in the online school exams implementation referred to SOP and schemes that have been designed. It included the implementation schedule for each subject, techniques in working with the question, and the supervisor of the operator for each class. Online school exams were held for four days, with seven subjects tested. The online school exam scheme and plan is as shown in Figure 1.
Based on Figure 1, the flow in working on online school exam questions by grade XII students began with the attendance process. Before starting the school exams, students should send the "Share Location," where they were working on online school exams to each supervisory operator for each class and fill out an online attendance list. In the next stage, they would get a link to work on the questions from the supervisory operator of each class. After completing the questions, they submitted the answers to the school exam committee.
The implementation of school exams online cannot be separated from the support of facilities and infrastructure. Supporting infrastructure in conducting online school exams required by the committee included laptops, PCs, LCD screens, and internet networks as the primary means. Meanwhile, the facilities and infrastructure needed by students were Smartphone/Android mobile, internet quota, and internet network.  Furthermore, process evaluation pursued to evaluate the implementation of plans in organizing online school exams. The process evaluation's focuses were the schemes and work plans' effectiveness, facilities and infrastructure, and obstacles in the implementation of online school examinations. Implementation in the process of conducting online school examinations implied to the scheme, work plan, and standard operating procedures that have been made by the committee and with the approval of the relevant officials. The implementation phase was executed well; however, there were technical obstacles and problems experienced by students. These problems included the problem of access to online exams and technical problems of geographical location, which caused the internet network to be substandard, and the lack of supporting facilities. An overview of the implementation of the online school exam work plan and scheme is presented in Figure 2.
Based on Figure 2, the stages of doing online school exams included four steps: (1) Attendance, (2) Share Location, (3) Access Link, and (4) Submit. On the first day to the fourth day, participants who could do online attendance were 181 people, and participants who were able to do Share Location were 181 people. At the stage of accessing the question link from the first day to the third day, some participants experienced problems accessing the question link and submitting the question. Through these data presentations, it can also be elucidated that the implementation of schemes and work plans has been going well, although there were still obstacles for students, especially from the first day to the third day. The obstacles experienced by participants in the implementation of online school examinations were inseparable from students' technical factors. Based on the interviews of several students through video calls, one factor that was an obstacle for students in taking online school examinations was the weakness of the internet network, which was influenced by the geographical location of the student's residence. These findings were also strengthened through the results of interviews from the head of the school exam committee, that most of the obstacles experienced by test participants were problems with the internet network where each student lived. The results of these findings prove that in process evaluation, problems in implementing school exams were more experienced by students. At last, product evaluation sought to gather information about the results or products in conducting school exams online. The products produced in the implementation of online school exams were in the form of test results from students or examinees. The results of student test work could be processed automatically through the Google Form application, so it was more efficient. The product results that were raised through the Google Form application were in the form of scores for each student. Besides that, through the application, the students' work could also be revealed in the form of answers results analysis so that the error level could be identified. The implementation level of online school exams was measured by the students' success in working on school exam questions. The success and smoothness of students in taking the school exam can be shown in Figure 3.  Through Figure 3, it can be explained that the distribution map of students' participation in each class in the online school exam activities was quite varied. It is shown in the diagram of students who took the make-up test and those who did not take the make-up test during the online school examination activities. The seven classes that took online school exams showed quite high participation results. The highest level of participation was shown by class XII KP B, with a percentage of 100%. Meanwhile, for class XII KP A, C, and D, and XII TB, some students had to repeat in doing the online school exams. The lowest percentage was indicated by class XII TLM, where seven students or 20% had to take a make-up exam. Referring to the results of data collection obtained, the product evaluation indicates that the implementation of online school exam product evaluations has been going well, but there were still obstacles experienced by test participants.

Discussion
The evaluation of the online school examination implementation at SMK PGRI 1 Salatiga using the CIPP model sought to provide an overview of the aspects of Context, Input, Product, and Process. The study results are expected to be a policy recommendation on follow-up in holding online school examinations in the future. Based on the research findings, in the context evaluation, it can be explained that online school exams implementation has been based on the aspect analysis of the background needs in the program implementation and based on a clear policy foundation. The release of a circular from the Minister of Education and Culture No. 4 of 2020 concerning the Implementation of Education Policy during the Emergency Spread of COVID-19 became the background for organizing online school examinations at SMK PGRI 1 Salatiga. The policy on the COVID-19 emergency period set by the government forced the government to forbid schools to bring students to come to school to carry out learning activities. The background in evaluating this context was based on policies issued by the government in handling COVID-19. It is in line with the research results (Yunus & Rezki, 2020, p. 227) that for the background in handling COVID-19, the government needs to issue a policy to cut off the spread of COVID-19 virus chain. The policy was the background of context evaluation in research, where there is a strong legal basis for conducting online examinations. On the one hand, the school should carry out school examinations, and it is a requirement of the school as one of the considerations for determining the grade or student graduation. Besides, it is consistent with Wirawan's opinion that identifications, needs, and policies are the primary basis in deciding context evaluation (Wirawan, 2012, p. 92).
Referring to the research findings and other research results and reinforced by a theoretical study, the context evaluation has been based on the need for online school exams. It was through policies that have been set by the government as the background of context evaluation. Based on this description, it can be concluded that the context evaluation in the implementation of online school examinations had a good category.
Moreover, the evaluation of the input aspect was a form of follow-up in conducting online school exams. At this stage, the school has established a policy on alternative steps needed to be taken in conducting school examination activities amid the pandemic of COVID-19. The input component in organizing school exams included planning and facilities used as support in the activities. In the input evaluation, there have been supporting components in organizing an online school exam program to run.
The Google Form application system was employed as a tool that is clearly capable of supporting the implementation of online school exam activities. With the support of an application system, it will facilitate performance in the program implementation. It is also supported by the research results (Liu & Yu, 2018, p. 841) which revealed that the existence of a system would provide support for a program. The same research results were also shown by Hémard and Cushion (2010, p. 119) that the need for full support in implementing a web application to implement a program. Referring to the research findings and supported by several relevant studies, a conclusion can be drawn that input evaluation in conducting online school exams at SMK PGRI 1 Salatiga had a good category.
Furthermore, the process evaluation stage was an assessment of plan implementation that has been made in the online school examinations. The findings in the process evaluation have shown some of the problems that existed in the online school exam implementation carried out at SMK PGRI 1 Salatiga. Obstacles were in the form of internet networks and inadequate facilities owned by students, which seems to be a significant obstacle in the program implementation. The geographical condition of the student's residence was also one of the factors that hindered the smooth running of the internet network. Thus, the implementation of online school exams did not always work well. Similar findings were also shown from (Capa-Aydin, 2016, p. 112) that the average usage level of online examinations did not always provide results and run well. However, the findings of the study are different from (Crews & Curtis, 2011, p. 865) that online tests or evaluations can run well if the level of respondents' participation is high. Performing online tests technically also requires considerable time and readiness, as stated by (Peterson, 2016, p. 18) Based on the analysis of research findings and other research results, it can be concluded that the process evaluation of conducting online school examinations had a good category, although some students experienced obstacles and technical problems in the field.
Product evaluation was the last stage in conducting online school examinations. The product evaluation measured the extent of the products produced in the organization of online school examinations at SMK PGRI 1 Salatiga. The use of online tests through an application was to measure students' abilities. Through an application, it will be easier to evaluate measurements (Nakayama et al., 2010, p. 173). The research findings stated that the product evaluation of student participation levels in online school exam activities was quite high. It was indicated by the percentage of students who took the make-up exam was no more than 30%, and students have successfully conducted online school examinations. However, technically, there were still insignificant obstacles that occurred in the implementation of online school examinations, especially in product evaluation. Referring to the research findings, online school exams have shown quite good results. The results of this study are in line with the results of research conducted by (Rosidin et al., 2019, p. 287) and (Fahrurrohman et al., 2019, p. 8), which concluded that the implementation of ujian nasional or UN (National exam) and ujian sekolah berstandar nasional or USBN (National Standard School Exams) through online applications or CBSE has been able to provide good results for educational units.
Referring to the research findings and discussion, the implementation of online school exam programs at SMK PGRI 1 Salatiga was included in the incremental decision category, namely decisions taken to improve the program (Rose & Nyre, 1997, p. 17). It can be interpreted that the implementation of the next online school exam still needs to be continued, but there needs to be improvement and development in its implementation to minimize obstacles and problems that have occurred before.

CONCLUSION
Based on the results of the research and discussion above, regarding the evaluation of the online school examination in SMK PGRI 1 Salatiga, the following conclusions can be drawn. (1) At the context evaluation stage, it can be stated that it was included in the good category. It was based on the Education Policy during the Emergency Spread of the COVID-19 pandemic, so the need for online school exams was held. (2) The input evaluation of this study results was included in a good category. It was indicated the preparation of work plans, SOPs, and the use of the Google Form application to be supporting input in organizing online