Integrating metacognitive strategies impact in virtual science experiments for undergraduate students’ hots

Higher Order Thinking Skills (HOTS) are essential variables that support the achievement of science learning objectives. However, distance learning during the Covid-19 Pandemic was the cause of the lack of maximum HOTS for undergraduate science students due to their lack of involvement in experiments. Therefore, this study aims to apply a metacognition-integrated science virtual experiment model and examine its impact on students' HOTS science. A quasi-experiment was involved with a randomized pretest-posttest comparison group design to see the impact of the implementation of this model. Participants in the two treatment groups were randomly selected from two private universities in Indonesia. The HOTS of participants were assessed using multiple-choice questions. Observation sheets were used to measure the implementation of the learning model being developed. A general linear model with MANOVA was used to test the effect of the model, while Partial Eta Squared was used to measure the effect size of the model on HOTS. The results showed that the virtual science experimental model integrated with metacognition strategies significantly affected students' HOTS. The effect size measurement shows a high effect in the experimental group. Researchers recommend that universities apply a similar model to encourage students' achievement of HOTS.

The Covid-19 pandemic provides the impetus for institutions to change the learning process (Giatman et al., 2020;Stevanović et al., 2021).This necessitates lecturers' and students' readiness to use technology and solve problems effectively (Gestiardi et al., 2021).The situation in the field is the lack of preparedness of lecturers and students to adapt to the rapid changes, so the followup to learning is also not optimal (Firmansyah et al., 2021).As a result, there are many learning losses (Storey & Zhang, n.d.), learning objectives are not achieved (Yusuf, 2021), and student interest and involvement in distance learning are very low (Nambiar, 2020).These problems have a direct impact on student HOTS.
Higher-order Thinking Skills (HOTS) play a significant role in developing and applying scientific concepts in adult learners.These skills assess an individual's memory and his or her capacity to analyze, synthesize, and evaluate (Thomas & Thorne, 2009).The higher-order thinking process is pursued through remembering, understanding, applying, analyzing, making judgments, and making decisions (Brookhart, 2010;Heong et al., 2011).
The strong demand for the development of HOTS in the Institutes of Education and Education Personnel (IEEPs) runs counter to the HOTS of pre-service teachers at these institutions.It means that many prospective teachers in IEEPs have low levels of thinking (lowerorder thinking skills) (Gradini et al., 2018;Wiyoko & Aprizan, 2020).According to studies, the learning models in IEEPs have not been able to promote HOTS in their students.Previous research has been conducted to assess the effectiveness of the following learning models in improving college students' HOTS: PBL (Fakhriyah, 2014), RMS (Reading, Mapping, and Sharing) (Diani et al., 2018), CUPs (Conceptual Understanding Procedures) (Saregar et al., 2016), Constructive Controversy (CC) and Modified Free Inquiry (MFI) (Pratiwi, 2014), film (Anthony et al., 2014), and Guided Inquiry Laboratory-Based Module (Prihmardoyo et al., 2017).Unfortunately, research in Indonesia still revolves around HOTS measurement and analysis instruments.Only a few academics have developed a distance learning method built on HOTS empowerment, and many overlook the incorporation of the aforementioned learning models into distance learning during the Covid-19 epidemic.
The key to success in distance learning is independence (Kauffman, 2015).The virtual science experiment model must be developed in such a way that student independence is maximized.One approach would be to incorporate metacognitive strategies into the learning model (Panahandeh & Asl, 2014).Metacognition is the knowledge of cognition as well as the regulation of cognition (Winne, 2017).The former involves metacognitive knowledge and experience, whereas the latter incorporates metacognitive strategies.This theory is relevant in this case because the metacognitive dimension is the most important dimension of knowledge after the factual, conceptual, and procedural dimensions.
Previous related studies show that metacognitive strategies have several advantages, including assisting students in monitoring their progress and controlling their learning process (via reading, writing, and problem-solving), contributing to the learner's desire to learn beyond his or her intellectual abilities; and improving student academic achievement across age, cognitive abilities, and learning domains, including reading, writing, math, reasoning, and problem-solving (Veenman et al., 2004;Wang et al., 1990).Undergraduate students' metacognition should be optimized for them to sharpen their thinking skills in overcoming real-world problems (Kleitman & Narciss, 2019).
Students can engage in metacognition activities in the classroom by reflecting on the thinking processes involved in the learning process; looking for other concrete examples from previous learning experiences and thought patterns; analyzing the benefits of using the mindset and the disadvantages of not using it, leading to an understanding of where the strategy should be used; and generalizing and formulating rules regarding these thought patterns (Zohar, 1999(Zohar, , 2004;;Zohar & Dori, 2012).Previous research has also succeeded in developing a metacognition-based learning model called MiSHE (Metacognition in science for higher education) (Maryani, Prasetyo, & Wilujeng, 2021).This model has the advantage of involving students from lesson planning to reflection.The MiSHE model is also claimed to be successful in training students' self-regulation for distance learning and managing the tasks assigned to them.Furthermore, one part of the MiSHE Model is virtual experiment-based learning.Thus, researchers are interested in implementing this learning model, which is integrated with metacognition strategies, to determine its effect on students' HOTS Science.

METHOD
This study used an experimental study with a randomized pretest-posttest comparison group design (Creswell, 2012), to know the impact of this model compared with the other model.
The experimental class consisted of 39 students, while the control class consisted of 40 students.The research design used a randomized Pretest-Posttest Comparison Group Design referring to Creswell (2012).
HOTS data is collected through a test that uses seven valid questions for each material (there are seven materials in this lesson).The indicators of HOTS questions are logic, reasoning, analysis, evaluation, creation, problem-solving, and decision making (Heong et al., 2011;R. Marzano, 2013;R. J. Marzano, 1993; R. J. Marzano & Kendall, 2006;Zohar & Dori, 2003).The validity and reliability of the questions are sought with response theory items for multiple-choice questions.The construct validity of the items was analyzed using item response theory with EFA (Explanatory Factor Analysis).The results can be seen in the attachment and previously published by Maryani, Prasetyo, Wilujeng, et al (2021) Data analysis was performed using the General Linear Model with Multivariate of Variance (MANOVA).After establishing the significant effect of the model on students' HOTS, the effect size was calculated.Effect size shows the degree of the experiment model's significant effect on students' HOTS.It is defined as the standard deviation between the control and experimental groups' scores.Cohen's d is the appropriate effect size in this circumstance.A large Cohen's d value indicates that the difference between the control and experimental groups is significant.The effect size was also computed in MANOVA using Eta squared.An Eta squared value of 0.01 suggests a small effect, 0.03 indicates a moderate effect, and 0.5 indicates a big effect (Bakker et al., 2019;Cohen, 1988;Mordkoff, 2019).
Hypothesis H0: The integrated metacognition virtual practicums have significant effects on students' HOTS.H1: The integrated metacognition virtual practicums have no significant effects on students' HOTS.

Finding
The syntax of the metacognition integrated virtual science learning model results from integrating metacognition theory into virtual experiments.The syntax for this model uses the MiSHE model, which was previously developed by Maryani, Prasetyo, et al (2021) and Maryani et al (2022) by integrating metacognition into project-based learning.The syntax is: awareness, essential questions, planning, monitoring, evaluation, and reflection.
The learning process took place over seven sessions in two groups at different universities.Researchers involved three practicing lecturers as observers who also provided input on the implementation of the model.Observations are carried out through monitoring in Google Classroom so that the syntax of this model can be monitored optimally.Learning is carried out online using Google Classroom, Google Meet, Google Forms, YouTube, and the PhET simulation to carry out each stage of the model.The practicum carried out by students can be seen in Table 1.Metacognition consists of aspects of knowledge and regulation.Metacognitive knowledge consists of three components, namely awareness of knowledge/person variables, awareness of thinking/task variables, and awareness of thinking/strategy variables (Thamraksa, 2005).Metacognition regulation is a person's internal subjective response to metacognitive knowledge.Metacognition regulation is monitoring cognitive activity and ensuring that cognitive goals have been achieved (Berry, 1983).
Metacognition activities in this model are carried out through five activities.The first activity is to reflect on students' thinking processes in virtual experiments.The second activity is to look for other concrete examples from previous learning experiences.The third activity analyzes the learning experience's advantages and disadvantages.The fourth activity is generalizing and formulating rules regarding the learning experience.The last activity is to name the learning experience as a learning strategy (Zohar, 1999(Zohar, , 2004;;Zohar & Dori, 2012).The metacognition learning model has components of planning, monitoring, and evaluating (Dimaggi et al., 2014).Based on the description of the integration of metacognitive knowledge and the regulation of metacognition, the syntax of the virtual science experiment model is metacognitively integrated, which consists of six steps, namely, awareness, posing essential questions, planning, monitoring, evaluating, and reflecting.After the model was implemented, students' HOTS data were obtained for each material.The pre-test was conducted once, while the post-test was performed seven times.The descriptive analysis of HOTS data can be seen in Table 2.
The data is then analyzed for prerequisites (normality and homogeneity) to determine whether the influence of the model on HOTS can be analyzed using parametric statistics.The results of the normality test conducted as the requirement for multivariate analysis are presented in Table 3.
Table 1 shows that all data are normally distributed, confirmed by the value of sig > 0.05 (α) in the Shapiro-Wilk column.The normality of the data in all treatment groups has met the assumption to continue testing the hypothesis using the General Linear Model (Multivariate of Variance).The second assumption test is the homogeneity test (Table 4).Box's M value shows the homogenity of the HOTS scores achieved by the experimental and control groups.Rencher and Cristensen made the following observation that M-test is available in many software packages including SPSS and the rejection of null hypothesis of equal covariance matrices is not a serious problem when the number of observations is the same in each group (Sarma & Vardhan, 2018).A sig value of 0.000 or below 0.05 indicates that the data are not homogeneous or that the HOTS scores in each treatment group vary greatly.In an experimental study, this inhomogeneity is not a problem, because it is difficult to get the same variation in scores in two groups that are subjected to different treatments.In a quasi-experimental design, the error factor (subject, sample, treatment, etc.) has a significant effect on the change in pretest to posttest scores, resulting in broad variation in the scores achieved by research subjects.Additionally, it is difficult for all subjects in the experimental group to see identical improvement in scores.Due to the impossibility of obtaining the same variance in scores between two groups given to different treatments (Widhiarso, 2011), this inhomogeneity can be overlooked (Blanca et al., 2017).MANOVA is a robust test for data homogeneity disturbances when the sample size difference between the two treatment groups is between 7 and 15 participants (Ramsey, 2007).
The GLM test with Multivariate Analysis of Variance (MANOVA) revealed an interaction between time (pre-post-test) and group (experiment-control).The interaction demonstrated that the difference in scores between the two groups (experiment-control) was substantially different from pre-to post-test.The MD value for the experimental group was -17.505 with a significance value of 0.000 (0.05), indicating that the experimental group saw a significant rise in HOTS.The MD value in the control group was -11.069* with a significance value of 0.001, showing a statistically significant increase.The greatest rise occurred in the experimental group, with a mean difference of 17.505 between pretest and posttest.Additionally, the findings of the multivariate test in Table 5 were evaluated to ascertain the virtual practicum's impact on students' HOTS.
Within each level a combination of the other effects shown.These tests are based on the linearly independent pairwise comparisons among the estimated marginal means.
The significance values in Table 3 indicate that virtual practicums influence increasing students' HOTS.As mentioned by (Leech et al., 2013), the treatment's efficacy can be seen in Wilks' Lambda column.In the experimental group, Partial Eta Squared of 0.745 indicates that the Cakrawala Pendidikan: Jurnal Ilmiah Pendidikan, Vol. 42 No. 2, June 2023, pp.380-391 given treatment successfully increased students' HOTS by 74.5%, while the observed increase in HOTS in the control group was only 35.4%.The partial eta square value indicates the magnitude of an effect of a treatment (with a small effect being 0.01; a medium effect being 0.3, and a large effect being 0.5) (Bakker et al., 2019;Cohen, 1988;Mordkoff, 2019).The effect size discovered in this study is regarded as quite high, as it exceeds 50%.Thus, with big effect size and a 74.5% rise in HOTS, it can be concluded that virtual practicums have a considerable influence on students' HOTS.This rise is bigger than the increase in HOTS observed in students studying using other models.

Discussion
Individualized The virtual experiment syntax established in this study is the outcome of incorporating metacognitive theory into the stages of virtual science experiment.Students who participate actively in experiment, the will demonstrate an enhancement in both their individual and collaborative cognitive and metacognitive functions (Zarouk et al., 2020).Metacognition is comprised of knowledge and regulation components.Metacognitive knowledge consists of three components, namely awareness of knowledge/person variables, awareness of thinking/task variables, and awareness of thinking/strategy variables.Declarative, procedural, and conditional knowledge are all examples of metacognitive knowledge (Thamraksa, 2005).Metacognition regulation is the subjective internal response of an individual to metacognitive knowledge.This response is likewise directed at problem-solving strategic tasks.Metacognition regulation is the process of observing cognitive activity and ascertaining if cognitive objectives are met (Berry, 1983).Planning, monitoring, and assessing are the components of the metacognition learning model (Dimaggi et al., 2014).These three elements then become part of the virtual experiment phases in the planning, monitoring, and reflection portions, which correspond to the PjBL model.
The virtual experiment model used in this study places a premium on students' autonomy and flexibility of thought when it comes to problem-solving through work-based projects.Students are compelled to explore contextual learning problems.The problem-solving activities conducted in the classroom include mind mapping, contextual project work in the surrounding environment, virtual project work using Tracker, PhET, and sound meter software, as well as making video presentations.Each lesson began with an activity that helps students identify their strengths and weaknesses (awareness) in terms of science topics, and then move on to developing problem-solving methods (planning, monitoring, evaluating).
The implementation phase of the model also showed that the students' HOTS increased due to the use of this model.The increase in students' HOTS in the areas of logic, reasoning, and analysis was seen in their activity of assessing science problems that arise in their environment (Ichsan et al., 2019).The students were tasked with the responsibility of resolving these issues through the development of works.Each session contained a variety of works, including mind mapping, science experiments (contextual and virtual), and video presentations.The students had to study and understand the information using logic and reasoning to complete the project in the form of mind mapping.They were required to examine difficulties in completing science projects such as building simple automobiles, electrical circuits, simple compasses, simple pendulums, and solar system simulations.Besides that, the students were also accustomed to discussing issues with other students to develop their problem-solving skills.
The increase in students' HOTS in the evaluation aspect occurred because they were required to evaluate the achievement of their learning objectives, the suitability of the work produced with the problem, as well as the suitability of time and strategy with the expected results.The increase in the students' creation scores happened because of students becoming accustomed to making items that correspond to the learning objectives.The students were allowed to collaborate to convey their thoughts.At this step, opinions were gathered, clarified, logically reasoned, and expressed to others (Mumford & McIntosh, 2017;Sodikova, 2020).Each student's product was unique in terms of shape, substance, and outcome since they used the materials available in their immediate area while leveraging their prior knowledge.
At each step of learning, students' higher-order thinking skills (HOTS), specifically their ability to solve problems and make judgments, were also developed.For instance, when students used Tracker software to analyze the motion of an object (a wind-powered car), they ran into numerous complications.Despite the availability of the tutorials, some students were unable to complete the project by the deadline.This occurred because some pupils were technically incapable of using the software used in the analytical procedure.Students who had completed the project were then asked to mentor other students during virtual face-to-face encounters.This accomplishment arose because of students' willingness to experiment with various methods for solving issues, such as using MS Excel for mathematical operations and graph creation.Another example is the experiment with electricity using PhET Simulation, where numerous electrical circuits burnt throughout the project due to faulty wiring and resistance.Students who ventured to experiment with alternate steps were successful in determining the correct order.Problemsolving is a fundamental cognitive function in humans that interacts with other skills such as abstraction, decision making, analysis, and synthesis (Drigas & Mitsea, 2020).Students who develop strong problem-solving and judgment abilities will develop into self-assured, creative, and autonomous thinkers.The society produced by these individuals is easily capable of resolving everyday difficulties (Özreçberoğlu & Çağanağa, 2018).
The advantages of this virtual experiment model are that it is designed using quantifiable scientific procedures and involving experts, adaptable to normal or pandemic conditions by adjusting the learning activities, consists of learning activities that teach students to make decisions, take responsibility for their actions, and complete complex tasks or assist other friends with their assignments, is grounded in real-world problems and emphasizes project-based learning, which is critical for the development of outcome-based education curriculum, and is comprised of projects that foster the emergence of open-ended solutions, thereby preparing students to be problem solvers.However, the efficiency of this virtual experiment is also influenced by other aspects, such as self-regulation (Sulisworo et al., 2020) and student technology readiness (Indriyanti et al., 2020).Future research can adopt or modify this model to measure its impact on these two variables.The linkage of metacognition to self-regulation makes it seem like an association (Kristiani et al., 2015;Review, 2018;Rhodes, 2019;Shen & Liu, 2011).Meanwhile, the virtual lab used during practicum requires technological readiness (Firdaus et al., 2020;Halamka & Cerrato, 2021).Continuous application of the model allows students to be trained in HOTS continuously.When in real life or in industry, it can support their ability to solve contextual problems for better work performance.

CONCLUSION
This study was successful in generating a virtual science experiment design that incorporates metacognitive strategies using syntax, including increasing awareness, posing essential questions, planning, monitoring, evaluating, and reflecting.This model has a significant effect on students' HOTS, particularly in the areas of logic, reasoning, analysis, assessment, invention, problem-solving, and making judgments.These criteria are met when a support system in the form of instructional materials and student worksheets is in place.Collaboration between students and communication between students and the instructor as a social system, as well as the principle of reactions that occur when the lecturer provides reinforcement, all contribute to the experiment model's feasibility.According to the findings, lecturers should monitor students' knowledge and reflection of learning objectives to ensure that learning activities truly engage the domain of metacognition.Although it will take some time, this strategy appears to be worth considering by university science departments as a science experiment solution during the pandemic age.

Table 3 . Tests of normality
*.This is a lower bound of the true significance.a. Lilliefors Significance Correction

Table 4 .
Tests of homogeneity through box's test of equality of covariance matrices Box's Test of Equality of Covariance Matrices Tests the null hypothesis that the observed covariance matrices of the dependent variables are equal across groups.a. Design: Intercept + Model Within Subjects Design: time