Developing the flipped learning instrument in an ESL context: The experts’ perspective

Numerous studies have accepted the flipped learning approach as an approach in implementing technological-based classroom environments. This article aims to identify the required constructs in developing an instrument for flipped learning in an ESL environment. This study uses the Fuzzy Delphi method to collect and analyze the viewpoints of 18 experts from relevant fields. An online questionnaire was developed to gather the experts‟ agreement towards seven constructs: flexible environments, the shift in learning culture, intentional content, progressive networking activities, professional educators, engaging & effective learning experiences


INTRODUCTION
Education of the 21st century is radically changing than any preceding decades before. Technology has become a determining factor in helping education and lessons being meaningful and successful to millennial students (Akçayır & Akçayır, 2018;Azman & Dollsaid, 2018;Tsay et al., 2018). These students are more comfortable engaged with technology and learning with it as it gave rise and prominence to tech-based educational approaches such as e-learning, blended learning, and flipped learning (Embi, 2014;Hamdan et al., 2013;Kenna, 2014).
Traditional didactic approaches are becoming more and more inefficient in dealing with 21st-century students (Kenna, 2014). Students today are more sensitive to their divergent abilities and needs in classrooms, which is essential in delivering a meaningful lesson. Teachers must be able to address these divergences in the classroom for having an effective lesson delivered (Lage et al., 2000). This is especially true for tertiary-level education, as global connectivity has seen a rapid rise since the development of digital technology in the last ten years (Enfield, 2013). The impact of digital technology is that students of this generation are more comfortable interacting and digesting information through many online interactive platforms.

The Flipped Learning Approach
The fact that the new generations prefer digital platforms over conventional ones spurred educators to use the digital technology platforms as an effective medium of teaching students and making learning a meaningful experience through different forms of interactions with the lesson content for different learning styles (Tsay et al., 2018). The flipped learning approach is a teaching approach spurred by digital technology in the classroom. The flexible and independent disposition of the approach jives well with the use of technology in education. Some experts believe that the approach allows for a cornucopia of pedagogical approaches to be implemented in a flip approach classroom, resulting in a flexible range of approaches that is Taylor-suited to each student"s learning styles (Baepler et al., 2014). Juhary and Amir (2018) debated that many past studies have proven the ability of the flip learning approach to empower students to be self-dependent learners. Furthermore, the shift of responsibility of learning that befalls on the learner themselves proliferates the usage of learner-centered approaches that, in turn, allows for the individual students of different learning styles and abilities to learn and develop at their own pace (Raihanah, in Ministry of Education Malaysia, 2015).
Several previous studies have shown a positive effect of the flipped learning approach involving students (Chen et al., 2014;Davies et al., 2013;McLaughlin et al., 2013). This proves that the flipped learning approach in the classroom is an alternative in learning approaches and strategies. However, most research on flipped learning is conducted in mathematics and engineering subjects (Baepler et al., 2014;Chen et al., 2014;McLaughlin et al., 2013). Besides, several studies on flipped learning in ESL are only related to the limited readiness of students and lecturers in using the flipped learning approach (Embi, 2014;Osman et al., 2014). Existing studies on model development and guidelines for applying the flipped learning approach are still relatively lacking. Existing studies also focus more on the general student population at universities (Baepler et al., 2014;Embi, 2014), so the flipped learning approach cannot be extended to a broader group. There are gaps in the literature related to research on the flipped learning approach in language learning, especially in English language learning in the ESL program. Thus, this gap in the literature needs to be addressed to examine whether there is consistency in the results and effects of flipped learning in ESL subjects and for engineering and mathematics subjects. The flipped learning approach"s effectiveness needs to be studied by developing a model framework in the ESL program at universities.
This study stands on the premise that besides the recorded positive advancement and development, the flipped learning approach has on teaching and learning of the 21st century. There exists minimal proof of a set perimeter to guide the use of the approach in an educational environment effectively (Baepler et al., 2014;Bishop & Verleger, 2013;O"Flaherty & Phillips, 2015). Thus, a conscious effort to establish an instrument for flipped learning has become a primary concern to implement the approach effectively. Experts" perspective of constructs recommended for developing an instrument for flipped learning is invaluable as their professional experience and knowledge on the subject matter should be pivotal in determining that such development is on the right path. The study sees the experts" perspective on the proposed constructs for developing an instrument to implement the flipped learning approach in an ESL environment. Thus, seven constructs were identified from Hamdan et al. (2013) and Chen et al. (2014), that were mapped out into sixty-eight items and used in the form of a questionnaire posted in the form of Google docs and distributed to twenty-two experts of educational technology or ESL and educational technology. The study received eighteen responses, analyzed by the Fuzzy Delphi method. This research aims to see the experts" view of the proposed constructs and quantify these views in the form of Fuzzy Delphi analysis.

RESEARCH METHOD
This study uses a design and developmental research (DDR) approach to develop and verify a flipped learning framework in an ESL context. There are two phases of the study. The first phase focuses on the design of construct for flipped learning framework utilizing literature review concerning the flipped learning approach. The second phase is on the development of the flipped learning construct, starting with the Fuzzy Delphi method.
The samples in this study were selected using the purposive sampling technique. Purposive sampling refers to a sampling procedure in which subjects that have ascertained specific characteristics needed for the research are selected as respondents in the study (Creswell, 2009), and it does not require underlying theories or a certain number of informants (Patton, 2002;Tongco, 2007) as its foundation. It is a "deliberate choice of an informant based on the qualities the informant possesses" (Tongco, 2007, p. 147) and is used in collecting the quantitative (Fuzzy Delphi and survey) data of the study.
There are two groups of samples used in this study. The first group involves a group of 18 experts in the education technology learning field. The experts" responses are gathered during the Delphi technique to obtain the second objective of the study.

Instruments
In the first phase, the researchers used the input or process to design the flipped learning questionnaire for the experts. The input from the literature provided a list of construct, dimensions, and items concerning ESL flipped learning at Universiti Kebangsaan Malaysia. Then, the list was converted into several statements to form an ESL flipped learning questionnaire for the experts to review.
The Fuzzy Delphi Method (FDM) was used to gather the experts" agreement on the dimensions and indicators. FDM aims to solve the problem of traditional Delphi method (Hidayat & Lawahid, 2020;Ishikawa et al., 1993). The method is based on group thinking of qualified experts to ensure the validity of collected data. In FDM, the experts were required to indicate the extent of their agreement with the statements. Also, the experts were encouraged to introduce or recommend any new dimensions or indicators, revise or make an adjustment to the existing statements in the list. Before the actual FDM, the questionnaire went through several face validation processes by the researchers" supervisors and experts identified.

Procedure
The data collection for this study consists of three phases. Each phase is elaborated as follows.

Phase I: Flipped Learning Framework Design in Context
The first phase involved a review of related past studies. The review was done to establish the constructs and items needed for the development of the questionnaire to develop the Flipped learning framework. The researchers conducted this by mapping out the literature to the constructs proposed in the study by Hamdan et al. (2013) and Chen et al. (2014). Once the construct, dimensions, and items were established, the questionnaire was developed and given to two experts to review its content and face validity. After discussions and reviews of the questionnaire, the questionnaire was ready for the next step.

Phase II: Flipped Learning Framework Development
The next step is to identify experts or participants for the Fuzzy Delphi Method (FDM). After the experts expressed their agreement to participate, emails with appointment letters signed by the researchers' supervisor were sent to the experts. The experts were then directed to a Google form website address to answer the questionnaire and give their opinions on the questionnaire. The findings from FDM were used to help in enhancing and developing the flipped learning questionnaire and framework.

Data Analysis
The first quantitative data of the study were collated from the experts in the second phase using the Fuzzy Delphi Method (FDM). FDM is conducted using the questionnaire instrument, which has a five-point Likert scale of importance ranging from "Strongly Important" (5) to "Strongly Unimportant" (1). FDM is chosen for this analysis as the method has been proven to produce statistically valid constructs, dimensions, and items for many previous kinds of research (Bouzon et al., 2016). Furthermore, the Defuzzification α-cut analysis and the (d) threshold value in accepting and rejecting items and constructs in developing frameworks and models were well established and valid (Hidayat, 2018). In terms of showing a consensus, FDM analysis can demonstrate this effectively (Sanchez-Lezama et al., 2014). This first quantitative analysis establishes the constructs, dimensions, and items of the study.

What Constructs Should Make Up a Framework for Assessing Flipped Learning Approach Effectiveness in an ESL Context, Based on Literature, in the Design Phase?
The overall findings from the literature analysis done for Phase I points to the importance of the seven constructs (flexible learning environment, shift in learning culture, intentional content, professional educators, progressive networking activities, engaging and effective learning experiences, diversified seamless learning platforms) in discussing what is important for the flipped learning approach implementation by researchers investigating the flipped learning approach in the educational environments. The constructs are essential but not apparent and are usually discussed directly or indirectly, either in the literature discussions or in the conclusions made. This may be due to the approach novelty, as many academics are still investigating what important factors contribute to the effective implementation of the flipped learning approach in educational environments. However, the framework can be divided into two main themes: the educator"s element and the student"s element, which are further divulged in this section.
The researchers have identified and organized several research papers that discuss one or several constructs as important elements to be considered in implementing and administering the flipped learning approach. To note, the importance of the educator"s elements that make up an integral part of the framework in ensuring an effective flipped learning approach classes have been discussed at length (Bishop & Verleger, 2013;Chen et al., 2014;Coufal, 2014;Embi, 2014;Hamdan et al., 2013;Hao, 2016;O"Flaherty & Phillips, 2015). The approach of providing flexible learning modes physical learning space in encouraging students to be engaged in a meaningful lesson cannot be undermined (Balan et al., 2015;Bergmann & Sams, 2012;Yemma, 2015). Hao (2016) stated that the implementation of the flipped learning approach is drawing attention from educators and students alike due to the shift in the learning culture where the main onus to learn is on the students making meaning of the lessons on a personal level, with independence and peer learning. The usage of content developed and customized according to the needs of the flipped environment to ensure meaningful lessons cannot be underestimated (Bergmann & Sams, 2012;Wiginton, 2013). Furthermore, the presence of educators who are aware of the job scope and responsibilities of a flipped learning educator is imperative in determining the success of the implementation (Nederveld & Berge, 2015;O"Flaherty & Phillips, 2015;Yemma, 2015).
The literature evidence pointing towards the importance of the student"s element in ensuring a successful flipped learning curriculum has been the focal point in academic discussions (Chen et al., 2014;Embi et al., 2014). The usage of activities that encapsulate networking through the use of technology is integral in ensuring effective lessons that propagate self-discovery learning and peer learning for students of the 21st century (Aw-Yong et al., 2013;Nederveld & Berge, 2015;O"Flaherty & Phillips, 2015;Yemma, 2015). Many research papers have concluded from their analysis that careful planning and thought should be invested in designing engaging and effective learning experiences as it is an integral part of determining the success or failure of the approach in classroom settings (Bergmann & Sams, 2012;Bishop & Verleger, 2013;Soltanpour & Valizadech in Bodomo, 2016;Yemma, 2015). Moreover, the idea of using multiple platforms online for learning is welcomed and seen as an effective and engaging medium to inculcate ideas of discovery learning, peer learning, ubiquitous learning, and many other important facets of 21st-century learning or learning characteristics in the Industrial Revolution 4.0 era. These traits are important characters needed for the students to become successful in the Industrial Revolution 4.0 era (Soltanpour & Valizadech in Bodomo, 2016;Coufal, 2014;Kafi & Motallebzadeh, 2014;Wiginton, 2013;Yemma, 2015).
All these research works have, one way or another, mentioned the importance of the factors proposed as the constructs of the framework suggested. What is missing from the literature so far are investigations that look at the factors together and deliberate the influence and effect these factors have comprehensively on the teaching and learning processes in the Flipped learning approach or environment. Hence, the study investigates the interactions of the factors proposed and their influence in an ESL context in Malaysia. All these researches point to the importance of the constructs proposed by this study in developing and ensuring that any implementation of the Flipped learning approach must have a glance of the factors in molding the lessons or curriculum with Flipped learning in their fore. The researchers also point to the need for a framework to implement the flipped learning approach. Currently, there are only mentions of the important factors to consider. However, the presence of a proper and comprehensive perimeter or framework that guides the implementation of the approach ensures an effective and meaningful learning experience is currently lacking from the literature.

How Effectively and Accurately Can These Constructs Be Determined to Measure the Flipped Learning Framework in an ESL Context in the Development Phase?
Each construct is mapped out to items representing it and put forth in the questionnaire form to the experts identified. The constructs and items are deduced from the literature review and mapped accordingly to form the basis of this study. The seven constructs identified are: (1) flexible environments (FE), (2) shift in learning culture (LC), (3) intentional content (IC), (4) professional educators (PE), (5) progressive networking activities (NA), (6) engaging and effective learning experiences (LE), and (7) diversified seamless learning platforms (LP).
These seven constructs concern the teaching and learning process and the student"s experience of the technology-based learning approach. These constructs will be the foundation of the items built and analyzed with the Fuzzy Delphi method. For deliberation and discussion of the findings, the (d) threshold value of the constructs and items and the percentage of experts" agreement are discussed in this section. The bench mark benchmark acceptance of a construct or the items will be ≥ 0.2 for the (d) threshold value and 75% for the percentage of experts" agreement. The results of the analysis are as follows.  7), and 88.88% (FE 8). Thus, these items have met the benchmark value of the (d) threshold and percentages of experts" agreement mentioned earlier, and are accepted by the experts. Item FE 5 of the construct has been rejected as the (d) threshold value (0.212). The experts" agreement percentage (33.33%) met the benchmark value. Overall, the "Flexible Environment" construct has a (d) threshold value of 0.180 and overall percentage of experts" agreement of 86.10%, which leads to the item being accepted by the experts as congruent and important for this study.   ). The experts have approved these items as important for this particular construct. Item LC7, however, is rejected as the (d) threshold value, and is at 0.142. The experts" percentage of agreement is at 66.67%. Therefore, it leads to the rejection of the item, because it fails to achieve the benchmark values of the analysis.

Construct of Shift in Learning Culture (LC)
In addition, the overall (d) threshold value and the percentage of experts" agreement of the construct "Shift in Learning Culture" as a whole are at 0.149 and 89.50%, respectively. This results in the construct being acknowledged as compatible and important for developing the instrument for gauging the flipped learning efficiency.  The "Intentional Content" construct has ten items investigated in this study, and the Fuzzy-Delphi analysis of each item is shown in Table 3. The (d) threshold values of each item accepted by the experts are: 0.167 (IC 2), 0.158 (IC 3), 0.151 (IC 4), 0.185 (IC 5), 0.147 (IC 6), 0.153 (IC 7), 0.187 (IC 8), 0.200 (IC 9), and 0.181 (IC 10), while the experts" agreement percentages of the accepted items are 100% (IC 2), 94.44% (IC 3), 100% (IC 4), 88.89% (IC 5), 94.44% (IC 6), 100% (IC 7), 88.89% (IC 8), 83.33% (IC 9), and 88.27% (IC 10). The rejected item of IC 1 has a (d) threshold value of 0.244 and experts" percentage of agreement of 44.44. Therefore, the item is rejected as being representative of the construct in question. The overall (d) threshold value and the experts" agreement percentage are 0.175 and 88.27%, which lead to the construct acceptance by the experts as a part of the development of a flipped learning instrument.  The overall (d) threshold value is 0.162, and the percentage of experts" agreement of the construct is at 90.30%. This shows a strong agreement by the experts in accepting the aforementioned construct as an essential part of the instrument development for analyzing flipped learning.   , and 0.112 (PE 8). Their respective percentages of experts" agreement are 38.9% (PE 2), 55.6% (PE 7), and 66.7%. (PE 8). As such, only three items out of fifteen for this construct are rejected by the experts. Conclusively, the construct's overall construct (d) threshold value is 0.157, and the percentage of experts" agreement for the construct is 87.8%. This shows the acceptance of the construct by the experts for the development of the flip learning instrument.   Item 7 of the construct (LE 7) is rejected because the (d) threshold value is 0.24, more than the 0.2 benchmark value, and 16.7% of the percentage of experts" agreement, which is below the 75% benchmark. The overall construct (d) threshold value stands at 0.189; meanwhile, the overall percentage of experts" agreement is at 83.3 %. Hence, the experts accept the construct of Engaging and Effective Learning Experiences as important in developing a flip learning instrument. Construct of Diversified Seamless Learning Platforms (DP) Table 7 shows the Fuzzy-Delphi analysis of "Diversified Seamless Platform" construct. There are no rejected items, which entails that all seven items are accepted and viewed important by the experts. The (d) threshold value of each item is: 0.193 (DP 1), 0.163 (DP 2), 0.152 (DP 3), 0.147 (DP 4), 0.147 (DP 5), 0.190 (DP 6), and 0.200 (DP 7). Meanwhile, the percentages of experts" agreement of the items are: 83.3% (DP 1), 94.4% (DP 2), 100.0% (DP 3), 100.0% (DP 4), 100.0% (DP 5), 88.9% (DP 6), and 83.3% (DP 7).

Construct of Engaging and Effective Learning Experiences (LE)
The overall (d) threshold value for this construct is 0.171, and the percentage of experts" agreement is 92.9%. This infers the acceptance of the experts and the importance of the construct in the development of the flip learning instrument.
The results of this study point to the acceptance and acknowledgment of the experts of the constructs proposed originally by Chen et al. (2014) and Hamdan et al. (2013) as being important seven constructs to be considered for the development of a flipped learning instrument in an ESL context. These constructs put the idea of the technology, the pedagogies, and the people, educators, and students experience in a continuum of teaching and learning spectrum. These constructs interact to create an instrument that serves as a parameter-gauger for the effective execution of the flipped learning approach in the ESL context.
The study consists of seven constructs that make the basis of the sixty-eight items used to solicit the experts" perspectives on these constructs. Out of the sixty-eight items asked, the experts rejected seven items in relation to the seven constructs. Overall, the experts agreed that the proposed constructs can be used to develop an instrument to check the efficiency of flipped learning in the ESL context. Hence, an instrument to gauge the effectiveness of the flipped learning approach in an ESL context is established as a result of the research.

CONCLUSION
Conclusively, the study identified the required constructs in developing an instrument for flipped learning in an ESL environment. Establishing the constructs can trail blaze investigations that lead to developing a framework or model to gatekeep the effective implementation of the flipped learning approach in general or even specific contexts.
The implication of the study can be seen in multiple facets. The first facet is for the policymakers and educational governing bodies. The identification of the constructs means the relevant bodies can now rely on these constructs in guiding and determining parameters needed for effective implementation of the flipped learning approach, especially in the ESL context. Furthermore, curriculum developers and teachers can use these constructs and their items to ensure their flipped learning approach classes are seen as meaningful and relevant by the students in developing their knowledge in a technology-supported environment. Secondly, these constructs can be used as the basis for further research to the development of established frameworks and models for the effective and meaningful flipped learning approach lessons.