USE OF THE DELPHY TECHNIQUE : A CASE FOR THE DEVELOPMENT OF ECOTOURISM IN WESTERN LOMBOK Soenarto Universitas

Delphi technique was developed in 1950 by researchers at the Rand Corporation led by Norman Delkey and Olaf Helmer and has since been used in hundreds of businesses forecasting in the public and private sectors. Delphi technique is a judgmental forecasting procedure for obtaining, exchanging, and developing an informed opinion about future events. Therefore, the Delphi technique is a systematic way to get a consensus of opinion among the experts who have related interests through a panel discussion. The objective of most Delphi is the reliable and creative exploration of ideas or the production of suitable information for decision making. The key features of the Delphi technique, namely: (1) systematic, (2) questionnaire, (3) expert judgment, (4) iteration process, and (5) feedback. Baseline characteristics of the Delphi technique (conventional) there are five, namely: (1) anonymity, (2) iteration, (3) controlled feedback, (4) statistical group response, and (5) expert consensus. While the characteristics of a policy Delphi also five, namely: (1) selective anonymity, (2) informed multiple advocacies, (3) polarized statistical response, (4) structured conflict, and (5) computer conferencing. Steps in the application of policy Delphi there are seven, namely: (1) issue specification, (2) selection advocates, (3) questionnaire design, (4) analysis of firstround results, (5) development of subsequent questionnaires, (6) organization of group meetings, and (7) preparation of final report.


History of the Delphi Technique
Delphi is actually the name of a valley in ancient Greece guarded by the dragon Apollo.Dragon Apollo is known to have the ability to see the future.However, William N. Dunn has another version.Dunn (1994) states that the Delphi technique (whose name was taken from the temple of Apollo I n Delphi, where the Greek shaman vowed to see the future) was developed in 1948 by researchers at the Rand Corporation and has since been used in hundreds of forecasting businesses in the public sector and private.
Initially, this technique was applied to military strategy problems, but gradually it could be applied to other contexts: education, technology, marketing, transportation, mass media, medicine/pharmacy, information processing, research and development, space exploration, housing, budget and quality of life.In the beginning, this technique emphasized the use of experts to study predictions based on data empirical, but then began to apply to value forecasting problems in 1960.
The Delphi technique has been used by analysts in various countries, from the United States, Canada, Britain to Japan and the Soviet Union.
The Delphi technique is finally widely used and accepted to achieve convergence of opinions about the real-world knowledge requested by experts in a particular topic.Based on the reason that, "two heads are better than one, or ...... n heads are better than one", the Delphi technique is designed as a group communication process aimed at conducting detailed checks and discussions on specific issues aimed at goal setting, policy inquiry, or predict the occurrence of future events.The general survey tries to identify "what is," while the Delphi technique attempts to overcome "what could/should be".

The purpose of the Delphi Technique
In the Delphi technique, there were no group discussions between experts.This is to minimize the possibility of a direct confrontation that might occur between experts.Thus, a consensus can be reached based on relevant information.The purpose of the Delphi technique is to explore the creative and reliable ideas or produce information that is suitable for decision making.

Definition of Delphi Technical
Then the question is what is the Delphi technique/method?Dunn (1994) states that "Delphi technique is a judgmental forecasting procedure for obtaining, exchanging, and developing informed opinion about future events".The Delphi technique is an opinion forecasting procedure to obtain, exchange, and make opinions about future events.
The Delphi technique is a systematic method of gathering opinions from a group of experts through a series of questionnaires, where there is a feedback mechanism through 'round questions held while maintaining the anonymity of respondents' responses (experts) (Foley, 1972).The Delphi method is a structured communication technique, originally developed as an interactive forecasting method that depends on a number of experts Linstone & Turoff (1975) states that Delphi may be characterized as a method for structuring a group communication process so that it is effective in allowing a group of individuals, as a whole, to deal with a complex problem".
The Delphi technique is a way to organize ideas among experts to improve a problem in the future (Weaver, 1971).With Delphi techniques, various opinions about a phenomenon among people who have related interests can be collected, sought similarity points, and summarized so that it is a joint consensus to determine a program plan (Soenarto, 1988).Therefore, the Delphi technique is a systematic way to get opinion agreement among experts who have related interests through panel discussions.Stufflebeam & Shinkfield (1985) suggested a number of agreements that could be achieved through the Delphi technique, among others: (1) determining the objectives of the institution; (2) direction and type of questions in the needs analysis using the need assessment, and (3) basic requirements that must be met to achieve the objectives.
In the standard version, experts answer the questionnaire in 2 or more rounds.After each round, the facilitator provides a summary of expert forecasting from the previous round and the reasons they gave for their assessment.Thus, experts are advised to revise the previous 94 − Jurnal Pendidikan Vokasi Volume 9, No 1, February 2019 answers based on the answers previously compiled.In this process, the answers will converge and finally, the group will find the correct 'answer'.In the end, this process will stop after the criteria that have been agreed to have been reached (the number of rounds, achieving consensus, and stability of results).
Other versions, such as Delphi Policy, have been designed for normative and explorative use, especially in the area of social policy and public health.In Europe, web-based experiments have used Delphi as a communication technique for interactive decision making and e-democracy.
Delphi is based on the principle that forecasting (or decision) of structured groups/ individual is more accurate than unstructured groups.This is indicated by the term "collective intelligence".This technique can also be adapted for the use of face-to-face meetings (therefore it is called a mini Delphi / ETE).Delphi has been widely used for business forecasting and has certain advantages over other approaches.

Characteristics of Delphi Technical
The application of the Delphi technique was initially driven by concern for the ineffectiveness of committee work, expert panels, and other group processes.This technique is designed to avoid various sources of communication distortion found in these groups, such as domination of groups by one or several people; pressure to follow core group opinions; differences in personality and interpersonal conflict; and difficulty in opposing people who are openly authorized.To avoid these problems, according to Dunn (1994), the initial application of the Delphi technique emphasizes five basic principles, namely: (1) Anonymity, all experts or knowledgeable people give responsiveness separately and anonymity (not knowing each other between them) is really maintained; (2) Iteration, assessment of each individual is collected and communicated back to all experts who participated commenting in two rounds or more, so that the social learning process takes place and it is possible to change the initial assessment; (3) Controlled feedback, Communication of the assessment is carried out in the form of a summary of the answers to the questionnaire, (4) Statistical group response.The summary of each person's responses is conveyed in the form of a measure of central tendency (usually median), dispersion (interquartile), and frequency distribution (histogram and frequency polygon); (5) Expert consensus, the main objective, with a few exceptions, is to create conditions in which the consensus among the experts is the final and most important result.
Meanwhile, according to Garrod (2007), the characteristics of the Delphi technique are: (1) qualitative research but with quantitative elements; (2) depends on the judgment of a number of experts; (3) An iterative process, which occurs during several "rounds"; (4) Positive points: (a) flexible; (b) suitable for getting issues/insights that have not yet appeared (below the surface); (c) suitable in finding difficult questions/experts; (d) more structured than conventional interviews; (5) but it is also criticized for: (a) often called "discredit", (b) allegedly anti-democratic/anti-participatory (c) lately, executions have often been careless, which has somewhat damaged the reputation of this technique.

Steps of Implementing Delphi Techniques
According to Jakaria (2009), the steps in implementing the Delphi method are seven (7), namely: First, problem identification and specification.The researcher identifies the issues and problems that are developing in his environment (his field), the underlying problems, or the problems faced which must immediately need resolution.
Second, personal identification and selection.Based on the areas of prolems and issues that have been identified, the researcher determines and selects experts who pay attention, and is interested in the field, which enables the achievement of goals.The number of respondents at least matches the subproblems, level of expertise, and/or authority.
Third, questionnaire design.The researcher composes the items of the instrument based on observed variables or problems to be solved.The instrument items should fulfill content validity.The question is in the openended question, except if the problem is specific.
Fourth, sending questionnaire and analy-sis of respondents for first round.The researcher sends the questionnaire in the first round to the respondent, then reviews the instrument and analyzes the answers to the instruments that have been returned.The analysis is done by grouping similar answers.Based on the result of the analysis, researchers revised the instrument.
Fifth, Development of subsequent questionnaires.The questionnaire resulting from the review in the first round was developed and improved, followed by the second and third rounds.Each revision results are sent back to the respondent.If you experience difficulties and doubts in summarizing, the researcher can ask for clarification from the respondent.In the Delphi technique, it is usually used up to 3-5 rounds, depending on the breadth and complexity of the problem until the consensus is reached.
Sixth, organization of group meetings.The researcher invited the respondents to hold a panel discussion, for clarification of the answers given.This is where argumentation and debate can occur to reach consensus in providing answers to the design of a product or research instrument.By face to face contact, researchers can ask in detail about the response that has been given.The final decision about the results of the polls is said to be good if a minimum of 70% consensus is reached.
Seventh, Prepare final report.Researchers need to make a report about the preparation, process, and results achieved in the Delphi technique.The results of the Delphi technique need to be tested in the field with respondents who will use a much larger number of models or products.

Delphi Technique Anvantage
According to Garrod (2007), the Delphi technique has a number of advantages, namely: (1) this technique is very flexible to be applied in various situations and various complicated problems, where often there is no suitable analytical method to apply; (2) the iterative procedure allows experts to rethink their assessment based on feedback from other expert colleagues; (3) this process also gives more time to participants to rethink their ideas before giving an assessment, this will certainly provide a better quality response; (4) an anonymity approach allows experts to be more free to argue, without pressure from any party; (5) the possibility of individual influence is also automatically eliminated; (6) the existence of "transfer issues" that come out of the main focus of the discussion can be controlled by the project manager (the researcher himself); ( 7) this process will produce notes from group thoughts that can be reviewed when needed; (8) this method can be used to evaluate the distribution of opinions from experts or consensus points (things agreed upon).

Delphi Technology Weaknesses
Besides having a number of advantages, the Delphi technique also has a number of weaknesses.According to Garrod (2007), the weaknesses of the Delphi technique are: (1) the Delphi technique can be very sensitive to the following: (a) the level of expertise of the panelists, (b) panelist composition, (c) clarity of questions, (d) the way project managers (researchers) report outliers, (e) administration of the questionnaire; (2) this technique assumes experts allow their judgment to be revised by the opinions of others; (3) the expert panel is vulnerable to attrition (weakening process) due to: (a) saturation with research subjects / topics, (b) dissatisfaction with the process, (c) lack of time to complete the Questionnaire; (4) some Delphi practitioners / users use the influence of money / persuasion morally to convince the panelists to "keep on track" in this case, but this can lead to bias related to the results of the study; (5) there is a possibility in the form of 'pseudo consensus', where the panelists agree and adjust with the assesment of the group; (6) this technique often requires a quality amount of time to complete the questionnaire and often takes a lot of time from the researcher (3-5 rounds).

Guide for Implementation of Delphi Engineering
Then the question is how to do the Delphi technique?According to Garrod (2007), there are ten (10) guidelines in using the Delphi technique, namely: (1) the Delphi technique should not be seen as the main tool in investigations but rather as a support / expansion tool for studies that have been developed and a more reliable method of investigation; (2) topics must be appropriate, there should be no biased answers to the questions asked; (3) questions must be tested first to avoid ambiguity; (4) researchers must be true experts in their fields; (5) panelists should consist of good combinations between disciplines and fields of expertise; (6) there should be sufficient time allocation for experts to think deeply about the questions asked; (7) after the next round begins, experts who are late in completing the previous round must immediately work on the next round; (8) the weakening of the panel can be minimized by selecting experts who have a strong interest in the output of the project / research; (9) it is recommended to use financial compensation and moral invitation as a tool to ensure that experts remain committed to the project/research; (10) experts must be sure that the Delphi technique is valid for dealing with existing problems.

Delphi Policy
Principles such as anonymity, iteration, controlled responses, statistical group answers, and expert consensus are characteristics of the conventional Delphi technique (Syafruddin, 2010).Conventional Delphi, which dominated this field until the late 1960s, must be distinguished from Delphi policy.Delphi policy is a constructive response to the limitations of conventional Delphi and is an attempt to create a new procedure that fits with the complexity of policy issues (Dunn, 1994).In the statement of the main architect: Delphi was originally created and practiced to address technical topics and seek consensus among homogeneous expert groups.But, Delphi policy, seeks to produce the most likely opposition views on the potential resolution of a policy issue ... a policy issue is something that no expert has about it, but only knowledgeable advisors and people who are used to reference.
While the Delphi policy is based on two of the conventional Delphi principles (i.e.iterations and controlled responses), this technique also introduces several new principles:

Selective anonymity.
Participants in a Delphi policy remain anonymous only during the initial round of forecasting efforts.After rival arguments about alternative policies emerge, participants are asked to openly debate their views.

Double Advocacy of Knowledgeable People (informed multiple advocacy).
The process for selecting participants is based on criteria of interest and level of knowledge, not merely "expertise".In forming a Delphi group, the investigator only tried to select representatives from a group of know-Use of the Delphy Technique:a Case for the Development … Soenarto, Rahmawati 97 ledgeable advocates who might be present in certain situations.

A Statistically Polarized Response
In summarizing the opinions or opinions of experts, it are used the ways that emphasize disagreement and conflict.While conventional measures can also be used (median, range, standard deviation), the policy Delphi adds to it with various measures of polarization between individuals and groups.

Structured Conflict
Starting from the assumption that conflict is something that is natural in policy issues, various efforts have been made to use disagreements and conflicts to creatively explore their alternatives and consequences.In addition, efforts are made to express and explicitly assumptions and arguments that underline conflicting opinions.However, the results of a Delphi policy are not completely open, so that consensus and continuing conflict can be something that appears in the process itself.

Computer Conferencing
If possible, computer consultation is used to design a continuous anonymous process between individuals who are physically separate.Computer conferences remove the need for several separate Delphi rounds.

Steps of Application of Delphi Policy
A Delphi policy can be done in various ways, depending on the context and ability of people who use this technique.Because Delphi policy is a study, it involves a large number of technical problems, including sampling, questionnaire design, reliability and validity, and analysis and interpretation of data.Although these problems are beyond the scope of this chapter, it is important to obtain a thorough understanding of the process of implementing a Delphi policy.According to Dunn (1994), a Delphi policy can be described as a series of interrelated stages.
In this case, the analyst must determine what issues advocates must comment on.For example, if the area of concern is a policy of abuse of medicine, one of the issues is "Per-sonal use of marijuana must or should not be permitted".One of the main problems of this stage is deciding what issues should be gathered from the participants, and what the analyst must produce.If the analyst is familiar with the area of concern, it is possible to list issues before the first Delphi round is held.These issues can be included in the first questionnaire, although respondents must be free to add or reduce issues.
Here key actors in an issue are selected.However, to isolate a group of advocates who represent opposing views, it is necessary to use sampling procedures in a frank manner.One way to do this is to apply the "snowball" sampling method.Here the analyst begins to identify an advocate, usually a person who is known to be influential in the issue area and asks him to give the names of two other people who agree and disagree with his views.The two people were also asked to do the same, which eventually got two or more people who were very agreeable and very disagreeable, and so on (hence the term "snowball").Advocates should be as different as possible, not only in their position, but also in their relative influence, formal authority, and group affiliation.The size of the sample can range from 10 to 30 people, although this depends on the nature of the issue itself.The more complex the issue is, and the more heterogeneous the participant is, the greater the sample needed to represent the range of advocates.
Step 3: Make a questionnaire.
Because a Delphi policy takes place in several rounds, the analyst must decide which items will be submitted in the questionnaire to be used in the first round and later rounds.However, the second round questionnaire can only be made after the results of the first round are analyzed; the third round questionnaire must wait for the results of the second round, and so on.Therefore only the first round of questionnaires can be made in full.Although the first round questionnaire may be less structured (with many open questions), this questionnaire can also be relatively structured as long as the analyst has a good idea of the main issue.The first round of questionnaires can include several types of questions: (1) forecasting questions that ask respondents to give subjective estimates of the probability of occurrence of an event, (2) questions about issues that ask respondents to sort issues on the importance of the issue, (3) questions about the purpose of asking assessment of the desirability and / or feasibility of the effort to pursue a goal, and (4) questions about choices that require respondents to identify alternative actions that might help achieve goals and objectives.
Several types of scales are available to measure the responses of each item.One pro-cedure is to use a different scale with different types of items.For example, a certainty scale can be used primarily for forecasting items; interest points for issue items; a scale of desire and feasibility for the purpose of the item; and a combination of these scales for selected items.The best way to show what is involved is to describe the way the items and scales are used in a Delphi policy questionnaire.This has been done in Table 1.
Mn = 2,8 Mn = 1,5 Mn = 1,5 distance = 3,0 distance = 3,0 distance = 1,0 distance = 1,0 Note: Median (Md) on a set of scores is the value of the score in the center when the scores are arranged on the basis of magnitude.If the score is even (as above), the median is the score in the middle between the two middle scores.Median is usually used to replace mean (Mn) when we do not know whether intervals between sizes (for example, intervals between 1 and 2 and 3 and 4) are the same distance.
Note that the scale in Table 1 does not allow for a neutral answer, although the answer "No Opinion" is possible for each item.This limitation of neural responses is designed to deal with conflict and disagreement, an important goal of Delphi policy.An important part of making questionnaires is to do pretest among a sample of advice and determine the reliability of the response.

Step 4: Analysis of first-round results
When the questionnaire is returned to the analyst after the first round, the analyst tries to determine the initial position of advocates about forecasting issues, objectives, and choices specifically, some items that are believed to be desirable or important are also believed to be inappropriate, and vice versa.Because there will be differing opinions among the aforementioned advocate, it is important to use a measure that summarizes so that it does not only reveal the main tendency in a number of responses but also expresses the extent of differences or polarization.Measures that summarize this are used not only to eliminate unimportant, unwanted, inappropriate and/or uncertain questions but also to function in the second round of questionnaires as a means to synchronize to participants the results of the first round.
The calculation and presentation of summarization measures from the main trends, dispersions, and polarization should be displayed in graphical form, to illustrate, it is assumed that ten advocates in the first round of Delphi policy hypothetics gave different opinions about the need and feasibility of two goals of drug control: to reduce the number of drug sales and to increase public attention to the difference between responsible and irresponsible drug use.Let us imagine that the response to this is shown in Table 2.
Note that some respondents (advocates 2, 8 and 9) believe that the goal of reducing illicit drug sales is not expected but it is possible to do or very feasible, while others (advocates 1, 5, 7 and 10) believe that this goal is very desirable but not very worthy.When we compare these inconsistencies between expectations and feasibility and responses to goal 2 (public attention), we find smaller inconsistencies in this second score.All of this states that the response to goal I, while lower in needs and feasibility, also reflects a kind of important conflict for which Delphi policies are intended to overcome them.In this case, the analyst is not willing to delete this item.Instead, they will report this conflict as part of the second round of questionnaires, asking respondents to provide reasons, assumptions or arguments that make them have such a different position.Another way to deal with this disagreement is to compile and report a measure of average polarization, which can be defined as the absolute difference between scores for all combinations of respondents who answer a question.

Step 5: Development of subsequent questionnaires
The questionnaire must be remade for the second, third, fourth, or fifth round (most Delphi policies do three to five times).As mentioned earlier, the results of the previous rounds are used as the basis for the next questionnaire.One of the most important aspects of the Delphi policy takes place in the following rounds because advocates have the opportunity to know the results after a round is completed and explicitly submit reasons, assumptions, and arguments for their opinions.Note that the later rounds not only contain information about the main trends, dispersion and polarization; they also include a summary of the arguments offered for the most conflicting opinions.In this way, the Delphi policy provides a logical debate and maximizes the probability of loss of opinion and opinion on the basis of feelings.Before the last round of questionnaires was completed, all advocates had the opportunity to state their initial position regarding forecasting, issues, objectives, and choices; to test and evaluate the reason why their position is different from the other positions, and to reevaluate and change their position.
Step 6: Organize group meetings One of the last tasks is to bring advocates together to face-to-face to discuss the reasons, assumptions, and arguments that relate their positions.This face-to-face meeting, because it takes place after all advocates give birth to a contemplation of their own position and the position of others & can create an atmosphere that is full of confidence that cannot be realized in a committee.Face-to-face discussions also create conditions in which advocates can debate their position intensively and receive feedback directly and immediately.

Step 7: Preparation of final report
There is no guarantee that respondents will reach consensus, but it is reasonable to hope that creative ideas about issues, goals, choices and their consequences are the most important results of a Delphi policy.Thus, the final report must include a review of the various issues and choices that arise, and explain in what way all the conflict positions & underlying arguments.This report can then be submitted to policymakers, who use the results of the Delphi policy as one of the information sources in reaching a decision.

Case Example 1
The case example is the selection of small and medium-sized industries for SMEs to be developed to support Ecotourism in the West Lombok District of NTB Province, the type of agro-industry that has bright prospects to develop.In this case, there are four decision makers (PK) consisting of managers of business development, marketing managers, Tourism Experts, Ministry of Industry in NTB. and experts in the development of Small and Medium Business Schools.From the brainstorming process were obtained 16 alternatives and 3 criteria.Sixteenth alternative results of the brainstorming process, namely Then each decision maker assesses the sixteen comprehensively with the method of valuation on a scale of 1 to 6.The preference values given by each decision maker towards alternatives can be seen in Table 3 and Table 4 From the final results, the alternatives to be followed up are alternatives that have a high average value (5), namely alternatives 1, 3, 5 and 16 which are respectively: (1) dairy products industry; (2) fish processing industry; (3) oil palm industry; (4) timber industry

Case Example 2
In relation to the study of vocational career center (VCC) models, the Delphi technique is used to seek agreement or consensus from industry HRD managers, production managers, industrial networking organizations, labor development HR experts, and competency testing organizations.Focus group discussions (FGD) involved in supporting research (Priyanto, 2010)  The results taken from the agreement or consensus focus group discussion are to examine and decide on various matters related to research problems, namely: (1) make changes and improvements to the curriculum in VCC learning on curricula that have been tested in a limited manner.The results of the consensus are attached; (2) conducting preparation and refinement of competency evaluation instruments at the end of the work internship with VCC learning.The results of the consensus are attached.
In more detail, the results of the consensus of the VCC learning model can be seen in Table 5,6,& 7.
) feedback: individual opinion moderated by the group.Examples of applications: (1) forecasting technology, for example: treatment in 1990; (2) Demand forecasting, for example: Hawaii tourism in 2000; (3) Forecasting of changes in hotel management in Hong Kong (1997); (4) Assessment of environmental impacts in the development of Salt's Mill; (5) Priority in the management of cultural performances; (6) Definition of ecotourism.

Table 1 .
Types of Items and Scales Used in the Delphi Questionnaire Policy Use of the Delphy Technique:a Case for the Development …Soenarto, Rahmawati 99

Table 2 .
Hypothetical Responses to Delphi Policy of the First Round: Desirability and Feasibility of the Purpose of Drug Control

.
Use of the Delphy Technique:a Case for the Development …

Table 4 .
Final Results of the Delphi Method are as follows: (1) the industrial group consists of 13 machining industries in thr regions of Surabaya, Malang, Sidoarjo, Gresik, Madiun, and Bandung; (2) job market institutions are from the East Java Industrial Network Forum; (3) quality assurance institution from Global Mandiri competency test at PT. PAL Surabaya; (4) educational institutions from the UPT Vocational Education Training and Development (PPPK) East Java Provincial Education Office.