EXPERIENCE of TURKISH DELPHİ SURVEY

Prof.Dr. Haluk Geray, University of Ankara– Designing Team Member of Vision 2023

Technology Foresight Exercise

A tool for exploring the future

• Named after the oracle at Delphi in ancient Greece

• Formally identified as such, the Delphi was originally developed by RAND Corporation researchers, who were involved on a U.S. Airforce sponsored project in the 1950’s.

A tool for exploring the future

• Initial experiments showed that predictions made by a group were more likely to be correct than predictions made by the same individiuals alone.

• A second finding: individual estimates showed a tendency to converge as the experiment continued in rounds

• But, recent applications show that it may be designed to explore difference as well

Description of the Delphi Method

• Structural survey with two or more rounds

• Make use of intuitive, tacit or explicit knowledge of experts

• Participants remain anonymous to each other

• Result of the previous rounds given as feedback

Description of the Delphi Method

• An interactive group communication process in which participants are ensured that they are not under pressure to conform to the perceived views of “dominant” colleagues.

• At least a couple of weeks, given to participants to shape their judgement on the survey topics.

Why Delphi?• Participation of a larger number of experts/stakeholders,

whose voices will hardly be present around a table with a limited number of seats.

• Creates interaction among people who are far away from each other.

• It is a fast and cheaper method compared to some

• Wider consultation about outcome of other methods is maintained

• When repeated, it may reflect changes or continuities in time

• “Experts” say they learn a lot in the process

Best for...

• Occurence of future events

• Probable trends in the future

• Ranking alternative goals, objectives or instruments

• Creating strategies

• Allocating scarce resources between competing options

Golden Rule of Any Delphi Survey Design

• What do you need to learn?– Purpose

– Needs

– Specific Context

• From whom you want to learn?

VISION 2023 – TECHNOLOGY FORESİGHT

Delphi Survey in the context of Turkey

TECHNOLOGY CAPABILITY

R&D MANPOWER INVENTORY

R&D INFRASTRUCTURE

2003-2023 STRATEGY DOCUMENT• S&T Vision• Strategic Technologies and R&D Priorities• Policy Recommendations

TECHNOLOGY FORESIGHT

Decision of Supreme Council for S&T

Foresight Panels

1- Education and Human Resources (Thematic)

2- Environment and Sustainable Dev. (Thematic)

3- Information and Communication (inc. electronics)

4- Energy and Natural Resources

5- Health and Pharmaceuticals

6- Defence, Aeronautics and Space Industries

7- Agriculture and Food

8- Machinery and Materials

9- Transportation and Tourism

10- Chemicals

11- Textiles

12- Construction and Infrastructure

Among 1st Phase Mandates of Panels were….

• Listing current trends and drivers• Analyzing SWOT of Turkey in respective sectors• Develop a vision• Set socio-economic (strategic) targets to realize

vision• Choose targets that can be influenced by

science&technology• Derive technologies and give priorities using criterias

1st Phase completed as of January 24, 2003

• 2nd PHASE: Focused consultation (End of March)

• 3rd PHASE: Wider consultation by Delphi (April-June, 2003)

• 4rd PHASE: Finalize expert panel reports by reviewing it following Delphi results.

Context of Delphi in Vision 2023 Project

Expected to serve

• Consultation of phase one results to wider “expert” groups

Turkish Delphi

• Delphi statement generation: In many countries special committees have beenset up but in Turkey it was done by members of expert panels, which was formed for Vision 2023 technology foresight.

• Virtual Delphi Survey• It was possible to design a virtual personal

survey form by using keywords• Software was developed in-house

Turkish Delphi

• Delphi variables were developed by expert panels and refined by the project office– Ankara Workshop was organized with

participation of foreign experts

• Index development: It was formulated by the project office based on decisions of the Steering Committee (the highest decisin making body of the foresight exercise)

Vision 2023 Technology Delphi

Place of Delphi in foresight Wider consultation. To guide and to asssist expert panels

Implementation Year 2003

Time span 2023 (20 year)

Number of panels 11

Delphi experience 1st

Number of respondents 2,300

Response rate 33% around 7,000

Survey type By post and on-line

Snowballing Allowed for on-line survey

Turkish Delphi Topics (Delphi Statements)

Turkey Japan

Topic type Technology topics with elucidation, development, practical use, widespread use

SAME

Topic generation

Generated by expert panels based on their vision and strategic aims

Generated by Delphi expert panels based on outcomes of the needs committee (social scentists and culture people)

Number of statements

513 413

Delphi statements

Delphi variables

Variables in Turkish Delphi

• Degree of expertise • Current situation• Research level to begin • Type of intervention• Realization Time• Expected effects• Importance• Feasibility

Expertise

• 1 (I’am not an expert)

• 2 (Little)

• 3 (Sufficient)• 4 (High)

• In the results page, high and sufficient was united as experts

Current situation

• Level of researhcers (1=None, 2=weak, 3=sufficient, 4=very good)

• Level of R&D infrastructure (labarotories and equipment)

• Basic science capability

• Existence of innovative companies

• Existence of competitive companies

Research level to begin

• Choose one:– Basic research– Applied research– Experimental (pre-competitive ind. Research)– Industrial development

Realization Time

• Choose one of them– 2003-2007 – 2008-2012– 2013-2022– 2023 and later– Never

Type of intervention (policy tools)

• Infrastructure support

• Project support

• Start-up support

• Guided projects

• Human resources support

• Public procumerment

Expected contribution

• Science, technology and innovation capacity (no contribution; little contribution; some contribution; high contribution)

• National value added (1, 2, 3, 4)

• Quality of life (1, 2, 3, 4)

• Competitivenes (1, 2, 3, 4)

• Environment (1, 2, 3, 4)

Comparison / VariablesVariable Turkey JapanDegree of expertise Yes YesImportance Derived from effects Asked as a variable

Realization Time Yes YesFeasibility Yes NoExpected effects Yes YesType of intervention Yes YesLeading countries No YesCurrent situation Yes NoPotential problems No YesResearch level to begin Yes No

TURKEY: IMPORTANCE

INDEX

CONTRIBUTİON TO Weight (wj)

COMPETİTIVENESS %28

SCIENCE, TECHNOLOGY and INNOVATION CAPACITY

%26

ENVIRONMENT and SUSTAINABİLITY

%16

NATIONAL VALUE ADDED %15

QUALITY of LIFE %15

TURKEY: FEASIBILITY INDEX

Existing Situation

Researcher Potential

Research Infrastructure

Basic ScienceCapacity

Innovative Companies

Competitve Companies

STAGE TO BEGIN with

WEIGHT (wij)

Basic Research % 25 % 25 % 25 % 15 % 10

Applied Research % 25 % 20 % 20 % 20 % 15

Experimental Research (PCIR)

% 20 % 20 % 15 % 20 % 25

Industrial Development % 20 % 15 % 10 % 30 % 25

59; 48

77; 2477; 22

77; 5261; 57

10,00

20,00

30,00

40,00

50,00

60,00

70,00

80,00

90,00

10,00 20,00 30,00 40,00 50,00 60,00 70,00 80,00 90,00önem

yapı labilirlik UZMANLARFeasibility

Importance

All ICT Statements Feasibility/Importance

10,00

20,00

30,00

40,00

50,00

60,00

70,00

80,00

10,00 20,00 30,00 40,00 50,00 60,00 70,00 80,00önem

yapýlabilirlik TÜM KATILIMCILARFeasibility

Importance

All respondants

All ICT Statements Feasibility/Importance

10,00

20,00

30,00

40,00

50,00

60,00

70,00

80,00

90,00

10,00 20,00 30,00 40,00 50,00 60,00 70,00 80,00 90,00önem

y apý labilirlik UZM A NLA RFeasibility

Importance

Experts

All Statements Feasibility/Importance

20,00

30,00

40,00

50,00

60,00

70,00

80,00

20,00 30,00 40,00 50,00 60,00 70,00 80,00önem

yapýlabilirlik TÜM KATILIMCILARFeasibility

Importance

All respondants

All Statements Feasibility/Importance

10,00

20,00

30,00

40,00

50,00

60,00

70,00

80,00

90,00

10,00 20,00 30,00 40,00 50,00 60,00 70,00 80,00 90,00önem

y apý labilirlik UZM A NLA RFeasibility

Importance

Experts

From respondants…

• “As a person who has been involved in research since 20 years and as a person who believes we did not use the last 10 years well in this context, this survey caused my hopes to resurface once again. This type of studies will help to maintain necessary quality of life to our people, to our children and to our environment. Thank you!

• Ömer Yavaş, ICT expert, man

From respondants…

• “This survey should have been done 15 years ago. İt is a very good and comfortable survey. Stamements are celar and understandable. I would like to think those who prepared. Results should be publicized.”

• Emin Kaya, from a private company in ICT sector. Man.

From respondants…

• “I am sorry I didn’t like Delphi statements. Those statements cannot be used for developing visions.”

• Emel Bedik, from a private company, woman

From respondants…

• You people, what do you want to do with this crazy survey! Did you ever try to answer yourselves!

• Selim Soylu from a university, Male.

Last notes…

• Process benefits, especially statement generation, proved to be more important than the outcome

• People said they loved to generate statements and that they learnt a lot from it

• In Turkish case it was consultative, expert panels could over ride

• Delphi Survey Final Report was not publicized enough• The reporting module of the Delphi exercise was not

developed so it was impossible to design a web site for it.