Transitions of Care Harmonization Process Jillian Wanner Harmonization Lead April 8, 2011.
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design Internet Business...
-
Upload
neal-stephens -
Category
Documents
-
view
214 -
download
0
Transcript of OECD/EC Task Force on Harmonization of Survey Operations and Technical Design Internet Business...
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design Internet Business Tendency Surveys
InternetBusiness Tendency Surveys
The author would like to thank all colleagues who have contributed to this research project:
Aggelos Tsakanikas, Alma Rutkauskiene, Aloisio Campelo, Bianca Maria Martelli, Braileanu Gheorghita and Nedelea Olga, Brigita Vrabic Kek, Casper Larsen, Dagmar Blahova, Dorthy Watson, Eliza Loucaidou, Evelin Ahermaa, Frank Hansen, George
Kershoff, Gerhard Schwarz, Hanna Sekowska, Illmars Vanags, Jan Henrik Wang, Jean-Paul Vonck and Isabelle De Greef, Jonathan Wood, Jose Manuel Almendros Ulibarri,
Kevin Caruana, Marie Hormannova, Paulo David, Penna Urrila, Philippe Scherrer, Raymund Petz, Richard Etter, Roger Knudsen, Sidney Vergouw, Soshihiro Masuda,
Teofana Genova, colleagues from the Ifo Business Surveys Dpt. and ECFIN
2/19
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design Internet Business Tendency Surveys
Previous ResearchTask Force on Harmonization of Survey Operations and Technical Design
PRACTICE OF INTERNET SURVEYS
Task Force Group on
Identification and Assessment of Recommended Practices for the Design of
Internet Surveys
Task Force MembersCurtin, R. (University of Michigan, United States), De Greef, I. (Bank of Belgium), Etter, R. (KOF / ETZ, Switzerland), Malgarini, M. (ISAE, Italy), Neubarth, W. (ZUMA, Germany), Nillson, Ronny (OECD), Scherrer, P. (INSEE, France), Stangl, A. (Ifo Institute, Germany), Wood, J. (CBI, United Kingdom), European Commission
Task Force on Harmonization of Survey Operations and Technical Design
PRACTICE OF INTERNET SURVEYS
Task Force Group on
Identification and Assessment of Recommended Practices for the Design of
Internet Surveys
Task Force MembersCurtin, R. (University of Michigan, United States), De Greef, I. (Bank of Belgium), Etter, R. (KOF / ETZ, Switzerland), Malgarini, M. (ISAE, Italy), Neubarth, W. (ZUMA, Germany), Nillson, Ronny (OECD), Scherrer, P. (INSEE, France), Stangl, A. (Ifo Institute, Germany), Wood, J. (CBI, United Kingdom), European Commission
3/19
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design Internet Business Tendency Surveys
1. Internet Adoption of the BTS Institutes Web Surveys E-mail Surveys
2. Practical Issues: Formats and Spam
3. The Experience with the Internet as Survey Instrument
4. Summary and Research Outlook
Overview
32 institutes (25 EU countries and 7 non-EU countries)
4/19
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design Internet Business Tendency Surveys
Internet Adoption of the BTS Institutes
Countries
Web is used for the following reasons: EU / non EU
Website of the institute 20 / 7
Display of survey results to the public 18 / 7
Display of individualized survey results for participants 4 / 1
Web questionnaire 10 / 4
None of the 14 institutes has abandoned the traditional survey mode yet.
5/19
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design Internet Business Tendency Surveys
Web SurveysImplemented in the year Manufacturing Retail Services Construction Investment
1998 Estonia Estonia Estonia Estonia
2000Sweden,
SwitzerlandSweden,
SwitzerlandSweden,
SwitzerlandSweden
2001 Finland, BrazilFinland,Germany
Finland
2002 LuxembourgGermany,
LuxembourgFinland
2003 Malta
2004Hungary, Germany,
Austria, NorwayHungary
Austria,Hungary
Austria,Hungary
2005 France Austria
2006Portugal,
South AfricaPortugal,
South AfricaFrance, Portugal,
South AfricaPortugal,
South AfricaPortugal
Existing Web Surveys 14 6 10 9 3
6/19
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design Internet Business Tendency Surveys
49%
95%85%
45% 45%35%
20%
5%
60%
45%
55%
10%
49%
50%
5% 5%
55% 55%65%
80%
95%
40%
55%45%
85%
50%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Paper Interview Web
EU Countries non-EU CountriesEU Average
Response Rates: Manufacturing Sector
7/19
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design Internet Business Tendency Surveys
51%
90%
47%
15%
70%
49%
10%
53%
85%
30%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Paper Interview W eb
EU Countries non-EU CountryEU Average
Response Rates: Retail Sector
8/19
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design Internet Business Tendency Surveys
39%
85%
40%
25%
40% 40%
5%
60%
56%
15%
30%
75%
60% 60%
95%
40%
5%
30%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Paper Interview W eb Email
EU Countries non-EU CountryEU Average
Response Rates: Service Sector
9/19
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design Internet Business Tendency Surveys
62%
98%87%
70% 70%
35%
1%
97%
60%
2%3%
30% 30%
65% 65%
99%
3%
40%
5%
37%
5%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Paper Interview W eb Email
EU Countries non-EU CountryEU Average
Response Rates: Construction Sector
10/19
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design Internet Business Tendency Surveys
E-mail Surveys
Implemented in the year
Manufacturing Retail Services Construction Investment
2000Slovakia,
NetherlandsSlovakia, Slovakia
2002Austria
(abandoned)Slovakia, Germany
Austria(abandoned)
2003 Italy, BelgiumItaly,
Belgium
Netherlands,Italy,
BelgiumBelgium
2004 Italy
2005Czech Rep. Romania
Czech Rep. Romania
Czech Rep. Romania
Czech Rep. Romania
Czech Rep. Romania
2006 Greece Greece Greece
Existing E-mail surveys
6 6 8 6 2
11/19
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design Internet Business Tendency Surveys
53%
80%
15%
30%20%
75%
99% 95%
33%
20%
85%
25%
1% 5%
14% 80%70%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Paper Interview W eb Email
EU Countries non-EU CountryEU Average
Response Rates: Manufacturing Sector
12/19
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design Internet Business Tendency Surveys
52%
75%
20%
45%
20%
60%
95% 98%
35%25%
80%
55%
5%
40%
3% 2%
13% 75%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Paper Interview W eb Email
EU Countries non-EU CountryEU Average
Response Rates: Service Sector
13/19
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design Internet Business Tendency Surveys
86% 85%
55%
100%92%
99% 97%
14% 15%
45%
8%1% 3%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Paper Interview W eb Email
EU Countries non-EU CountryEU Average
Response Rates: Retail Sector
14/19
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design Internet Business Tendency Surveys
74%85%
50%
90%
45%
99% 95%
26%15%
50%
10%1% 5%
55%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Paper Interview W eb Email
EU Countries non-EU CountryEU Average
Response Rates: Construction Sector
15/19
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design Internet Business Tendency Surveys
Practical Issues: Formats and Spam Filter
E-mail format: - 5 institutes send their E-mail in the HTML format- 17 institutes adhere to the plain-text format
Web questionnaire format: - 9 of 10 institutes in the EU have chosen the one-page design- 2 of 4 institutes outside the EU have multiple-page Web forms
The majority of institutes reported an overall bounce rate below 5% According to the institutes’ estimations, of those E-mails that do not
reach the recipients, about 20% are due to Spam-filter.
16/19
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design Internet Business Tendency Surveys
Countries For Participants For the Surveyor Paper questionnaire Email questionnaire Web questionnaire
1 J JJ J J2 JJ JJ K L JJ3 LL L J K L4 JJ JJ K JJ5 J JJ JJ JJ6 J J J LL L7 J JJ K JJ8 JJ L JJ J K9 JJ JJ J K J10 J J J11 JJ JJ JJ JJ12 J JJ K JJ13 J JJ J JJ14 L LL J J15 JJ J J J J16 L L JJ JJ17 J19 JJ21 JJ
26 L J JJ LL27 JJ JJ L L28 JJ JJ J JJ29 J J J J30 JJ JJ K JJ31 J J J32 L L JJ L
Practicability of the Internet as a survey instrument
Experience with the different questionnaire modes
Non-EU Countries
EU Countries
17/19
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design Internet Business Tendency Surveys
Practicability of the Internet as Survey Instrument for Participants and the Surveyor
Practicability of the Internet for surveying
Very practical
PracticalSomewhat problematic
Problematic Total
For Participants 8 9 4 1 22
For the Surveyor 12 6 4 1 23
18/19
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design Internet Business Tendency Surveys
Experience with the Different Survey Modes
Experience with the different survey modes
Very good
Good SatisfactoryRather bad
/ badTotal
Paper questionnaire 9 12 4 0 25
Email questionnaire 4 3 2 4 13
Web questionnaire 6 4 1 4 15
19/19
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design Internet Business Tendency Surveys
19/
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design
PRACTICE OF INTERNET SURVEYS
Reliabilty
Internet respondents tend to read more quickly
Internet environment may lead to a higher inaccuracy
Respondent has to learn the structure of the hypertext
Internet questionnaires affords higher cognitive effort
Higher risk of measurement errors
Prevention and Control
19/
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design
PRACTICE OF INTERNET SURVEYS
Reliabilty
Internet respondents tend to read more quickly
Internet environment may lead to a higher inaccuracy
Respondent has to learn the structure of the hypertext
Internet questionnaires affords higher cognitive effort
Higher risk of measurement errors
Prevention and Control
23/
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design
PRACTICE OF INTERNET SURVEYS
Validity – Differences in Responses Findings vary greatly
Selection bias may influence responses
-40
-30
-20
-10
0
10
20
30
40
50
60
70
Q1/20
00
Q2/20
00
Q3/20
00
Q4/20
00
Q1/20
01
Q2/20
01
Q3/20
01
Q4/20
01
Q1/20
02
Q2/20
02
Q3/20
02
Q4/20
02
Q1/20
03
Q2/20
03
Q3/20
03
Q4/20
03
Q1/20
04
Q2/20
04
Total Panel Web Respondents
Balances of the assessments of the present economic situation (IT companies)
23/
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design
PRACTICE OF INTERNET SURVEYS
Validity – Differences in Responses Findings vary greatly
Selection bias may influence responses
-40
-30
-20
-10
0
10
20
30
40
50
60
70
Q1/20
00
Q2/20
00
Q3/20
00
Q4/20
00
Q1/20
01
Q2/20
01
Q3/20
01
Q4/20
01
Q1/20
02
Q2/20
02
Q3/20
02
Q4/20
02
Q1/20
03
Q2/20
03
Q3/20
03
Q4/20
03
Q1/20
04
Q2/20
04
Total Panel Web Respondents
Balances of the assessments of the present economic situation (IT companies)
20/
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design
PRACTICE OF INTERNET SURVEYS
Item Non-response
Lower item non-response for Web surveys
On-line participants tend to be less thorough
Internet is perceived to be anonymous
Lower perception of responsibility
On-line readers are less patient
Internet usage is associated with costs
20/
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design
PRACTICE OF INTERNET SURVEYS
Item Non-response
Lower item non-response for Web surveys
On-line participants tend to be less thorough
Internet is perceived to be anonymous
Lower perception of responsibility
On-line readers are less patient
Internet usage is associated with costs
3/18
Task Force on Harmonization of Survey Operations and Technical Design
PRACTICE OF INTERNET SURVEYS
Coverage Problems
50
60
70
80
90
100
Fin
lan
d
Sw
ede
n
Den
mark
Can
ad
a
Au
stralia
Czech
Rep
.
Au
stria
Japa
n
New
Zeala
nd
Germ
any
Ireland
Sp
ain
No
rwa
y
Neth
erland
s
Lu
xem
bo
urg
Sw
itzerland
Italy
Po
rtug
al
Greece
UK
Mexico
%10-49 50-249 250 and more
Percentage of Businesses Using the Internet by Number of Employees
Source: OECD 2001 or latest available year 3/18
Task Force on Harmonization of Survey Operations and Technical Design
PRACTICE OF INTERNET SURVEYS
Coverage Problems
50
60
70
80
90
100
Fin
lan
d
Sw
ede
n
Den
mark
Can
ad
a
Au
stralia
Czech
Rep
.
Au
stria
Japa
n
New
Zeala
nd
Germ
any
Ireland
Sp
ain
No
rwa
y
Neth
erland
s
Lu
xem
bo
urg
Sw
itzerland
Italy
Po
rtug
al
Greece
UK
Mexico
%10-49 50-249 250 and more
Percentage of Businesses Using the Internet by Number of Employees
Source: OECD 2001 or latest available year
20/19
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design Internet Business Tendency Surveys
Research Outlook
Methodological Manual
Coverage Response patterns Reliability Validity
Creation of a Sample Internet Questionnaire
Technical Manual
OECD/EC Task Force on Harmonization of Survey Operations and Technical Design Internet Business Tendency Surveys
Email: Anna Stangl [email protected]
Ronny Nilsson [email protected]