WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management...
-
Upload
kellie-wilkinson -
Category
Documents
-
view
215 -
download
0
Transcript of WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management...
![Page 1: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/1.jpg)
WERS the Validity: What’s The Current State of Play?Dr Andrew R Timming
Reader in Management
University of St Andrews
![Page 2: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/2.jpg)
Preface
• WERS is the single greatest source of data on work and employment in the history of not just the United Kingdom, but also the world
• No other dataset comes close
• But scientific advancement depends on critique
![Page 3: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/3.jpg)
Background
• From WIRS (1980, 1984, 1990) to WERS (1998, 2004, 2011)
• WERS 2011: (i) management questionnaire, (ii) financial performance questionnaire, (iii) worker representative questionnaire, (iv) survey of employees and (v) employee profile questionnaire
• 2,680 face-to-face interviews with managers; 1,002 interviews with employee reps; 21,981 respondents completed the survey of employees
![Page 4: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/4.jpg)
Structure of Presentation
• Look at my top 10 “best practices” for survey design
• Evaluate how the WERS instruments stand in relation to those top 10 practices
• Focus mainly on survey of employees
• Summarize key points to draw from the presentation
![Page 5: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/5.jpg)
Tip 1
• Surveys should be relevant and timely
• Relevant according to whose judgement?
• Evidence of timeliness is clear: workplaces in the shadow of recession
• But we still have to ask how much value is added by some of the questions.
![Page 6: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/6.jpg)
Timeliness
![Page 7: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/7.jpg)
Relevance
![Page 8: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/8.jpg)
Tip 2
• The layout should be clear, not cluttered
• If the survey is too spread out, it reduces response rate; too cluttered, it reduces the quality of the data
• On this point, the Survey of Employees gets an A+
• Nice aesthetics, clear instructions, inviting, good boundaries and delineation between items, clear flow
![Page 9: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/9.jpg)
Clear Layout
![Page 10: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/10.jpg)
Tip 3
• Strike a balance on length of the survey
• MQ response rate=46.3%; WR response rate=63.9%; Survey of Employees=54.3%; financial performance questionnaire= 31.8%
• Face-to-face administration is usually much higher than postal, but in WERS 2011 we see the opposite
• Was the length of the instruments driving away respondents?
![Page 11: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/11.jpg)
Tip 4
• Items should be short and simple
• Principle of parsimony
• Express what you need to express in as few words as possible and as clearly as possible
• On this point, WERS does a pretty good job, but not perfect . . .
![Page 12: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/12.jpg)
![Page 13: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/13.jpg)
Tip 5
• Avoid leading questions
• ‘Do you agree that Scotland should be an independent country?’ vs ‘Should Scotland be an independent country?’
• How we ask a question can result an an expected response
![Page 14: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/14.jpg)
Overall, how satisfied or dissatisfied are you with …
![Page 15: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/15.jpg)
Tip 6
• Response categories should be exhaustive and mutually exclusive
• Exhaustive: all possibly choices are listed
• Mutually exclusive: no two choices overlap
• WERS is generally good on this point
![Page 16: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/16.jpg)
Mutually exclusive?
![Page 17: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/17.jpg)
Tip 7
• Respondents should be knowledgeable about the topic or topics.
• Albanian foreign policy towards China . . .
• No problem with survey of employees, but there may be concerns with management and employee rep questionnaires
![Page 18: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/18.jpg)
Are workers’ reps qualified to speculate?
![Page 19: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/19.jpg)
Can a manager really know the answer to this question?
![Page 20: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/20.jpg)
Tip 8
• Items must not be double-barreled
• ‘Double-barreled’ implies that a single item is asking two questions in one
• Double-barreled items lead to serious questions about whether we’re measuring what we say we’re measuring
• On this point, WERS could be improved
![Page 21: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/21.jpg)
Double-barreled items
![Page 22: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/22.jpg)
Tip 9
• Complex constructs require multiple items
• Multiple items allow for the construction of latent variables
• On this point, WERS does a really good job
![Page 23: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/23.jpg)
![Page 24: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/24.jpg)
Tip 10
• Survey instruments should always be piloted and reformed according to results
• Obviously, the fact that there are different iterations of the survey implies sufficient piloting
• But on the question of reform, we need to continue to refine the instrument, even though this may lead to a break in continuity from previous versions
![Page 25: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/25.jpg)
Summary
• On the whole, WERS 2011 is a fantastic resource and a very well designed set of instruments . . . but it’s not perfect
• What is included and excluded from surveys is a political choice and influenced by power
• Do we tolerate imperfection for the sake of continuity?
![Page 26: WERS the Validity: What’s The Current State of Play? Dr Andrew R Timming Reader in Management University of St Andrews.](https://reader036.fdocuments.in/reader036/viewer/2022062423/5697bf771a28abf838c814e6/html5/thumbnails/26.jpg)
Counter-Arguments
• ‘We shouldn’t change or reduce the amount of data gathered because they are always relevant to someone’.
• ‘Other surveys have used these same measures, so why shouldn’t we as well’?
• ‘Our response rates are, in fact, much better than other comparable surveys’.
• ‘Effecting changes now will make it difficult to do comparable longitudinal analyses’.
• ‘You’ve cherry picked a few examples to make the whole survey look bad’.
• ‘Making changes would cost too much money without an effective return’.