Abstract Number: 020-0186
Creating an Enabling Tool for Facilitating Engagement in Continuous Innovation Programmes
Dr Helen T Wagner*1, Dr Susan C Morton1, and Prof Chris J Backhouse1
1 Manufacturing Organisation Group, Wolfson School of Mechanical & Manufacturing Engineering, Loughborough University, UK
POMS 22nd Annual Conference, Nevada, USAApril 29 to May 2, 2011
Abstract
Although Lean Manufacturing is an established concept in both academia and industry,
consideration of the stages that follow a company-wide Lean initiative has received far less
attention. Pursuing continuous innovation (CI) takes commitment from all involved, and the
gap between knowing about continuous innovation and actually doing it needs to be filled. To
facilitate organizational CI, a need has been identified for a bespoke tool that will enable
managers to understand their people and support problem solving activities, at the
supervisory/team management level in particular. Having identified five main constructs that
contribute to successful engagement of employees in CI and their respective diagnostic tools,
the process of questionnaire development was researched. This offered guidelines for
planning, question wording, ordering and presentation, which were actioned in the
development. The work has resulted in a bespoke tool for facilitating engagement that will
add to the information available to managers and academics alike.
Introduction* Corresponding author:Dr H T Wagner Research Associate: NIMBLE ProjectWolfson School of Mechanical & Manufacturing EngineeringLoughborough University, LoughboroughLeicestershire, UK LE11 3TUEmail: [email protected]
Lean Manufacturing is a well established concept in both academia and industry; however,
what is required when moving on from a lean initiative to attain further benefits is not such
an established field. What lies beyond Lean in relation to performance improvement requires
further investigation; the gap between knowing about continuous innovation (CI) and doing it
also needs to be removed, or reduced at the very least. To facilitate continuous organizational
innovation, a requirement has been identified for a new diagnostic tool for use by managers
to assess all levels of the organization and to assist with problem solving at the level of
supervisory/team management in particular.
Although the information needed could be collected by interviewing workers within an
organization, several advantages to using a standardized questionnaire exist. Studies show
that people are often more honest when completing a self-administered questionnaire [1, 2].
They provide an easy and time effective route to collecting data from many people, provide
anonymity and limit researcher bias, and the structured format ensures each respondent reads
and answers the exact same questions, which makes for robust analysis [3].
Devising a new questionnaire is not an easy task [Error: Reference source not found] and
often researchers underestimate what is required, thinking that because they have knowledge
of a topic they are capable of developing a good questionnaire [4]. In fact, it is a highly
complex and time consuming process [5] that cannot be shortcut, no matter how tempting it
may be [Error: Reference source not found]. The process requires not only thorough
knowledge, but attention to detail [Error: Reference source not found] and a ‘stringent and
scrupulous’ approach to ensure the data collected provides what is required in a usable form
[Error: Reference source not found]. This is essential, as the consequences of the decisions
made during the design phase impact directly on the results obtained [6] and, therefore, the
findings and validity of the study.
It is, therefore, the purpose of this paper to introduce the process undertaken to create a new,
bespoke questionnaire, created to assess the factors affecting employee engagement in
continuous innovation programmes.
Existing Tools that measure the constructs affecting employee engagement in CI
There are five identified constructs that affect the potential for employees to engage with the
CI programme, as outlined by Wagner et al. [7]. In order to incorporate each of the constructs
of creativity, empowerment, leader-member relationship, team role and leadership style, into
the new questionnaire, the existing tools that measure these constructs were assessed.
Job Diagnostic Survey
The Job Characteristics Model (JCM) proposed by Hackman and Oldham [8] is a tool for
analysing the satisfaction and motivating potential offered by a job role. The quantitative
calculation of each component of the model is facilitated by the Job Diagnostic Survey (JDS)
[9], which was developed for use “in research and evaluation activities aimed at assessing
the effects of redesigned jobs on the people who do them” [10].
Questions are set out to identify the core characteristics using two techniques; standard
questions and reverse scored questions. Some researchers have experienced problems with
this approach and have sought to make improvements [11], but when their revised questioning
was tested by others, in a direct comparison, no improvement was found and the authors
recommended continued use of the original questionnaire [12]. The results of the JDS are
entered into the Motivating Potential Score (MPS) equation, also created by Hackman and
Oldham [Error: Reference source not found], where each component in the equation is scored
from 1 to 7, with results ranging from 1 to 343 and scores commonly around 150 [13].
Leader-Member Exchange (LMX)
Leader-Member Exchange (LMX), with its roots in Social Exchange Theory [14], is based on
the two-way, dyadic relationship [15] between a leader and an individual subordinate [16].
Each relationship becomes differentiated [Error: Reference source not found,17] based on
factors affecting the level of interaction, communication, understanding and trust [18] between
the two; constraining or facilitating the development of the relationship [19]. The measure of
the multidimensional relationship [Error: Reference source not found] is based on the
perceptions of both the leader and subordinate, and so in its study, it is vital to view it
objectively from both sides [20].
The LMX-7 questionnaire, put forward by Graen and Uhl-Bien [21], assesses the quality of
the relationship of the supervisor with each individual team-member. The tool is made up of
seven items that exemplify different aspects of the leader-subordinate working relationship
[Error: Reference source not found] and is measured on a five-point scale.
The Belbin team roles model
The Belbin team roles model [22] identifies the nine potential roles each individual could
exhibit when working in a team, outlining the specific behaviours and skills each brings to
the team dynamic.
The Self Perception Inventory (SPI) [23] tool comprises seven questions, which ask the
respondent to distribute ten points between ten different response options. The points can be
allocated where they like, but it is important to use all ten points. Subjects explored include a
person’s contribution to a team and what they feel they lack, their approach to tasks and
problems, and working with others or in a group. Participants completing the questionnaire
indicate their own perceptions of their behaviour in each situation. This can be complemented
by the addition of the Observer Assessment [Error: Reference source not found], where other
team members or the team supervisor provides their perceptions of the participant, to give an
outsider view.
Although some researchers question its validity, many support the tool. They [e.g. Fisher et
al. [Error: Reference source not found] and Partington and Harris [Error: Reference source
not found]] suggest that it has made a significant contribution to understanding [24], also
suggesting its value in use is more important than its psychometric validity [25] and
recognizing that to set aside the work because of doubt would be a great pity [Error:
Reference source not found].
Research Tools – KEYS to Creativity
The KEYS to Creativity instrument (KEYS) was developed by Amabile et al. [26], to meet the
need for research in organizational theory and practice, by using theoretical knowledge from
literature to create a tool that would test real organizational settings. It looks at creativity
within the working environment [27], examining the intrinsic motivation of individuals to be
creative and assessing perceived barriers and enablers to creativity [28]. It is said that “the
value of KEYS lies in its capacity to accurately identify the conditions necessary for
innovation to occur” [29]. Aimed at assessing all levels within an organisation, from the shop
floor team to supervisory and organizational, KEYS concentrates on the effects of
environmental factors on an individual’s perceptions, which influence the creativity of their
work [30].
The KEYS instrument itself is made up of 78 questions assessed using a four point scale
[Error: Reference source not found]; purposely designed to force a response by not offering a
neutral option [Error: Reference source not found]. Of the 78, 66 are related to the work
environment with the remaining 12 assessing performance in terms of creativity and
productivity [Error: Reference source not found]. The work environment factors are split
between management practices that encourage and those that inhibit creativity [Error:
Reference source not found], encompassing: organizational encouragement; supervisory
encouragement; work group supports; sufficient resources; challenging work; freedom;
organizational impediments; and workload pressure.
As a tool it has been extensively empirically tested [Error: Reference source not found] and
has been shown to be both robust and rigorous [31], demonstrating its validity and reliability
[Error: Reference source not found] through research with more than twelve thousand
research subjects [Error: Reference source not found].
Extant Literature on Developing Questionnaires
The Oxford English Dictionary defines a questionnaire as: “A formulated series of questions
by which information is sought from a selected group, usually for statistical analysis; a
document containing these” [32], however, some see it as much more. Labaw [33] sees it as not
only a series of questions or just a series of words [34], but a layered structure, where “it is a
totality, a gestalt that is greater than the sum of its individual questions... with each part vital
to every other part and all parts must be handled simultaneously to create this whole
instrument” [Error: Reference source not found].
Although there may be more to it, the basics of the dictionary definition also hold true; as a
tool for data collection in written format, suitable for large numbers of respondents [Error:
Reference source not found] and a series of attitude or opinion statements and questions
developed to elicit a response, which can then be used to measure the variable being studied
[Error: Reference source not found]. Well designed questionnaires can provide an
understanding of the details of an organization’s manufacturing strategy [Error: Reference
source not found] and can aid in driving-in and measuring the success of organizational
change [Error: Reference source not found]. An ideal questionnaire should be clear,
unambiguous and suitable to collect the data required to test the research question or
hypothesis set [Error: Reference source not found]. In order to meet these requirements it
must be designed with the respondents in mind; this will dictate the type of questions,
wording and concepts that can be explored [Error: Reference source not found].
The Process of Development
Before question writing begins there is much work to be done; this starts with knowing what
is the purpose of the research [Error: Reference source not found] and what you wish to
accomplish [Error: Reference source not found]. Questionnaire construction takes place in
stages, which begin with setting objectives [Error: Reference source not found], clearly
defining what will be studied [Error: Reference source not found] and to what level of detail
and accuracy [Error: Reference source not found]. This should involve reviewing appropriate
literature [Error: Reference source not found]. Once all of this is known, research questions
or hypotheses should be developed [Error: Reference source not found]. It is likely that this
initial planning phase will take up a third to a half of the development time of the
questionnaire [Error: Reference source not found].
Another fundamental part to the early stages of questionnaire development, that must run
parallel to both the planning and question development stages, is that of analysis design
[Error: Reference source not found]. A questionnaire must be designed with analysis as an
integral part; so that it can be assured that the data collected will be suitable for analysis
[Error: Reference source not found]. This statistical analysis will allow researchers to study
data on individual respondents or questions, but will also facilitate the presentation of results
and testing of hypotheses [Error: Reference source not found]. It must, however, be
remembered that data collection is paramount and no amount of statistical manipulation can
make up for poor questionnaire design [Error: Reference source not found]. With this in
mind, the practicalities of questionnaire design must next be examined.
Question Wording
The consideration of question wording is one that receives much attention in extant literature.
While all recommend it be given careful consideration, some think that the specific wording
of questions has a much greater impact than others. Brigham [35] suggests that wording has
considerable effect on results; a belief that is supported by Synodinos [Error: Reference
source not found] who found that even small changes in wording can produce response
effects. However, this belief is not shared by all. Labaw [Error: Reference source not found]
suggested that wording variations have little impact on the stability of results, a position that
was corroborated by the findings of Gendall [Error: Reference source not found] who stated
that it is possible to ask the same question in different ways with no effect on respondents’
understanding.
Further contention exists in the phasing of attitude statements. Murray [Error: Reference
source not found] considers that all such statements should be worded positively; however,
Gendall [Error: Reference source not found] found no evidence that wording positively or
negatively has any influence on response. He did find that the strength of a word had an
impact on response, with words such as ‘forbid’ being less acceptable than ‘not allow’ [Error:
Reference source not found].
It is sometimes taken for granted that the respondent reads and understands the question as
the researcher intends, unfortunately this is not always the case [Error: Reference source not
found]. With this in mind, a table of the suggestions for questionnaire wording has been
compiled, outlining the advice of many sources (Table 1).
In addition to question wording, answer format must be considered. It should provide a clear
structure so that respondents know what is required of them [Error: Reference source not
found]. Closed questions offer a fixed choice of answers that come in several different
formats. Yes/No formats are commonly used, but should be limited to avoid guessing [Error:
Reference source not found]. Checklists where the respondent is asked to tick all that apply
[Error: Reference source not found] can be used when multiple answers may be applicable. In
this instance, the use of an ‘other’ box is also recommended in case a possible option has not
been thought of [Error: Reference source not found], although this may not entirely make up
for omissions [Error: Reference source not found]. Category answers are possible [Error:
Reference source not found] as are quantities or bands of figures [Error: Reference source not
found].
Table 1: Advice for wording questions
Rules for wording of questions
Use closed questions where possible to ensure the context is the same for all [Error: Reference source not found, Error: Reference source not found, Error: Reference source not found]
Questions should be simply worded and structured, unambiguous, focussed and short
[Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, Error: Reference source not found]
Less than 20 words [Error: Reference source not found]
Less than 12 words [Error: Reference source not found]
Questions should be clear and precise and not woolly, so all understand and interpret as intended
[Error: Reference source not found, Error: Reference source not found,
1 Marshall, G. “The purpose, design and administration of a questionnaire for data collection”, Radiography, Vol.11, pp131-136, 2005
2 Kelly, P. “Questionnaire design, printing, and distribution”, Government Information Quarterly, Vol.17, No.2, pp147–159, 2000
3 Gillham, B. Developing a Questionnaire, Real World Research, Continuum, London, 2000
11 Idaszak, J.R. and Drasgow, F. “A revision of the Job Diagnostic Survey: Elimination of a Measurement Artefact”, Journal of Applied Psychology, Vol.72, pp69-74, 1987
12 Kulik, C.T., Oldham, G.R. and Langner, P.H. “Measurement of Job Characteristics: Comparison of the Original and the Revised Job Diagnostic Survey”, Journal of Applied Psychology, Vol.73, pp462-466, 1988
Error: Reference source not found, Error: Reference source not found, 36]
Use language appropriate to the target population [Error: Reference source not found]
Phrase to the lowest education level of respondents [Error: Reference source not found,Error: Referencesource not found]
Do not patronise or make too elitist [Error: Reference source not found]
Do not use jargon, unusual words, acronyms, and abbreviations [Error: Reference source not found, Error: Reference source not found]
Avoid unfamiliar, difficult words or words that sound similar to others [Error: Reference source not found]
Consider of words have an alternative meaning [Error: Reference source not found]
Avoid double negatives [Error: Reference source not found, Error: Reference
14 Harris, K.J., Wheeler, A.R., and Kacmar, K.M. “Leader-member exchange and empowerment: Direct and interactive effects on job satisfaction, turnover intentions and performance”, The Leadership Quarterly, Vol.20, pp. 371-382, 2009
15 Tierney, P. “Work relations as a precursor to a psychological climate for change”, Journal of Operational Change Management, Vol.12, No.2, pp. 120-133, 1999
16 Kim, B.P. and George, R.T. “The relationship between leader-member exchange (LMX) and psychological empowerment: A quick casual restaurant employee correlation study”, Journal of Hospitality and Tourism Research, Vol.29, No.4, pp. 468-483, 2005
21 Graen, G.B., and Uhl-Bien, M. “The relationship-based approach to leadership: Development of LMX theory of leadership over 25 years: Applying a multi-level, multi-domain perspective”, Leadership Quarterly, Vol.6, No.2, pp. 219-247, 1995
source not found]
Double barrelled questions should be separated into single concept questions
[Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, Error: Reference source not found]
Avoid leading or loaded questions [Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, Error: Reference source not found]
Avoid assuming/presuming questions [Error: Reference source not found]
Imprecise conditions such as frequently, generally, normally should be avoided
[Error: Reference source not found, Error: Reference source not found]
Questions should not challenge the respondents’ knowledge, only asking what they are easily able and to answer
[Error: Reference source not found, Error: Reference source not found, Error:
32 Oxford English Dictionary Online, “definition of questionnaire”, http://www.oed.com accessed 15 Dec. 10
33 Labaw, P. Advanced Questionnaire Design, Abt Books, Cambridge, Massachusetts, 1985
34 Gendall P. “A Framework for Questionnaire Design: Labaw Revisited”, Marketing Bulletin, Vol.9, pp28-39, 1998
Reference source not found, Error: Reference source not found]
Do not ask respondents to think too far back, not more than 6 months [Error: Reference source not found, Error: Reference source not found]
Hypothetical questions are difficult to answer and should be avoided [Error: Reference source not found, Error: Reference source not found, Error: Reference source not found]
Questions that ask people to predict the future should be used with caution [Error: Reference source not found]
Although different questions require different styles of response formats, it is the attitude or
opinion statements that seem to be the most contentious. Ranges of mutually exclusive
answers, such as strongly disagree to strongly agree [Error: Reference source not found,
Error: Reference source not found] are commonly used to quantify these questions. However,
Gendall and Hoek [Error: Reference source not found] suggest that agree-disagree questions
are the most likely to be affected by question wording and, therefore, the answer format
should be a forced choice. In further work, Gendall [Error: Reference source not found]
reinforces that there should be no mid-point or neutral alternative offered, in order to measure
intensity of feeling, but states that a no opinion option should always be included.
Question Order
Question order is another issue that must be considered during the design phase of a new tool.
Unlike question wording, most authors are in agreement as to the best way to order questions.
The first and most fundamental point is to establish whether each question is in fact necessary
to complete the study [Error: Reference source not found], as the length of the questionnaire
should be kept to its optimal minimum. Once the questions are deemed to be necessary they
can be ordered based on the generally accepted advice.
The questionnaire should begin with easy, basic questions that are neither sensitive nor
threatening, in order to ease the respondent into the process [Error: Reference source not
foundError: Reference source not found, Error: Reference source not found, 37]. Questions
should then develop logically [Error: Reference source not found, Error: Reference source
not found, Error: Reference source not found], be grouped by theme [Error: Reference source
not found, Error: Reference source not found, Error: Reference source not found] and flow
smoothly from one to the next [Error: Reference source not found]. Questions that are more
sensitive or embarrassing should be left until late on in the order [Error: Reference source not
found, Error: Reference source not found, Error: Reference source not found]. Advice is
divided on where important questions should be placed, with some feeling that these
questions should be first as later responses could impact on these issues, but others suggest
that important questions should be approached slowly [Error: Reference source not found].
Similarly, division is found in the positioning of demographics questions. While Synodinos
[Error: Reference source not found] recommends that some screening questions be placed at
the end of the introduction section, he thinks that demographics questions are likely to be the
most sensitive in the questionnaire so, in line with previous advice, these should be
positioned at the end. Oppenheim [38] concurs on the positioning, although his reasoning
comes more from the desire not to dissipate the initial enthusiasm by diluting it with
questions not related to the main topic of the questionnaire. This is in direct opposition to
Drummond et al. [39], who found that placing demographics questions first actually increased
response rates in a postal survey. The effects of question order on response rates were also
found by Synodinos [Error: Reference source not found] and Dunn et al. [Error: Reference
source not found], and it was thought that this could be influenced further by the gender of
the respondent [Error: Reference source not found].
Questionnaire Presentation Formatting
Whilst much of the advice concerning questionnaire presentation is aimed at self-
administered tools that are completed as part of a postal survey, there are lessons to learn to
improve the presentation of all questionnaires based on these findings. Jepson et al. [40] found
that the overall length of the questionnaire had a direct impact on response rates with a
response rate at 60% for a questionnaire of 849 words, but only 16.7% when the words are
increased to 1800, concluding that there is an acceptable threshold for questionnaire length.
Although response rate is not an issue in organizational studies with full participation, the
findings on questionnaire length may help to ensure that focus can be maintained by those
completing the questionnaire.
Two things are likely to create an immediate impression on respondents, which makes them
vitally important. The first is the introduction, which is needed to build rapport with the
respondent [Error: Reference source not found]; it sets the scene and can build interest in
completing. Second is the graphic design of the tool itself [Error: Reference source not
found], which has the potential to either arouse interest or discourage respondents from
taking the time to complete [Error: Reference source not found].
To maximise the likelihood of completion, practical advice on questionnaire format is offered
by many, and is summarised in Table 2.
Table 2: Advice for presentation of questionnaires
Rules for presentation of questionnaires
Begin by assuring respondents of confidentiality of answers and results [Error:Referencesource notfound]
Use a large, distinct typeface/font, appropriate to the target population [Error:Referencesource notfound, Error:Referencesource notfound]
Use a different, clear typeface/font for instructions [Error:Referencesource notfound]
Instructions should be clear and explicit – illustrated with an example if necessary
[Error:Referencesource notfound]
Questions should be numbered and not split across page breaks [Error:Referencesource notfound]
Pages should be numbered, with PTO at the bottom of each page (where further pages exist)
[Error:Referencesource notfound]
Only place questions on one side of the paper [Error:Referencesource notfound]
End by thanking participants [Error:Referencesource notfound, Error:Referencesource notfound]
The Process of Development – The FACETS Questionnaire
Developing the FACETS questionnaire forms part of a two year study on employee
engagement in continuous improvement, which began with much background research on the
constructs that affect engagement and how these can be synergistically combined to create a
more successful improvement program. At the outset, the objectives and research questions
were set, purpose of the work defined and the final output agreed upon. Literature suggests
that the initial planning phase will likely take up a third to a half of the development time of
the questionnaire [Error: Reference source not found], and this has certainly proved to be true
for this questionnaire, with early phases used not only for literature review, but also initial
testing using the tools developed and recommended by previous authors.
Developing a questionnaire when well established tools exist has proven a challenging
exercise. It is essential to ensure that the new tool measures all of the topics considered
important from the existing questionnaires as effectively as the original without using their
questions. In most cases there were substantial parts of the existing questionnaires that were
considered unnecessary for this study, as they went into aspects not thought to link directly to
engagement in the CI process. In order to ensure effective assessment, the existing questions
were reviewed, but this was supplemented by the background research on the constructs and
factors that affect and make them up. Aligning the two, allowed new questions to be written
that have the potential to still measure the construct without plagiarising the work of extant
tool developers. Therefore, creating a new and, more concise tool that measures only the
aspects considered to directly influence behaviour.
Throughout the early development stages, designing the data analysis to be undertaken
[Error: Reference source not found] has always been under consideration. This will ensure
that the final questionnaire not only meets its purpose in providing knowledge on the
individual and their role in the CI program, but also that it facilitates the proof required to
validate the model and theory developed on the interconnectivity of the constructs considered
to effect CI [Error: Reference source not found].
Question Wording
Wording of questions has been carefully considered to meet as many of the suggested
guidelines as possible. At this early stage all statements have been worded positively, as per
the advice of Murray [Error: Reference source not found], but this will be carefully analysed
to ascertain if it has had any effect on results.
A decision was taken to align all sections of questions to a standard answer format. Rather
than questions, as such, attitude/factual statements were developed to measure strength of
feeling or level of agreement; all in a closed format [Error: Reference source not found]. A
five point scale was chosen, to allow for a middle position, although this is labelled as
moderate as opposed to neutral, in order that the participant not see it as a opt out. Congruent
to this, no ‘don’t know’ option has been allowed, with the assumption that if it is possible to
answer, participants will choose and if it is not they will simply leave the question
unanswered; as outlined in the instructions.
Referring back to Table 1, the steps undertaken were reviewed in-line with advice in extant
literature, to produce Table 3.
Table 3: Actions on advice for wording questions
Actions taken on wording of questions
Only used closed questions [Error: Reference source not found,
Error: Reference source not found, Error: Reference source not found]
Questions kept concise and simple [Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, Error: Reference source not found]
100% of questions below 20 words [Error: Reference source not found]
92.5% of questions 12 words or below [Error: Reference source not found]
Questions kept clear and not confusing [Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, 41]
Language kept simple but not patronising, accessible to all [Error: Reference source not found, Error: Reference source not found,
Error: Reference source not found]
No jargon, acronyms or abbreviations used [Error: Reference source not found, Error: Reference source not found]
Words carefully selected for singular meaning and common usage [Error: Reference source not found, Error: Reference source not found, Error: Reference source not found]
No questions contain double negatives [Error: Reference source not found, Error: Reference source not found]
Double barrelled questions only used when combination effects sought [Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, Error: Reference source not found]
Questions are neither loaded or assuming/presuming [Error: Reference source not found, Error:
Reference source not found, Error: Reference source not found, Error: Reference source not found]
Imprecise conditions not used, conditions such as regularly considered acceptable
[Error: Reference source not found, Error: Reference source not found]
All questions based on current knowledge and strength of feeling, easy to answer
[Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, Error: Reference source not found]
Limited hypothetical/future questions used to illustrate some vital aspects of [Error:
6 Sinclair, M.A. “Questionnaire design”, Applied Ergonomics, Vol.6, No.2, pp73-80, 1975
7 Wagner, H., Morton, S., Backhouse, C., Burns, N., and Dani, S. “Building a model of the Synergistic Effects of Constructs Affecting Engagement in Continuous Improvement Programs”, (in review - Human Resource Management Review, 2010)
8 Hackman, R., and Oldham, G.R., Work Redesign, Addison-Wesley Publishing Company, Reading, Massachusetts, 1980
9 Munz, D.C., Huelsman, T.J., Konolold, T.R., and McKinney, J.J. “Are There Methodological and Substantive Roles for Affectivity in Job Diagnostic Survey Relationships?”, Journal of Applied Psychology, Vol.81, No.6, pp795-805, 1996
10 Hackman, R., and Oldham, G.R. “Development of the Job Diagnostic Survey”, Journal of Applied Psychology, Vol.60, pp159-170, 1975
20 Cogliser, C.C., Schriesheim, C.A., Scandura, T.A., and Gardner, W.L. “Balance in leader and follower perceptions of leader-member exchange: Relationships with performance and work attitudes”, The Leadership Quarterly, Vol.20, pp. 452-465, 2009
relationship Reference source not found, Error: Reference source not found, Error: Reference source not found]
Wording effects will only really be measurable once the questionnaire has been pilot tested
on a representative sample of the population, at which stage any felt to be affecting results
will be amended.
Question Order
22 Belbin, R.M. Management Teams: Why They Succeed or Fail. Heinemann, London, 1981
23 Belbin Associates, “What are Belbin Team Role profiles?” available online (accessed 08 Jan. 10) http://www.belbin.com/rte.asp?id=10
26 Amabile, T.M., Conti, R., Coon, H., Lazenby, J., and Herron, M. “Assessing the work environment for creativity”, Academy of Management Journal, Vol.39, No.5, pp. 1154-1184, 1996
27 Mathisen, G.E., and Einarsen, S. “A Review of Instruments Assessing Creative and Innovative Environments Within Organisations”, Creativity Research Journal, Vol.16, No.1, pp. 119-140, 2004
28 Morton, S.C., Burns, N.D., and Michaelides, R. “OPEN innovation in operation: Organisational Performance and Engineering Networks”, Proceedings of the Third World Conference on POM, Manufacturing Fundamentals: Necessity & Efficiency, Tokyo, Japan, August 5-8, 2008
29 CCL, KEYS® to Creativity, Centre for Creative Leadership, KEYS publicity brochure, available at http://www.ccl.org/leadership/ assessments/KEYSOverview.aspx accessed 20 May, 2009. 2007
30 Morton, S.C., and Burns, N.D. Understanding and Overcoming Resistance to Innovation, in Creating Wealth from Knowledge: Meeting the innovation challenge, Bessant J. and Venables, T. Editors. Edward Elgar, Cheltenham. 2008
35 Brigham, F.R. “Some quantitative considerations in questionnaire design and analysis”, Applied Ergonomics, Vol.6, No.2, pp90-96, 1975
36 Gendall P. and Hoek J. “A Question of Wording”, Marketing Bulletin, Vol.1, pp25-36, 1990
The combination of previous questionnaires utilized in the early phase of research, resulted in
participants being asked to complete an arduous 237 questions. In light of this, a target was
set of 100 questions for the final tool. At this early stage more questions (120) have been
written, to be refined after initial analysis takes place [Error: Reference source not found].
40 Jepson, C., Asch, D.A., Hershey, J.C., and Ubel, P.A. “In a mailed physician survey, questionnaire length had a threshold effect on response rate”, Journal of Clinical Epidemiology, Vol.58, pp103–105, 2005
41 Gendall P. and Hoek J. “A Question of Wording”, Marketing Bulletin, Vol.1, pp25-36, 1990
4 Synodinos N.E. “The ‘art’ of questionnaire construction: some important considerations for manufacturing studies”, Integrated Manufacturing Systems, Vol.14, No.3, pp221-237, 2003
5 Murray, P. “Fundamental issues in questionnaire design”, Accident & Emergency Nursing, Vol.7, pp148-153, 1999
13 Arnold, J., Robertson, I. and Cooper, C. Work Psychology. Pitman Publishing, London, 1991
17 Tse, H.H.M., Dasborough, M.T., and Ashkanasy, N.M. “A multi-level analysis of team climate and interpersonal exchange relationships at work”, The Leadership Quarterly, Vol.19, pp. 195-211, 2008
18 Scandura, T.A. and Pellegrini, E.K. “Trust and leader-member exchange: A closer look at relational vulnerability”, The Journal of Leadership and Organizational Studies, Vol.15, No.2, pp. 101-110, 2008
19 Aryee, S. and Chen, Z.X. “Leader-member exchange in a Chinese context: Antecedents, the mediating role of psychological empowerment and outcomes”, Journal of business research, Vol.59, pp. 793-801, 2006
24 Fisher, S.G., Macrosson, W.D.K., and Sharp, G. “Further evidence concerning Belbin team role Self-Perception Inventory”, Personnel Review, Vol.25, No.2, pp.61-67, 1996
25 Partington, D., and Harris, H. “Team role balance and team performance: An empirical study”, Journal of Management Development, Vol.14, No.6, pp.35-41, 1995
31 Adams, R., Bessant, J., and Phelps, R. “Innovation Management Measurement: A review”, International Journal of Management Reviews, Vol.8, No.1, pp. 21-47, 2006
37 Dunn, K.M., Jordan, K., and Croft, P.R. “Does questionnaire structure influence response in postal surveys?”, Journal of Clinical Epidemiology, Vol.56, pp10–16, 2003
38 Oppenheim, A. N. Questionnaire Design, Interviewing and Attitude Measurement, Continuum, London, 2000
In a questionnaire that is primarily based on attitudes, it is difficult to ascertain which
questions are likely to be most sensitive for participants, with the potential for questions to
affect some more than others. In light of this, the questionnaire begins with a general section
on continuous improvement, to both ease the participant into completion [Error: Reference
source not foundError: Reference source not found, Error: Reference source not found, 42]
and provide a clear, contextual start on which to base future answers. Further sections are
then used to move through the constructs being measured in a logical [Error: Reference
source not found, Error: Reference source not found, Error: Reference source not found]
themed way [Error: Reference source not found, Error: Reference source not found, Error:
Reference source not found]; progressing from topics considered less to more challenging
[Error: Reference source not found, Error: Reference source not found, Error: Reference
source not found]. However, within each section the questions are purposely randomized so
that questions on each sub topic are split up, to encourage them to be read carefully each time
rather than creating a lead from one to the next.
The division shown on the placement of demographics questions [Error: Reference source not
found, 43, 44] has lead to an approach that allows participants in the pilot phase to choose
whether they complete these questions before or after the main body. Results of this will
dictate the final positioning of the demographics section.
Questionnaire Presentation Formatting
During the development phases, practical information offered in extant literature, as outlined
in Table 2, was kept in mind and the measures taken to meet these recommendations are
summarised in Table 4.
39 Drummond, F.J., Sharp, L., Carsin, A-E., Kelleher, T., and Comber, H. “Questionnaire order significantly increased response to a postal survey sent to primary care physicians”, Journal of Clinical Epidemiology, Vol.61, pp177-185, 2008
Table 4: Actions on advice for presentation of questionnaires
Action taken on presentation of questionnaires
Introductory section assuring confidentiality and anonymity [Error:Referencesource notfound]
Questions in Arial font [Error:Referencesource notfound, Error:Referencesource notfound]
Instructions given in Calibri font, headings Arial Bold [Error:Referencesource notfound]
Clear instructions given at the start of the questionnaire, covering all sections which are completed in the same way
[Error:Referencesource notfound]
All questions are numbered and not split across page breaks, column headings for answer options on top of each page
[Error:Referencesource notfound]
Pages numbered, PTO considered unnecessary [Error:Referencesource notfound]
Printing only on one side of the paper [Error:Referencesource notfound]
Participants thanked at end of introduction and after questions [Error:Referencesource notfound, Error:Referencesource notfound]
Although much can be learned from the literature on postal survey response rates,
information on how number of words affects response rate was considered non-influential in
this case, as the questionnaire will always be administered as part of a company based study
with participants completing on company premises, in company time. However, as with
number of questions, overall length of the questionnaire was generally considered throughout
the development stages.
The introduction to the questionnaire was given much attention, due to its potential influence
on participants [Error: Reference source not found]. Formatted on a single sheet, it outlines
the purpose of the study, what it will measure, and how the information will be presented to
management. It assures confidentiality and anonymity of data and results. The different types
of questions (demographic and research) are explained, along with instructions on how to
complete both sections, giving the choice of completion order. The introduction ends by
thanking participants.
Graphic design was also considered important [Error: Reference source not found, Error:
Reference source not found]. General appearance was kept simple, with questions
formatted in a tabular structure, using a highlight for every other question, in order to both
distinguish between questions and to draw the eye across to the correct answer line for a
question. Color was also deliberated. It was decided that the paper should be white, partly as
this was shown to make little difference in previous studies [45] and partly as it creates a clear,
simple background [46, 47]. Highlight colors were researched for their potential meaning and
their practical application in printed form.
Shortlisted colors were pale blue, pale green, yellow, orange and grey, which was included
for simplicity of questionnaire reproduction. Each has a defined meaning and possible effect
on participants and was chosen for these advantages. Blue is “often associated with depth
and stability. It symbolizes trust, loyalty, wisdom, confidence, intelligence” [Error: Reference
source not found], “Green calms and soothes the mind, stimulates creativity, is easy on the
eye” [Error: Reference source not found], “Yellow stimulates mental activity, and attracts
attention” [Error: Reference source not found], whereas “Orange increases oxygen supply to
the brain, produces an invigorating effect, and stimulates mental activity” [Error: Reference
source not found].
Piloting the Questionnaire
As literature suggests [Error: Reference source not found], an initial pre-pilot was undertaken
using staff and PhD students within the university. This allowed for early detection of basic
errors in typing and grammar. It also highlighted a small number of issues in regard to the
42 Dunn, K.M., Jordan, K., and Croft, P.R. “Does questionnaire structure influence response in postal surveys?”, Journal of Clinical Epidemiology, Vol.56, pp10–16, 2003
43 Oppenheim, A. N. Questionnaire Design, Interviewing and Attitude Measurement, Continuum, London, 2000
44 Drummond, F.J., Sharp, L., Carsin, A-E., Kelleher, T., and Comber, H. “Questionnaire order significantly increased response to a postal survey sent to primary care physicians”, Journal of Clinical Epidemiology, Vol.61, pp177-185, 2008
45 Beebe, T.J., Stoner, S.M., Anderson, K.J., and Williams, A.R. “Selected questionnaire size and color combinations were significantly related to mailed survey response rates”, Journal of Clinical Epidemiology, Vol.60, pp1184-1189, 2007
46 Sibagraphics, “The Meaning of Colour in Web Design”, Available online at http://www.sibagraphics.com/colour.php accessed 24 January 2011
47 Color Wheel Pro. “See Color Theory in Action”, http://www.color-wheel-pro.com/color-meaning.html accessed 24 January 2011
instructions given and wording in a couple of questions. These issues were rectified and the
questionnaire was then considered ready to be used. Information from pre-pilot participants
led to the decision to use a questionnaire with blue highlighting during the pilot phase.
The next stage in the development will be a pilot at the organisation being studied, with a
small sample, representative of the general population to be studied. At this stage the
wording, question order and presentation will be assessed for their suitability in meeting the
requirements for a fully commercial tool.
From there, the questionnaire will be completed by large sections of the shop-floor
populations of two of the organization’s larger plants, in order to provide a wide testing
ground in different cultures to give the data needed to assess reliability and robustness of the
tool.
Acknowledgment
Research was supported by the EPSRC Loughborough Innovative Manufacturing and
Construction Research Centre http://www.lboro.ac.uk/imcrc
References
Top Related