Job Satisfaction

Running head: COMPARISON OF METHODS ASSESSING JOB SATISFACTION

Comparison of Four Methods to Assess Job Satisfaction

Montclair State University

Assessing Job Satisfaction 2

Abstract

Job satisfaction is an important construct not only to researchers but to the general public.

There are many measures that are available to organizations, psychologists, and

researchers. The Job Descriptive Index, the Job Diagnostic Survey, the Job Satisfaction

Scale, and the Minnesota Satisfaction Questionnaire are all discussed here. We find the

reliabilities on these measures, including the coefficient alpha, are fairly high, however

the construct validity is often lacking. This is due to the inability to separate job

satisfaction from various other related constructs. In the future, those using these

assessments must be careful to account for this.


Introduction

Job satisfaction is not only a construct that industrial / organizational

psychologists attempt to study, but it is a facet of life that all of us are aware of. We can

define job satisfaction in a succinct manner: the amount of which a person likes their job.

As we measure job satisfaction, it is usually not only measured in a large, overall score,

but it is broken up into different facets as well. Often, employers can use these specific

facets to really understand what is going on with their organization, and thus can take

steps to correct it.

Because job satisfaction is perhaps the most studied aspect of organizations,

researchers have needed to develop tools that are not only easy and rational to administer,

but still remain robust. This means that researchers and psychologists have to struggle

between the hundred questions, 20 construct monster tests and the 8 question, 3 facet

short tests. The technical advantages and disadvantages of four of the main tests used in

this manner will be discussed here. These include the Job Descriptive Index (JDI), the

Job Diagnostic Survey (JDS), the Job Satisfaction Scale (JSS), and the Minnesota

Satisfaction Questionnaire (MSQ).

Measures

Job Descriptive Index

The JDI is one of the main measures a psychologist may use to test job

satisfaction. The JDI is based on theory that shows there are five main traits that need to

be measured for job satisfaction; work on present job, supervisor, coworker, pay, and

opportunity for promotions (Donova, Drasgow, & Probst, 2000). According to Nagy

(2002), the JDI consists of 72 items that make up the previously mentioned five sub-


scales. He also explains that it is a self-report questionnaire that is completed via pencil

and paper. Items on the questionnaire are short; a phrase is usually given with a choice

for three responses, such as “yes,” “no,” or “unsure” (Donova et al., 2000).

Job Diagnostic Survey

The JDS is another major measurement that researchers can use to determine job

satisfaction. This instrument is based on Job Characteristics Theory by Hackman and

Oldham in which they propose jobs should create high motivation, satisfaction, and

performance (Barnabe & Burns, 1994). The test comprises of 83 items on 5 subscales:

skill variety, task identity, task significance, autonomy, and feedback (Cordery and

Sevastos, 1993). Spector and Fox (2003) describe that each item on the questionnaire has

7 choices, with a high score representing a high level of the characteristic it is intending

to measure.

Job Satisfaction Scale

The JSS can be administered in two formats, either the short or the full item scale.

As Sharma et al. (1997) describe the measure; the full scale has a total of 78 items

categorized into seven subscales. These include: work, supervision, coworkers, pay and

promotion, work environment, training, and position. Each subscale consists of 7 to 19

items that reflect that construct. It is a self-report, likert scale where each worker

describes their job by answering the item on a scale of 1 to 3, with 1 being not at all and 3

being very much. This version is older and is somewhat outdated; also its authors admit

the psychometric properties are in question. More important is the shorter version,

introduced by Koeske, Kirk, Koeske, & Rauktis (1994). It consists of just 14 questions,

targeting the human service field, and has only three subscales: intrinsic satisfaction,


organizational satisfaction, and salary and promotion (Brewer & Clippard, 2002). This

version will mostly be discussed here.

Minnesota Satisfaction Questionnaire

The MSQ is another widely used questionnaire in regards to job satisfaction.

There are two forms of the MSQ that can be discussed, the long version and the short

form. The long MSQ consists of 100 items sampling 20 scale areas: ability utilization,

achievement, activity, advancement, authority, company policies and practices,

compensation, co-workers, creativity, independence, moral values, recognition,

responsibility, security, social service, social status, supervision-human relations,

supervision-technical, variety, and working conditions. Each of these scales has 4 items,

and a worker rates their satisfaction from very dissatisfied to very satisfied. Each of these

answers is given a score from 1 to 4, which is then multiplied by the number of items to

get the scale score. Overall scores would have very dissatisfied being 50 to 150, while a

very satisfied score would be 351-400 (Demato & Curcio, 2004). The short form of the

MSQ is very similar, except it only has one question for each of the 20 scale areas (Tang

& Gilbert, 1995).

Reliability

Test-retest

Test-retest reliability measures how well an instrument measures what it intends

to measure over time. It is often reached by giving a test to a group, then giving the test

to the same group at a later date.

JSS. In Koeske et al.’s (1994) study, the JSS was administered to groups in nine

month and 15 month intervals. They found a reliability of r=.80 for the nine months


between times two and three of administration, and an r=.64 for the fifteen month interval

between administrations three and four. Also, despite having relatively no data for

psychometric properties, Sharma et al. (1997) report that over 6 months in a pilot study

they find the original, long JSS to be reliable.

MSQ. Demato and Curcio (2004) report test-retest correlations for various scales

of the MSQ, but not the full measure. They say these scores ranged from .66 on the co-

workers scale to .91 for the working conditions scale on a one week retrial. After one

year, scores ranged from .35 on the independence scale to .71 on the ability utilization

scale.

Internal Consistency

Internal consistency is a measure that shows us how each item on the assessment

asses the same characteristic, in this case, job satisfaction. It is often measured as

“Cronbach’s Alpha.”

JDI. In his study, Nagy (2002), reports α on the various facets of the JDI. This

includes scores of .83 for work, .84 for pay, .86 for promotions, .89 for supervision,

and .90 for coworkers.

JDS. Cordery and Sevastos (1993) report α scores for the revised JDS on each

subscale. They report scores of .79 for autonomy, .77 for task identity, .80 for skill

variety, .75 for task significance, and .78 for feedback. In addition, Spector and Fox

(2003) report α scores of greater than .70 for each subscale.

JSS. Brewer and Clippard (2002) report α scores for the JSS overall to be

between .83 and .91. For the intrinsic satisfaction subscale, this number is between .85

and .90, while the organizational satisfaction subscale shows scores of .78 to .90.


MSQ. Demato and Curcio (2004) report scores of α in-between .73 to .94 for the

20 scales, while α is .97 for the overall scale. In another study, Scarpello & Vandenberg

(1992) find α level of .87 for the overall score on the short version of the MSQ.

Validity

Construct Validity

Construct validity of a measure shows us if a measure is measuring the construct

or theoretical concept it intends to measure. One of the ways we can define this is

through convergent validity, which tests a measure against measures that are known to

measure the same construct. On the other hand, discriminant validity makes sure a

measure does not measure a construct it should not be measuring.

JDI. The JDI has been tested against various other tests of job satisfaction, and

held up well against them. For example, Nagy (2002) finds that the JDI and a single-item

measure of job satisfaction correlate significantly, r=.65, p<.01. Unfortunately, multiple

studies find the JDI can be measuring positive and negative affect as well. In their study,

Cropanzano, James, & Konovsky (1993) find positive affectivity relates to job

satisfaction at a 0.40 rate, while negative affectivity relates to job satisfaction at a -0.17

rate. However, in Connoly & Viswesvaran (1999), they report that in fact the JDI does a

better job of capturing the variance between affectivity and job satisfaction then other

measures of satisfaction. This may be in part to the years of refinement and the JDI’s

ability to really capture the construct of job satisfaction.

JDS. The JDS also has been tested against other measures of job satisfaction to

try and establish convergent validity. Spector and Fox (2003) find that the JDS correlates

with job satisfaction at a .21 rate. In addition, they test discriminant validity, and find the


JDS autonomy scale correlates significantly with the other scales of JDS, at a rate of .51

to .67. The supervisor scores also correlate significantly from .64 to .69.

JSS. Koeske et al. (1994) present data comparing the JSS to another scale of job

satisfaction, the Work Environment Scale. Out of the 10 subscales, seven were

significantly correlated with the JSS. These scores ranged from .26 to .67. Overall the

WES correlated .61 (p < .01) with the JSS. Testing discriminant validity, the authors say

they tested the JSS against burnout, stress, and affect measures. They state these results

show the JSS has good discriminant validity because it did not correlate highly with any

of these factors. However, they do not provide data.

MSQ. Scarpello & Vandenberg (1992) test the short form of the MSQ against

self-reported values of overall job satisfaction. These numbers correlated at a r=.524, (p

< .01) which the authors felt was a significant overlap between construct and

measurement, helping establish convergent validity. In terms of discriminant validity,

Irving, Coleman, and Bobocel (2005) report results of a strong relationship between

negative affectivity and job satisfaction. They find NA and procedural justice combined

account for 17% of the variance in job satisfaction from the short form of the MSQ. The

regression coefficient for negative affectivity in relation to job satisfaction was b=-.41, p

< .001.

Content validity.

JDS. Barnabe and Burns (1994) bring up some concerns in regards to content

validity. While they feel the autonomy scores measure the construct of experienced

responsibility well, they also feel the score may be experiencing some amount of error as


the measure may be measuring how the person feels in their classroom, but not overall in

their job in general.

JSS. The authors do not provide specific data on the content validity of the JSS,

however they do admit that due to the low amount of questions on their measure, the

content validity is most likely lower than they would like (Koeske et al., 1994).

MSQ. Scarpello & Vandenberg (1992) find data that suggests the content

domains of the MSQ may are not capturing a sufficient area that they are attempting to

capture. They state that parts of the MSQ may be more indicative of occupational

commitment then job satisfaction. The data they present shows people with a higher

level of career progress perceptions were 1.7 times more likely to be positive for job

satisfaction.

Other Issues and the Future

JDI

The JDI provides an interesting example of what a solid test can show us, yet still

have room for improvement in a variety of ways. One of the most interesting changes to

the JDI that I saw was in Donovan et al. (2000) where they test a computerized version of

the JDI. Their results show that the computerized version highly correlates with the usual

pen and paper version, but more testing in this area definitely needs to be done. The

other very interesting variation to the JDI was a much shortened, one question per facet

variety of the test that Nagy (2002) used. This can be a much cheaper and easier to

administer test, yet Nagy finds that it still retains much of the validity of the full JDI.

JDS


The convergent and divergent validity scores for JDS come as a concern to me.

While none of the articles showed high scores for convergent validity, they did show

fairly high scores for divergent validity. Perhaps this test needs to be re-examined in

some way, to make sure the constructs that each subscale say they are measuring really

are measuring. In addition, the results found by Barnabe and Burns (1994) may be

another reason to reexamine the scale. These two issues combined, not only the domain

of a construct but the actual construct itself, lend me to believe that I would not

recommend using this measurement over some of the other more valid measurements.

Perhaps Cordery and Sevastos (1993) have made some inroads to improving the

JDS. They feel that some of the negatively worded items give the average respondent

problems, and often are unable to accurately answer the question. While the newer,

revised JDS improves on this issue, they still see that people with higher education tend

to not make these types of mistakes as much. In the future, another revised edition could

reword the negative items and put them in plainer and simpler English so as to alleviate

these issues.

JSS

One of the issues with the JSS can be found by examining Koeske et al.’s (1994)

article, which is mostly exhaustive, yet still skims over the issue of discriminant validity.

As the JDI showed previously, issues such as affect can easily come into play on job

satisfaction measures. Likewise, authors including Brewer and Clippard (2002) find the

JSS to be correlated highly with burnout. Using multiple regression, they find a multiple

R of .50 between the facets of burnout and overall job satisfaction using JSS. This is the

only chink in JSS’s armor; based on these studies, it is a very thoroughly researched test


whose data is easily available, and that is also a rational choice for people to administer

under the appropriate circumstances.

MSQ

The MSQ, much like the JSS, is very well reviewed and widely used. However,

the size and length of the test for the long form may be prohibitive for employers and

researchers to use effectively. The short form needs to be looked at more closely, and put

through more tests regarding its reliability and validity. So far though, the short form

seems to be a better overall price verse performance indicator of job satisfaction.

Much like the other measures of job satisfaction, the MSQ suffers from poor

discriminant validity, as Irving et al. (2005) describe. Like some of the previous

measures, MSQ too often is caught measuring another construct, this time it is once again

affectivity. In revised editions, researchers must really work on a scale that will not

overlap so much with other constructs. Perhaps one or more of the scales can be

eliminated in order to do this.

Conclusion

The four measures of job satisfaction presented here are all good measures in

various ways, with at least decent psychometric properties and a few flaws. The one

thing that is evident is that the construct validity of these tests are one of the weakest

areas. This is most likely due to the inherent overlap between job satisfaction and

constructs such as overall affectivity, career satisfaction and motivation, as well as

burnout. With that being said, I would recommend using the short form of the Job

Satisfaction Scale first, with the short form of the Minnesota Satisfaction Questionnaire

second. The short length and the excellent psychometric properties of the JSS make it


my top choice for a job satisfaction scale. However, during any administration of this, I

would also want to use a very concise scale that would try to minimize the confounding

variables of burnout, affectivity, and commitment. Perhaps two questions on each of

these facets may be enough to statistically eliminate or adjust for those variables.

This research has taught me not only about each of these job satisfaction

measures, but about the construct as a whole and what other related constructs need to be

looked at. I gained an understanding of when short and long forms on the various tests

could be used, and I also learned how a few of these tests are so well versed that they can

be used to normalize other related measurements. In the future as I become an

industrial / organizational psychologist, I will be able to use these tests and learn even

more about their quirks. This has been a good start.


References

Barnabe, C., & Burns, M. (1994). Teachers’ job characteristics and motivation.Educational Research, 36(2), 171-185.

Brewer, E.W., & Clippard, L.F. (2002). Burnout and job satisfaction among studentsupport services personnel. Human Resource Development Quarterly, 13(2),169-186.

Connoly, J.J., & Viswesvaran, C. (1999). The role of affectivity in job satisfaction: a meta-analysis. Personality and Individual Differences, 29, 265-281.

Cordery, J.L., & Sevastos, P.P. (1993). Responses to the original and revised job diagnostic survey: is education a factor in responses to negatively worded items?Journal of Applied Psychology, 78(1), 141-143.

Cropanzano, R., James, K., & Konovsky, M.A. (1993). Dispositional affectivity as a predictor of work attitudes and job performance. Journal of OrganizationalBehavior, 14(6), 595-606.

Demato, D.S., & Curcio, C.C. (2004). Job satisfaction of elementary school counselors:a new look. Professional School Counseling, 7(4), 236-245.

Donova, M.A., Drasgow, F., & Probst, T.M. (2000) Does computerizing paper-and- pencil job attitude scales make a difference? new IRT analyses offer insight. Journal of Applied Psychology, 85(2), 305-313.

Irving, P.G., Coleman, D.F., & Bobocel, D.R. (2005). The moderating effect of negativeaffectivity in the procedural justice-job satisfaction relation. Canadian Journalof Behavioral Science, 37(1), 20-32.

Koeske, G.F., Kirk, S.A., Koeske, R.D., & Rauktis, M.B. (1994). Measuring the Monday blues: validation of a job satisfaction scale for the human services. Social Work Research, 18(1), 27-35.

Nagy, M.S. (2002). Using a single-item approach to measure facet job satisfaction. Journal of Occupational and Organizational Psychology, 75, 77-86.

Scarpello, V., & Vandenberg, R.J. (1992). Generalizing the importance of occupational and career views to job satisfaction attitudes. Journal of Organizational Behavior, 13(2), 125-140.

Sharma, J., McKelvey, J., Hardy, R., Epstein, M.H., Lomax, R.G., & Hruby, P.J. (1997).Job satisfaction of child welfare workers in an urban setting: status and predictors.Journal of Child and Family Studies, 6(2), 209-219.


Spector, P.E., & Fox, S. (2003). Reducing subjectivity in the assessment of the jobenvironment: development of the Factual Autonomy Scale (FAS). Journal ofOrganizational Behavior, 24(4), 417-432.

Tang, T.L., & Gilbert, P.R. (1995). Attitudes toward money as related to intrinsic andextrinsic job satisfaction, stress, and work-related attitudes. Personal IndividualDifferences, 19(3), 327-332.

Job Satisfaction

Documents

Transcript of Job Satisfaction