Accuracy or consequential validity: which is the better standard for job analysis data?

Accuracy or consequential validity: which isthe better standard for job analysis data?

JUAN I. SANCHEZ1* AND EDWARD L. LEVINE2

1Florida International University, Miami, Florida, U.S.A.2University of South Florida, Tampa, Florida, U.S.A.

Summary The value of research on the accuracy of job analysis is questioned. It is argued that thetraditional criteria employed to evaluate job analysis accuracy (i.e., interrater agreementand deviations from proxy true scores) provide information of little practical value.Alternative criteria focusing on the consequences of job analysis data are suggested.Consequence-oriented criteria are clari®ed through a review of the various inferentialleaps or decision points that job analysis supports. In addition, the consequences of jobanalysis are also thought to be a function of the rules governing the making of job-analysis-based inferences which, unfortunately, are sometimes unspeci®ed in even themost molecular job analysis methodologies. Copyright # 2000 John Wiley & Sons, Ltd.

Introduction

Job analysis (hereafter referred to as JA) is the process of gathering, analyzing, and structuringinformation about a job's components, characteristics, and job requirements (Harvey, 1991;Levine, 1983; McCormick, 1976). Given the pivotal role played by JA in providing the substratafor many organizational decision processes such as those concerning selection, training, jobclassi®cation, compensation, and downsizing, it is not surprising that researchers have beenconcerned about inaccuracy in JA. Inaccurate JA information will presumably impair decisionquality and reduce individual and organizational e�ectiveness and e�ciency. In addition, intoday's highly litigious environment, an organizational decision process informed by erroneousJA data may be ®ercely challenged in court.

The literature on accuracy and bias in JA data has identi®ed multiple social, cognitive,dispositional, and even methodological artifacts that may potentially impede accuracy (seeMorgeson and Campion, 1997, for a recent review of potential sources of inaccuracy in JA).Many of these presumptive sources of inaccuracy mirror the biasing mechanisms that pervadesocial judgment as uncovered in other areas like cognitive and social psychology. For instance,the work of Tversky and Kahneman (1974) on heuristics or simplifying judgment strategies hasbeen deemed relevant to the process of gathering JA data, because raters may simplify (and hence

Copyright # 2000 John Wiley & Sons, Ltd.

Journal of Organizational BehaviorJ. Organiz. Behav. 21, 809±818 (2000)

* Correspondence to: Juan Sanchez, Department of Psychology, Florida International University, University Park,Miami, FL 33199, U.S.A.

bias) the rating task by focusing on the most vivid job events or on a relatively small andunrepresentative sample of job events (Sanchez and Levine, 1994).

Still another example of a potential source of inaccuracy in JA is cognitive categorization,which reduces the complexity of the external world so that subsequent inferences are made interms of the category as opposed to the unique features of the object (Wyer and Srull, 1981). Thispotential source of bias may also result in job incumbents perceiving their job in too small a set ofdimensions or facets (Stone and Gueutal, 1985).

The present manuscript critically examines the meaning and, perhaps most importantly, theusefulness of the notion of accuracy in JA data. To substantiate our arguments, we borrow somebasic ideas from several streams of research, especially from the literature on social judgment,albeit adapted to the case of JA. Then, we frame the notion of accuracy in JA in a di�erent light:one that focuses on its consequences.

At the outset, it seems necessary to dissect the notion of JA accuracy itself. We must cautionthat a basic assumption of any attempt to assess JA accuracy is that there is some underlying`gold standard' or unquestionably correct depiction of the job. This assumption is problematic atbest, for any depiction of a complex set of behaviors, tasks, or actions subsumed under the labelof the term job is of necessity a social construction (Connell and Nord, 1996). Guion (1965, p. 34)notes that the concept of accuracy, when de®ned in terms of proximity to a known standard, hasno legitimate meaning in psychological measurement. Even if we assume for the sake of argumentthat there is an underlying reality in the way positions are clustered into a grouping we refer to asa job title, such reality is still open to various interpretations. These interpretations are not freefrom extraneous in¯uences such as, for instance, the purposes we wish to accomplish throughforming the cluster.

Another philosophical concern is that JA has been de®ned as a way to express in words andsymbols what is done in a job and what attributes people need to accomplish the job (Levine,1983). If there is an insu�cient lexicon to capture aspects of jobs and human attributes this byitself adds inaccuracy to the job's depiction. For instance, Sanchez and Levine (1999) havepointed out that despite an increasingly team-based orientation in the design of work, traditionalJA has focused on within-job rather than between-job activities and, as a result, taxonomies forrepresenting team-based behaviors and team-oriented worker attributes are lacking. However,the concept of accuracy acquires great practical signi®cance when, instead of assuming theexistence of a gold standard, it focuses on the predictions that the measurement facilitates(Guion, 1965, p. 34). We begin our critique with a discussion of the criteria that are normallyemployed in evaluating the accuracy of JA data, followed by a description of the type ofconsequential criteria that we are instead advocating here.

Criteria for Accuracy in JA

Interestingly, calls for research on the accuracy of JA data have relevated the criteria by which thequality of data should be assessed to the background of the discussion. Often, authors havesimply assumed that the sources of inaccuracy that purportedly bias human judgment in thelaboratory also apply to the process of analyzing jobs in the ®eld.

A review of the literature reveals that, in a majority of studies, the criteria employed to evaluateJA accuracy fall into either one of the following two categories: (1) the `reliability' of the JA data,usually measured in the form of interrater reliability or interrater agreement; and (2) the degree ofcorrespondence between JA data and a set of proxy true scores presumably closer to the `real'

Copyright # 2000 John Wiley & Sons, Ltd. J. Organiz. Behav. 21, 809±818 (2000)

810 J. I. SANCHEZ AND E. L. LEVINE

value of job characteristics and requirements. Interestingly, these criteria have a parallel in theones employed in social judgment research (Funder, 1987), which fall under two similarcategories: (1) how well judges agree on their evaluations of the same stimuli; and (2) deviationsfrom an external, more realistic criterion. When applied to the JA domain, these approaches aresomewhat short-sighted, because they ignore the impact of JA data on the actual decisions thatJA supports. Next, we take a closer look at the reasons that, in our view, account for theweaknesses of these two criteria when evaluating JA data.

Inaccuracy as disagreement

Conventional wisdom dictates that disagreement between two judges indicates that at least one ofthem must be wrong. Thus, it is not surprising that disagreement among Subject Matter Experts'(SMEs') judgments of the same job is often thought to signal the presence of error in the JA data.There is nodoubt that SMEdisagreement feeds scepticismabout the qualityof the JAdata, therebybothering the end users of job analysis. The appearance of consensus seems to be a critical factor inthe end users' acceptability of JA results. However, as the French thinker Pascal is reported to havesaid, `there are truths on this side of the Pyrenees that are falsehoods on the other.' In JA, just likebetween observers sitting on opposite sides of the Pyrenees, accuracy may be relative not absolute.

Let us illustrate our point with an example. The correlation between ratings of task importanceand ratings of both time spent and di�culty of learning the task might vary in accordance withthe rater's experience with the task. In correlating job experience with correlations between taskratings, Sanchez (1990) reported that job experience was positively correlated with the correlationbetween importance and time spent ratings (r � 0.38, p5 0.05), whereas job experience wasnegatively correlated with the correlation between importance and di�culty of learning ratings(r � ÿ0.46, p5 0.05). Taken together, these ®ndings suggest that job experience may a�ect theway task importance is evaluated. That is, experienced SMEs seem to rely on time spent whenjudging task importance, whereas inexperienced ones rely on the di�culty of learning the task.For new employees, the memory of how hard it was (or still is) to learn tasks is fresh. On the otherhand, for experienced SMEs this memory has probably faded and, therefore, time spent on eachtask provides the simplest way to evaluate task importance. The level of con®dence in these cross-sectional results has been boosted by a later replication using a longitudinal design (Ford et al.,1991). The question is, which raters provide the most accurate rendition of task importance? Onemay argue that experienced SMEs are more job-knowledgeable and therefore their judgments arecloser to the truth. However, if the JA data are used to design a training programme, newcomers'judgments, which are a�ected by still fresh memories of di�culty of learning, are probably best.Thus, accuracy may be less critical than validity of the data for the purpose at hand.

Let us examine another example. In a JA of sales representatives in charge of placing temp-orary personnel, a pattern of task ratings was found for a subset of incumbents who produced$50,000 more a year in sales performance that was di�erent from the average incumbent in a lessproductive group (Sanchez et al., 1998). When selecting new sales reps, should this organizationsearch for individuals who ®t the job characterization that results from aggregating across all taskpro®les or, on the contrary, select individuals who ®t the job pro®le associated with highest sales?Similar ®ndings have been reported by Borman, Dorsey, and Ackerman (1992), who found thathigh performers declared di�erent time spent amounts for some tasks than low performers. In thestudies just described, the answer to the question of which set of JA data is more accurate shouldbe informed primarily by the consequences of the JA data on training and selection of the moste�ective performers rather than by perceptions of which set of judgments is most accurate.


ACCURACY IN JOB ANALYSIS 811

Equating disagreement with inaccuracy is often done in a discourse that is consistent withthe principles of classical reliability theory. However, contrary to their synonymous meaning inthe English language, errors in classical reliability theory do not need to be mistakes, whichWebster's New Collegiate Dictionary (1976) de®nes as à wrong action or statement proceedingfrom faulty judgment'. Researchers sometimes fail to distinguish between measurement errorsand mistakes, hence assuming that disagreement is a sure sign of mistaken judgment in at leastone of the parties. Instead, disagreement may simply indicate systematic depictions of alternatebut equally valid views. At least in the JA arena, and as underscored also by Morgeson andCampion (1997), disagreement does not always represent inaccuracy.

Once we establish that sources of disagreement in JA ratings may not always be spurious, itshould be obvious that examining such di�erences may yield fruitful insights for human resourceprogrammes. However, an atheoretical or `shotgun' examination of rating di�erences as afunction of available demographic characteristics (e.g., race, gender) is not likely to clarify themeaning of disagreement. Even when statistically signi®cant di�erences are found, the e�ect sizestend to be small, and their practical signi®cance questionable (Arvey, Passino and Lounsbury,1977; Arvey et al., 1982; Landy and Vasey, 1991; Schmitt and Cohen, 1989). Following Harvey'slead (1991), we advocate a thorough scrutiny of within-job disagreement that goes beyondclassical reliability theory; speci®cally, a generalizability-theory frame of mind is needed touncover the sources of variance behind within-job di�erences. In an example of the kind of studythat we are suggesting here, Sanchez, Zamora, and Viswesvaran (1997) found that agreementbetween incumbents and non-incumbents was moderated by job complexity and job satisfaction,with agreement being highest when jobs were not complex and incumbents were not highlysatis®ed. Such ®ndings illustrate that understanding the sources of disagreement may be morepractically and theoretically meaningful than simply quantifying the level of disagreement.

Inaccuracy as deviation from a `true score'

As noted by Funder (1987), studying human error is always challenging because it requires acriterion or, in other words à presumption that the actual correct state of a�airs can be knownwith certainty' (p. 76). The availability of criteria a�orded by the laboratory setting, wheredeterminations of correct responses to experimental tasks are made with absolute certainty, hasprobably fueled the sheer volume of research on human error in probability judgment (Tverskyand Kahneman, 1974). In contrast to the laboratory, the absence of sound criteria plagues the JAdomain. Thus, it is not surprising that research concerned with accuracy in JA data has oftenrelied on comparisons between SMEs' judgments and externally derived, presumably morerealistic criteria. This line of research continues the social±experimental tradition of usingcontrived stimuli in experimental settings to test the biasing processes thought to underlie personperception (e.g., Einhorn and Hogarth, 1981; Fiske and Taylor, 1984; Hogarth, 1981; Tverskyand Kahneman, 1974). In fact, because human judgment is central to many JA methodologiesthat rely heavily on SMEs, a myriad of potential distortions of judgment accuracy uncovered inlaboratory settings are deemed applicable to the JA domain (Morgeson and Campion, 1997).

Although laboratory research on judgmental error portrays human judgment as ìncorrigiblyinaccurate and error prone' (p. 395, Kruglanski, 1989), several authors have taken issue with thevalue of such research (Funder, 1987; Einhorn and Hogarth, 1981; Kruglanski, 1989; Kruglanskiand Azjen, 1983). It appears that errors in the laboratory and inaccuracy in a larger sense may notbe interchangeable, as many of such èrrors' may have functional or adaptive value in the`continuous' environments in which judgments need to be made (Hogarth, 1981). Similarly,



classifying SME judgments as erroneous simply because they deviate from a questionable set ofproxy true scores is precarious when the consequences of the presumed inaccuracy on actualorganizational decisions remain undetermined. In fact, we argue that JA accuracy should bede®ned by the consequences of the data and that, given the lack of `true score' criteria in ®eld JA,de®nitions of accuracy that do not observe this premise have little utility. As Hogarth (1981)pointed out, `. . . judgment is primarily exercised to facilitate action' (p. 199). Thus, theconsequences of judgment are the proper standards for assessing judgment quality.

Take the case of broad categorizations of primary job responsibilities. These broad statementsare not less accurate than very detailed task inventories including hundreds of tasks if the users ofsuch data sets are simply interested in establishing wide compensation bands or pay grades, as isthe case at the Saturn division of General Motors, which relies on only ®ve job classi®cations.Under these circumstances, one may even argue that the broad job responsibilities are more cost-e�ective than the costly and cumbersome task inventories. Of course, the detailed accounts of jobtasks may come in handy when restructuring tasks around work ¯ow, hence illustrating that,when examining the consequences on work process redesign, the detailed task inventories aremore accurate than the broad job responsibility statements. Interestingly, the debate provoked bythose who advocate replacing traditional job analysis with `competency modelling', whichpromotes seemingly broader depictions of job requirements, has rarely been framed along thelines of the consequences of either approach, with the possible exception of the potential legalchallenges triggered by broadly de®ned job requirements or competencies. In essence, theaccuracy of the JA data is de®ned by their e�ects, and it cannot be determined in the absence ofknowledge of such e�ects.

Because traditional accuracy research has assumed the veridicality of a set of proxy true scores,the computation of di�erence scores or deviations from the proxy true scores has been commonin many accuracy studies. Even though di�erence scores have been criticized due to their poormeasurement properties (Cascio and Kurtines, 1977; Cronbach, 1955; Edwards and Parry, 1993),the alternatives to di�erence scores like polynomial regression do not solve the focal problem,which is centered on the dubious utility of the proxy true scores, not on the means by whichdeviations from such scores are computed. That is, if the proxy criterion itself is conjectural andtherefore subject to revision, the same judgment that previously deviated from the criterion couldlater concur with the revised criterion and thus be deemed correct. The sophisticated computa-tional procedures represented by polynomial regression or Cronbach's accuracy scores, however,are still useful because they deepen our insight into the sources of the deviations.

The superiority of the proxy true score is often predicated on the high level of agreementobserved among the judges that participated in the process of criterion development. Asexplained before, agreement is not necessarily a sign of utility or even positive consequences ofsuch judgments. In other cases, the proxy criterion is deemed superior to individual judgmentsbecause it represents an average of several judges, whose individual biases should cancel eachother out according to the axioms of classical reliability theory. However, unless the utility orpractical value of the average is demonstrated in consequential terms, one should remainsceptical of the proxy's superiority. In a similar vein, despite impressive SME quali®cations likelong job tenure, academic credentials and the like, the superiority of SME judgments should notbe taken for granted, unless such judgments are proven consequential.

In domains other than JA, others have preceded us in calling for pragmatic utility as a standardof accuracy (McArthur and Baron, 1983; Swann, 1984). According to the pragmatic view,judgment accuracy is a function of the extent to which the judgment leads to goal attainment orbrings a desired outcome. In a similar vein, and closer to the JA domain, Messick (1995) hasdefended an expanded view of validity that includes an examination of social consequences.



Organizational politics also advise a move towards a more consequential model of JA dataevaluation. Complaints about organizational research not receiving enough attention frompractitioners are often voiced in academic circles. Such complaints are likely to persist amongresearchers who fail to illustrate the value-added or practical impact of JA data on organizationaldecision processes. Research on JA accuracy that follows the traditional paradigm will not ®ndavid consumers among those in charge of committing the organization's resources to JA e�orts,regardless of impressive levels of SME agreement or minimal deviations from laboratory-madetrue scores. In the next section, we outline a model for the kind of evaluation that we aresuggesting here.

A Model for the Evaluation of JA Data andJA-Based Inferences

JA is essentially a tool intended to facilitate inferences involving job requirements. Thus, theevaluation of JA data should focus on: (1) the inferences derived from such data; and (2) the rulesgoverning the making of such inferences.

Inferences supported by JA data

Gatewood and Feild (1994) identi®ed four inferential leaps that JA directly or indirectlyfacilitates. We have modi®ed Gatewood and Feild's model to accommodate uses of JA other thanpersonnel selection and also additional inferential leaps not covered in their original classi®ca-tion. A graphic representation of these inferential leaps is provided in Figure 1. However, theprimary inferences supported by JA still are:

. Work±worker attribute leap. Work-related information (e.g., tasks, work behaviors, criticalincidents) is translated into human attributes as Knowledge, Skills, and Abilities (KSAs).

. Worker attribute±organizational intervention leap. Human attributes or KSAs are translatedinto selection instruments (e.g., tests, interviews, application blanks), training programmes, orother interventions that presumably tap into such KSAs.

Figure 1. A model for the evaluation of JA data and JA practices



. Work±performance measure leap. Work-related information (e.g., tasks, critical incidents) istranslated into employee performance measures (e.g., performance appraisals, productivityassessments).

. Organizational intervention±performance measure leap. The inferences made using selectioninstruments or the impact of training programmes are examined by validating them vis-a-visperformance measures.

Interestingly, inferences vary along a continuum ranging from those that are immediatelysupported by JA data (e.g., inferring worker attributes from task inventories) to those that aremore distally supported by JA data (e.g., inferring the validity of JA-based selection instrumentsby examining their correlations with performance measures). Classifying consequences of JAalong this continuum is consistent with the distinction between immediate and intermediatecriteria drawn by Thorndike (1949). The ultimate criterion embodies everything that ultimatelyde®nes success on the job. Because the ultimate criterion is not subject to perfect measurement,JA researchers interested in evaluating the quality of their data would need to de®ne conse-quences in more tangible terms and in a shorter time frame. Although the ultimate criterionre¯ects long-term success and is practically unavailable, researchers should remain cognizant ofthe limitations of examining only immediate and intermediate consequences of JA data. Suchconsequences provide only partial information about the impact of JA on decision making.

In general, the further away from immediate consequences, the more meaningful the criterion.For instance, studying the impact of work data (e.g., di�culty of learning the job tasks) on thecontent of training programmes does not appear to be as practically impactful as studying theindirect e�ect of this kind of JA data on performance changes resulting from the trainingprogramme. For the sake of illustration, let us outline the design that such a study may take.There should be more than one group, or at least a group whose training was based on JA data,including di�culty of learning job tasks and a second group whose training programme wasinformed by no JA data or by other formats of JA data. The evaluation will focus on perform-ance measures that should be a�ected by the training programme, and one would expect betterresults for the group whose training was designed according to the information on di�culty oflearning job tasks. Of course, the longer (i.e., more intermediate) the inferential leap, the morelikely it is that interpretations of the evaluation results are threatened by confounds andalternative explanations. For example, di�erences in performance between groups may be due toattrition in one of the groups rather than to the quality of the JA data used to decide theirtraining content.

An example of the type of consequence-oriented evaluation that we are endorsing here isprovided by Levine, Ash, and Bennett (1980), who showed that di�erent depictions of jobsanalyzed by di�erent methods led human resource professionals to develop very similarexamination plans in the selection context. However, does a null result like this necessarily meanthat the two di�erent types of JA employed in that study were equally `accurate'? We are afraidnot. One of these JA data sets may be, at least potentially, more consequential than the other,even though the speci®c inferences drawn from the JA data in question failed to turn in suchconsequences. The next section explains the importance of clearly de®ning the inferential rulesgoverning the JA process.

The rules governing JA-based inferences

Contrary to the conclusions often drawn in summaries of this kind of research, the absence ofsigni®cant di�erences between JA-based outcomes (e.g., job classi®cation) produced by di�erent



JA methodologies varying in the degree of detail (Cornelius, Schmidt and Carron, 1984; Sackett,Cornelius and Carron, 1981) does not imply that detailed JA produces results that are necessarilyequivalent to those of cursory JA. The lack of signi®cant di�erences between the outcomesgenerated by the two JA methodologies may be a function of the rather simple goals of the targetJA-based inferences (e.g., create job groupings or families). When the organization wishes tomake more complex inferences (e.g., decide the content of selection procedures), strict rulesgoverning JA-based inferential leaps may actually pay o�. In other words, if the organizationadopts speci®c rules concerning the making of linkages between JA data and JA-basedorganizational products, previously inconsequential JA data may turn fruitful results.

Take for instance the elaborate procedures for establishing linkages between JA data andtraining programmes outlined by Goldstein (1993). If the organization follows such procedures,chances are that the level of descriptive detail that was formerly inconsequential may now turnout training programmes whose content di�ers markedly from previous ones. In a similar vein,Levine (1983) speci®ed a series of cut-o� scores on various KSA scales so that KSAs to be used inselecting applicants can be separated from those to be targeted by training programmes.Selection procedures designed according to the rules established by these cut-o� scores are likelyto di�er from selection procedures that leave these choices to loosely de®ned criteria like`professional judgment,' regardless of the fact that both procedures may be based on the same JAdata. Another example of inference-making aid is the use of a two-way matrix to facilitate thegeneration of hypotheses concerning linkages between job activities and KSAs or between KSAsand selection procedures (Arvey, Salas and Gialluca, 1992; Drauden and Peterson, 1974; Guion,1980; Sanchez and Fraser, 1994). Thus, evaluations of JA need to examine not only theconsequences of JA data, but also the practices and the rules governing the way these data areused to support the various inferential leaps.

In our role of reviewers of selection procedures undergoing legal scrutiny, we have been puzzledby instances in which very detailed JA data are divorced of any rules by which to turn such datainto relevant selection procedures. Levine et al. (1991) have pointed to the need for greaterattention to standards for job analysis that would speak to such issues. This absence should belamented because the long hours spent in carrying out detailed JA fail to bring a proportionalreturn on investment. In addition, the failure to demonstrate that detailed JA does indeed mattermay raise scepticism about the need to invest in fruitless yet time-consuming procedures.

In summary, we hope that the arguments introduced here will stimulate a re-examination of thevalue of research on the accuracy of JA. Please note that we do not wish to argue that accuracy inJA does not matter; quite the contrary, we maintain that accuracy does not exist in a vacuum butit is de®ned precisely by the consequences of such data. In our opinion, rethinking the notion ofJA accuracy implies carrying out evaluation studies centered on the consequences of JA data andthe consequences of rules governing JA-based inferences on organizational programmes likeselection and training.

References

Arvey RD, Davis GA, McGowen SL, Dipboye RL. 1982. Potential sources of bias on job analyticprocesses. Academy of Management Journal, 25: 618±629.

Arvey RD, Passino EM, Lounsbury JW. 1977. Job analysis results as in¯uenced by sex of incumbent andsex of analyst. Journal of Applied Psychology, 62: 411±416.



Arvey RD, Salas E, Gialluca KA. 1992. Using task inventories to forecast skills and abilities. HumanPerformance, 5: 171±190.

Borman WC, Dorsey D, Ackerman L. 1992. Time-spent responses as time allocation strategies: relationswith sales performance in a stockbroker sample. Personnel Psychology, 45: 763±777.

Cascio WF, Kurtines WL. 1977. A practical method for identifying signi®cant change scores. Educationaland Psychological Measurement, 37: 889±895.

Connell AF, Nord WR. 1996. The bloodless coup: the in®ltration of organization science by uncertaintyand values. Journal of Applied Behavioral Science, 32: 407±427.

Cornelius ET, Schmidt FL, Carron TJ. 1984. Job classi®cation approaches and the implementation ofvalidity generalization results. Personnel Psychology, 37: 247±260.

Cronbach LJ. 1955. Processes a�ecting scores on `understanding of others' and `assumed similarity'.Psychological Bulletin, 52: 177±193.

Drauden GM, Peterson NG. 1974. A Domain Approach to Job Analysis, Minnesota Department ofPersonnel, Test Research and Development Section: St. Paul.

Edwards JR, Parry ME. 1993. On the use of polynomial regression equations as an alternative to di�erencescores in organizational research. Academy of Management Journal, 36: 1577±1613.

Einhorn HJ, Hogarth RM. 1981. Behavioral decision theory: process of judgment and choice. AnnualReview of Psychology, 32: 52±88.

Fiske ST, Taylor SE. 1984. Social Cognition, Addison-Wesley: Reading, MA.Funder DC. 1987. Errors and mistakes: evaluating the accuracy of social judgment. Psychological Bulletin,101: 75±90.

Ford JK, Smith EM, Sego DJ, Quinones MA. 1991. Impact of task experience and individual factors ontraining-emphasis ratings. Journal of Applied Psychology, 78: 583±590.

Gatewood RD, Feild HS. 1994. Human Resource Selection, 3rd edn, The Dryden Press: Orlando.Goldstein IL. 1993. Training in Organizations, 3rd edn, Brooks/Cole: Paci®c Grove, CA.Guion R. 1965. Personnel Testing, McGraw-Hill: New York.Guion RM. 1980. Job Analysis and Validation Procedures: Applications to Construct Identi®cation. ThirdAnnual Conference on EEO Compliance and Human Resource Utilization. Chicago, IL, May.

Harvey RJ. 1991. Job analysis. In Handbook of Industrial and Organizational Psychology, Dunnette MD,Hough LM (eds.), 2nd edn, Consulting Psychologists Press: Palo Alto, CA; 71±163.

Hogarth RM. 1981. Beyond discrete biases: functional and dysfunctional aspects of judgmental heuristics.Psychological Bulletin, 90: 197±217.

Kruglanski AW. 1989. The psychology of being `right': the problem of accuracy in social perception andcognition. Psychological Bulletin, 106: 395±409.

Kruglanski AW, Ajzen I. 1983. Bias and error in human judgment. European Journal of Social Psychology,19: 448±468.

Landy FJ, Vasey J. 1991. Job analysis. The composition of SME samples. Personnel Psychology, 44: 27±50.Levine EL. 1983. Everything You Always Wanted to Know about Job Analysis, Mariner: Tampa, FL.Levine EL, Ash RA, Bennett N. 1980. Exploratory comparative study of four job analysis methods. Journalof Applied Psychology, 65: 524±535.

Levine EL, Sistrunk F, McNutt KJ, Gael S. 1991. Exemplary job analysis systems in selected organizations:a description of process and outcomes. In Applying Psychology in Business: The Manager's Handbook,Jones JW, Ste�y BD, Bray DW (eds.). Lexington Books: Lexington, MA.

McArthur LZ, Baron RM. 1983. Toward an ecological theory of social perception. Psychological Review,90: 215±238.

McCormick EJ. 1976. Job and task analysis. In Handbook of Industrial and Organizational Psychology,Dunnette MD (ed.). Rand-McNally: Chicago, IL; 651±696.

Messick S. 1995. Validity of psychological assessment. American Psychologist, 35: 1012±1027.Morgeson FP, Campion MA. 1997. Social and cognitive sources of potential inaccuracy in job analysis.Journal of Applied Psychology, 82: 627±656.

Sackett PR, Cornelius ET, Carron ET. 1981. A comparison of global judgment versus task-orientedapproaches to job classi®cation. Personnel Psychology, 34: 791±804.

Sanchez JI. 1990, April. The E�ects of Job Experience on Judgments of Task Importance. Paper presented atthe Annual Conference of the Society for Industrial and Organizational Psychology, Miami.

Sanchez JI, Fraser SL. 1994. An empirical approach to identify job dutyÐKSA linkages in managerialjobs: a case example. Journal of Business and Psychology, 8: 309±325.



Sanchez JI, Levine EL. 1999. Is job analysis dead, misunderstood, or both? New forms of work analysisand design. In Evolving Practices in Human Resource Management: Responses to a Changing World ofWork, Kraut AI, Korman AK (eds). Jossey-Bass: San Francisco; 43±68.

Sanchez JI, Levine EL. 1994. The impact of raters' cognition on judgment accuracy: an extension to the jobanalysis domain. Journal of Business and Psychology, 9: 47±57.

Sanchez JI, Prager I, Wilson A, Viswesvaran C. 1998. Understanding within-job title variance in job-analytic ratings. Journal of Business and Psychology, 12: 407±419.

Sanchez JI, Zamora A, Viswesvaran C. 1997. Moderators of agreement between incumbent and non-incumbent ratings of job characteristics. Journal of Occupational and Organizational Psychology, 70:209±218.

Schmitt N, Cohen SA. 1989. Internal analysis of task ratings by job incumbents. Journal of AppliedPsychology, 74: 96±104.

Stone EF, Gueutal HG. 1985. An empirical derivation of the dimensions along which characteristics of jobsare perceived. Academy of Management Journal, 28: 376±396.

Swann WB. 1984. Quest for accuracy in person perception: a matter of pragmatics. Psychological Review,91: 457±477.

Thorndike RL. 1949. Personnel Selection: Test and Measurement Techniques, Wiley: New York.Tversky A, Kahneman D. 1974. Judgment under uncertainty: heuristics and biases. Science, 185: 1124±1131.

Webster's New Collegiate Dictionary. 1976. G. & C. Merriam Company: Spring®eld, MA.Wyer RS, Srull TK. 1981. Category accessibility: some theoretical and empirical issues concerning theprocessing of social stimulus information. In Social Cognition: The Ontario Symposium, Higgins ET,Herman CP, Zanna MP (eds). Erlbaum: Hillsdale, NJ; 161±197.



Accuracy or consequential validity: which is the better standard for job analysis data?

Documents

Transcript of Accuracy or consequential validity: which is the better standard for job analysis data?