A MATTER OF CONTEXT: A META-ANALYTIC INVESTIGATION OF...

PERSONNEL PSYCHOLOGY2012, 65, 445–494

A MATTER OF CONTEXT: A META-ANALYTICINVESTIGATION OF THE RELATIVE VALIDITYOF CONTEXTUALIZED AND NONCONTEXTUALIZEDPERSONALITY MEASURES

JONATHAN A. SHAFFERWest Texas A&M University

BENNETT E. POSTLETHWAITEPepperdine University

The empirical evidence that has accumulated in support of the notionthat personality is a valid predictor of employee performance is vast,yet debate on the matter continues. This study investigates frame-of-reference effects as they relate to the validity of self-report measuresof personality. Specifically, we compare the validities of general, non-contextualized personality measures and work-specific, contextualizedmeasures. The findings suggest that personality measures are a morevalid predictor of performance when the scale items or instructionsare framed specifically so as to reference work-specific behaviors. Wefound that the validities for noncontextualized measures of personalityranged from .02 to .22, with a mean validity of .11. The validities forcontextualized measures ranged from .14 to .30, with a mean of .24.Additional moderator analyses were conducted in an effort to examineseveral alternate explanations for these validity differences. Specifically,we examined differences between the developmental purpose (generaluse vs. workplace use) and reliabilities of each type of personality

We thank Frank Schmidt and Amy Colbert for their helpful comments on earlier versionsof this paper. We also thank two anonymous reviewers for their comments. Finally, we thankall of the test publishers that provided us with specific information related to the personalityscales included in our analyses.

Text in bold represent corrections added on 15 May 2013 after initial online publication on2 August 2012. Values in tables and text have been corrected in order to rectify publicationbias and the validity of conscientiousness in Appendix B.

In the initial publication, Appendix B included a study that was not reflected in the meta-analysis, and omitted another study that was included in the meta-analysis. In addition,the ks, Ns, and estimates that were reported in Table 3 for four of the conscientiousnessanalyses were incorrect, and several of the rows for agreeableness also contained errors.For the most part, the errors resulted in observed correlations that were incorrect by .01,affecting the calculations for the confidence intervals of those estimates. Nevertheless, thediscussion of the authors’ findings for Table 3 is still valid. The authors would like toapologize for any inconvenience caused by the errors, and would like to thank Sven Kepesand Mike McDaniel for making us aware of the need for the corrections.

Correspondence and requests for reprints should be addressed to Jonathan A.Shaffer, Assistant Professor of Management, Department of Management, Marketing,and General Business, West Texas A&M University, Box 60809, Canyon, TX 79016;[email protected]© 2012 Wiley Periodicals, Inc.

445

446 PERSONNEL PSYCHOLOGY

measure. We also compared the validities from published studies tothose from unpublished studies. Results suggest that these moderatorsdid not have an impact on the validity differences between noncontex-tualized and contextualized measures.

We are one thing to one man and another thing to another. There are partsof the self which exist only for the self in relationship to itself. We divideourselves up in all sorts of different selves in reference to our acquaintances.We discuss politics with one and religion with another. There are all sortsof different selves answering to all sorts of different social reactions. It isthe social process itself that is responsible for the appearance of the self; itis not there as a self apart from this type of experience.

A multiple personality is in a certain sense normal . . .

—George H. Mead (1934, p. 142)

The relationship between self-report measures of personality and jobperformance has been the focus of an enormous body of research. Thoughearly studies concluded that personality was not a meaningful predictorof job performance (Ghiselli, 1973; Guion & Gottier, 1965; Schmitt,Gooding, Noe, & Kirsch, 1984), later work concluded that personalityis, in fact, a useful predictor of performance (Barrick & Mount, 1991;Salgado, 1997). The empirical evidence that has accumulated in supportof the notion that personality is a valid predictor of performance seems tobe robust enough that some researchers have gone so far as to recommendthat researchers focus their efforts on other avenues of study (Barrick,Mount, & Judge, 2001). Despite this recommendation, debate about thevalidity of self-report measures of personality continues. Proponents ofself-report personality measures argue that their validity is sufficient towarrant their use in most, if not all, selection contexts (Ones, Dilchert,Viswesvaran, & Judge, 2007). Still, others maintain that the validity ofself-reported personality tends to be “disappointingly low” and “not veryimpressive” (Morgeson et al., 2007, p. 693). In light of the disagreementsurrounding the use of personality measures in employee selection, furtherexamination of the issue is needed.

One issue that recently was addressed by Morgeson et al. (2007) isthat current self-report measures of personality do not adequately predictjob behaviors because the measures themselves may be deficient. For ex-ample, it has been shown that different personality scales do not correlatehighly with each other, even though they are based on the same personalitymodel (Hough, 1992; Hough, Eaton, Dunnette, Kamp, & McCloy, 1990).One potential solution to this issue that has been implemented in previousresearch is to apply statistical corrections for construct unreliability tothe validity estimates of personality measures (Mount & Barrick, 1995;

SHAFFER AND POSTLETHWAITE 447

Salgado, 1997). However, Schmitt (2004) suggests that such statisticalcorrections do not address the underlying issue of low observed validi-ties. Schmitt further notes that the observed validities reported in variousmeta-analyses have not changed in decades and that instead of applyingnumerous corrections for measurement error to the observed validities,researchers should instead endeavor to improve upon the personality mea-sures themselves. Specifically, it has been proposed that the validity ofpersonality for predicting workplace outcomes is lower than might beexpected due, in part, to the fact that personality measures typically arenot highly job relevant when designed to capture broad, global differencesbetween individuals as opposed to more specific, work-related differences(Robie, Schmit, Ryan, & Zickar, 2000). Morgeson and colleagues (2007)also raised concerns about the job relevance of personality measures andsuggest that simply contextualizing personality scales by making themrefer specifically to the workplace might increase their validity. The au-thors go on to offer a straightforward solution to this issue—simply add“at work” to each personality scale item.

This notion is not a new one. Previous research has suggestedthat context-specific, or contextualized, personality measures shouldbe stronger predictors of performance than broad, noncontextualizedmeasures (Hunthausen, Truxillo, Bauer, & Hammer, 2003; Robie et al.,2000; Schmit, Ryan, Stierwalt, & Powell, 1995). Schmit et al. (1995)referred to this phenomenon as the frame-of-reference (FOR) effect. FOReffects occur when responses to personality scales—and the subsequentvalidity of those scales—vary based on the specific behavioral contextthat respondents choose as a referent when completing individual scaleitems. FOR effects can present problems for studies of the validity ofpersonality because personality measures that are designed to assessbroad, noncontextualized personality may not be the most effectivepredictors of situation-specific, contextualized behavioral outcomes(Heller, Watson, Komar, Min, & Perunovic, 2007).

Thus, as it relates to the prediction of performance in an employmentsetting, the use of noncontextualized personality scales is not ideal becauseindividuals may present themselves differently across situations (e.g.,work, home, and school). Davison and Bing (2009) suggest that whenpersonality is measured for the purposes of personnel selection, “if the testtaker is uncertain as to whether the item calls for presentation of the workself or the nonwork self, then accurate self-presentation may be hindered,and the criterion-related validity of the personality test would be reduced”(p. 501). The implication here is that contextualized personality scalesmay show higher validities than do noncontextualized scales becausecontextualized scales give test takers a reference point for describing their


work-specific behaviors, and those descriptions are stronger predictors ofwork-specific performance.

Interest in the validity of contextualized measures of personality hasbeen increasing (Barrick, Stewart, & Piotrowski, 2002; Berry, Page,& Sackett, 2007; Gill & Hodgkinson, 2007; Lounsbury, Gibson, &Hamrick, 2004; Loveland, Gibson, Lounsbury, & Huffstetler, 2005), butdirect comparisons of the validity of contextualized and noncontextualizedpersonality measures have been limited mainly to academic settings (Bing,Whanger, Davison, & VanHook, 2004; Lievens, De Corte, & Schollaert,2008; Reddock, Biderman, & Nguyen, 2011; Robie, Born, & Schmit,2001; Schmit et al., 1995). The extent to which contextualizing person-ality measures increases their validity for predicting job performancehas not been explored fully. This study fills this void in the literature bymeta-analytically comparing the validity of noncontextualized personalitymeasures to those of contextualized personality measures for predictingjob performance. In the remainder of this study, we first review theoreticaland empirical work that suggests contextualized measures may be morevalid than are noncontextualized measures. Second, we discuss severalimportant moderators that may account for the hypothesized validity dif-ferences between the two types of measures. Last, we present the results ofour analyses and discuss the implications of our findings for both researchand practice.

Theory and Hypotheses

The theoretical basis for exploring FOR effects on the validity of per-sonality can be found in person–situation interaction theory (Mischel,1973). Person–situation interaction theory rests on the notion that person-ality is not necessarily a consistent predictor of behavior across varioussituations. Instead, the theory predicts that individual behavior in a givensituation is a function of both the personality of the individual and thesituation itself. From this perspective, although “some situations may bepowerful determinants of behavior, others are likely to be exceedinglytrivial” (Mischel, 1973; p. 255). Wright and Mischel (1987) later referredto this view of the relationship between personality and behavior as theconditional model of dispositions. For an example of the potential in-teraction between an individual’s personality and the situation in whichindividual behaviors are expressed, consider an individual who scores lowon Conscientiousness. This person may demonstrate highly conscientiousbehavior at work (e.g., maintaining a tidy workspace, keeping a detailedschedule, etc.), but may exhibit highly unconscientious behaviors at home(e.g., letting dirty laundry accumulate, not cleaning up after oneself, etc.).The conditional model of dispositions would predict that this individual


may be more likely to display conscientious behaviors at work becausesuch behaviors are highly valued in a work environment. At home, how-ever, the situation may offer fewer incentives for exhibiting conscientiousbehaviors. Thus, although an individual’s level of Conscientiousness maybe a good predictor of his or her behavior in some situations, the situationitself may be a stronger predictor of behavior in others. Some researchershave likened this phenomenon to the playing of a social role, arguing that(a) different situations call for individuals to display different roles, and(b) individuals’ self-perceptions of personality can systematically changebased on the role that is being fulfilled (Allport, 1961; Donahue & Harary,1998; Heller, Ferris, Brown, & Watson, 2009).

The possibility that situation-specific personality measures may bemore valid than nonspecific measures has been explored in academicand employment contexts and has met with initial support. Schmit et al.(1995) compared the validity of noncontextualized measures of Consci-entiousness to contextualized measures that had been revised to reflectConscientiousness “at school.” The authors found that the school-specificConscientiousness measure was a stronger predictor of cumulative collegegrade point average (GPA) than was the noncontextualized measure. Binget al. (2004) extended these results by showing that measures of school-specific, contextualized Conscientiousness incrementally predicted col-lege GPA above and beyond ACT scores and noncontextualized measuresof Conscientiousness. Lievens et al. (2008) assessed Conscientiousness ina group of undergraduate students using two contextualized measures spe-cific to a school setting and a workplace setting. The results showed thatwork-specific Conscientiousness was a poor predictor of GPA, whereasschool-specific Conscientiousness was a strong predictor of GPA, suggest-ing that personality measures may be most valid when they are framed toreflect the context in which the criterion of interest ultimately occurs.

Some studies have examined FOR effects on the validity of person-ality in a work setting. Hunthausen et al. (2003) assessed the Big Fivepersonality traits in two groups of customer service managers. The firstgroup completed a noncontextualized personality measure, whereas thesecond group completed a contextualized measure. The results showedthat the contextualized personality measure was a better predictor of per-formance than was the noncontextualized personality measure for four ofthe Big Five traits (Conscientiousness, Emotional Stability, Extraversion,and Openness to Experience). Furthermore, the contextualized personal-ity measures incrementally predicted job performance after controlling forcognitive ability, whereas the noncontextualized measures added no incre-mental prediction over cognitive ability. Other studies that have directlycompared the validity of contextualized and noncontextualized personality


measures have met with mixed results (DeGroot & Kluemper, 2007; Pace& Brannick, 2010; Small & Diefendorff, 2006).

In summary, the conditional model of dispositions predicts that re-vising personality measures in such a way that contextualizes them to aworkplace setting will increase their validity. A noncontextualized person-ality scale may not predict job performance as well because such scalesare not designed to capture work-specific behaviors. Based on the abovediscussion, we expect:

Hypothesis 1: The validity of personality measures that contain a FORthat is specific to a workplace context will be greaterthan the validity of noncontextualized personality mea-sures.

Moderators

It is possible that any differences between the validities of contextual-ized and noncontextualized personality measures are the result of some-thing other than the FOR contained in the measures. That is, even if thevalidities of contextualized and noncontextualized personality measuresare not equivalent, it is important to examine alternative explanations forthese differences. Thus, we identified several moderators that may influ-ence the validity of contextualized and noncontextualized personality forpredicting job performance.

Developmental Purpose

It is important to determine whether personality measures that wereoriginally designed for general purposes are as valid as measures thatwere designed specifically for use in the workplace. Although both typesof measures assess the same basic personality dimensions, those designedfor general use are intended to predict behaviors across a wide rangeof contexts, whereas those designed for workplace use are intended topredict behaviors that are more relevant to a work context. The use ofgeneral-purpose personality assessments for predicting job performancecan be suboptimal because such measures were not intended for use inpersonnel selection (Gill & Hodgkinson, 2007).

Personality scales that are designed for general use are constructedusing empirically driven, exploratory methods that are meant to identifyscale items that show high factor loadings on a given trait while alsoshowing low factor loadings on other traits. The typical approach to con-structing a personality measure designed for general use would be togenerate a large pool of representative scale items for each of the Big Five


traits and then conduct several factor analyses on those items. Throughoutthe factor analysis process, items are eliminated that do not load highlyon given traits. Items with the highest factor loading for a given trait areretained in the scale for that particular trait, the end results being a measurethat contains distinct scales for each of the Big Five traits (Digman, 1990;Goldberg, 1992; Saucier, 1994).

In contrast, personality measures that have been designed for use inthe workplace typically consist of items that have been chosen based ontheir theoretical connection to the criteria of interest: job performance.For these measures, the development process may begin with a job anal-ysis that determines the activities or duties that workers are expectedto perform. Personality scales are then constructed that consist of itemsdesigned not only to fit the five-factor model of personality but also todisplay high levels of face validity relative to the activities and duties thatare relevant to a work environment (Page, 2009; Schmit, Kihm, & Robie,2000). For example, the items “I have a rich vocabulary” and “I do thingsaccording to a plan” can be found in the International Personality ItemPool (Goldberg, 1999). However, the second item is more face valid rel-ative to job performance criteria. Thus, this latter item might be retainedby a test developer for use in a workplace measure of personality, whereasthe former item would not be retained. A second way that developers mayattempt to make a personality measure more relevant to the workplaceis by revising scale items such that they use work-related vocabulary inan effort to make the items more easily understood by job incumbentsor applicants. The test manual for one personality measure designed forworkplace use explained that during scale development the authors at-tempted to eliminate “psychological” language from items completely(Abraham & Morrison, 2009).

We note that FOR (contextualized vs. noncontextualized) and devel-opment purpose (general vs. workplace) are two independent dimensions.Thus, a scale classified as contextualized may or may not have been orig-inally developed for use in the workplace. For example, validity studieshave been conducted using personality measures that were originally de-signed for general purposes but had been subsequently revised to reflecta work-specific FOR (e.g., Halfhill, Nielsen, Sundstrom, & Weilbaecher,2005; Hunthausen et al., 2003). Other studies have used measures thatwere designed for use in the workplace that consist of noncontexual-ized items that do not explicitly measure work-specific personality (e.g.,Postlethwaite, Robbins, Oh, & Casillas, 2010; Witt & Carlson, 2006).1

1A summary of the information that we used to categorize each of the personalitymeasures included in this study, including example items and instructional anchors can befound in Appendix A.


Because measures that have been designed specifically for use in theworkplace are designed to identify worker characteristics that are mostrelevant to actual job performance (Pearlman & Sanchez, 2010), suchmeasures should be more valid predictors of job performance.

Hypothesis 2: The validity of personality measures that have been de-signed for specific use in the workplace will be greaterthan the validity of those designed for general use.

Reliability Differences

Contextualized personality measures are thought to have higher levelsof internal reliability than do noncontextualized measures (Lievens et al.,2008). This is because when responding to a noncontextualized person-ality measure, it is possible that respondents describe themselves basedon different situational referents for each test item. A respondent maydescribe their behavior at work on one item, their behavior at home onanother item, and their behavior at school on yet another item (Wang,Bowling, & Eschleman, 2010). On the other hand, when responding to acontextualized personality measure, respondents should be more likely tolimit their descriptions of themselves to the specific context to which themeasure refers. From this perspective, variance in responses to test itemsthat occurs due to respondents describing themselves based on differentsituational referents for each test item is considered error variance thatreduces the reliability of noncontextualized personality measures (Robieet al., 2000).

Only a few studies have tested whether FOR effects influence the re-liability of personality tests. Schmit et al. (1995) tested four of the BigFive traits (excluding Openness) and found that error variance was lowerfor contextualized measures of personality, which thereby resulted inhigher reliability estimates for such measures. Robie et al. (2000) extendedthis research by analyzing facet-level measures of noncontextualized andschool-specific, contextualized Conscientiousness in a large sample of un-dergraduate students. They found greater amounts of error variance—andthus, lower reliabilities—for five of the six facets when the measures werenoncontextualized. Lievens et al. (2008) examined the reliability of all ofthe Big Five traits. Their results showed that contextualizing personalitymeasures increased the reliability of the personality scales. Although fewstudies comparing the reliability of contextualized and noncontextualizedmeasures of personality have been conducted, early evidence tends tosupport the view that the higher validity of contextualized measures maybe due to their higher levels of internal reliability.


Hypothesis 3: Contextualized measures of personality will havehigher internal reliabilities than will noncontextualizedmeasures of personality.

Publication Status

Finally, the issue of publication bias as it relates to the validity ofpersonality has garnered recent interest. Publication bias refers to thenotion that only a portion of all studies conducted on a given topic hasbeen published and that the results of the published studies differ fromthose of the unpublished studies in a systematic and meaningful way(McDaniel, Rothstein, & Whetzel, 2006). Publication bias has also beencalled the “file drawer problem,” a term that refers to the possibility thatsignificant results are published and nonsignificant results are put in theback of a file drawer and forgotten (Rosenthal, 1979). Hunter and Schmidt(2004) refer to this issue as “availability bias.” The authors explain that thegeneral implication of publication bias for meta-analysis is that if studiesthat contain nonsignificant results are not readily available for inclusion ina given meta-analysis, the results of that meta-analysis will be upwardlybiased.

As it relates to this study, the topic of publication bias is of particu-lar relevance for several reasons. First, McDaniel et al. (2006) analyzedvalidity information provided by four test publishers and found initial ev-idence that publication bias existed for several of the data sets containingvalidity studies of personality. Thus, there is already some indication thatpublication bias is present in the personality validity literature. Second, itis possible that studies comparing the validity of contextualized person-ality measures to noncontextualized measures may be more likely to bepublished if those studies find significant differences between the validityof the two types of measures. More specifically, it may be the case thatpublished studies are more likely to be those studies that find higher va-lidities for contextualized measures. Based on both the conceptual natureof publication bias and the empirical evidence that is currently available:

Hypothesis 4: The validities reported in published studies of personal-ity measures will be higher than the validities reportedin unpublished studies.

Method

We conducted an extensive search for both published and unpub-lished papers to include in this meta-analysis. First, we conducted an elec-tronic search of PsycINFO, PsycARTICLES, EBSCO, Web of Science,


ProQuest Dissertations and Theses, and the Defense Technical Informa-tion Center (DTIC). The electronic search included, but was not limitedto, the following keywords: personality, Big Five, five factor, Conscien-tiousness, Agreeableness, Emotional Stability, Neuroticism, Openness,Extraversion, and job performance. Second, we conducted a manualsearch of the following journals for the time period from 1977 to 2011:Journal of Applied Psychology, Personnel Psychology, Psychological Bul-letin, International Journal of Selection and Assessment, Human Perfor-mance, Organizational Behavior and Human Decision Processes, Journalof Vocational Behavior, Journal of Organizational Behavior, Journal ofOccupational Psychology, Journal of Occupational and OrganizationalPsychology, Journal of Management, and Academy of Management Jour-nal. Third, we searched the conference programs of both the Society forIndustrial and Organizational Psychology and the Academy of Manage-ment meetings for the time period of 2004–2010 for additional unpub-lished papers. Fourth, we searched the reference sections of previouslypublished, relevant meta-analyses for studies that had not been uncoveredin our previous searches (e.g. Barrick & Mount, 1991; Hurtz & Donovan,2000; Salgado, 1997). Fifth, we contacted test publishers to request techni-cal reports. Finally, we sent out an electronic request through the Academyof Management mailing list servers for any additional unpublished datathat were available.

Inclusion Criteria

After acquiring all promising studies, we examined the abstracts andevaluated the results of each study to determine its relevance to our studypurposes. We used several decision rules in order to determine if the studyshould be included in this analysis. First, the study had to be empirical innature and had to examine job performance in a field setting. Given this,laboratory studies were excluded. Second, the study needed to include ameasure of at least one of the Big Five personality traits and a measureof job performance. Third, the study had to report sample sizes and corre-lations between personality and job performance, or enough informationthat the reported statistics (univariate F-values, t-values, chi-square values,differences scores, or means and standard deviations) could be convertedinto usable effect sizes. Fourth, the study must have reported data basedon an independent sample. We found several studies that reported resultsthat seemed to be based on the same data set. In such cases, we includedonly the study with the largest sample size. Following these criteria, weidentified a total of 90 studies from which we obtained usable data. We ob-tained multiple effect sizes from some studies; the number of effect sizesand total sample sizes available for each personality trait varies across


our analyses. We obtained 66–91 (N = 10,866–16,078) effect sizes fornoncontextualized measures of personality and 14–22 (N = 2,178–3,478)effect sizes for contextualized measures.

Description of Coding Procedures

After developing a set of coding instructions and a coding sheet,each author independently coded each of the studies included in ouranalysis.2 We held meetings periodically to cross-check and discuss ourcoding procedures, implementing changes when needed. We resolved anydisagreements through discussion. Before performing any data analyses,we reached complete agreement. To ensure the independence of our data,we recorded the uncorrected observed correlations and sample sizes aslisted in the studies. In the case of the dependent variables, some stud-ies reported several effect sizes for job performance. When this occurredwe employed three main decision rules to determine which coefficientto retain. First, we prioritized measures of supervisory ratings of overalljob performance if such ratings were available. Second, when measuresof multiple job performance facets were provided (e.g., ratings of taskperformance and contextual performance were reported separately) wecomputed a composite correlation for overall job performance wheneverpossible.3 Third, in the few cases in which multiple job performancecriteria were reported but it was not possible to compute a compositecorrelation, we averaged the correlations for the individual performancemeasures to obtain an effect size for overall job performance. Finally,one study reported correlations between personality traits and perfor-mance ratings from more than one supervisor. In this case, we averagedthe resulting correlations.

Description of Variables

Personality

Only personality measures that were designed based on the five-factormodel of personality were included in this study (Hurtz & Donovan,

2A summary of the meta-analytic database can be found in Appendix B.3To compute composite correlations, we used formula 10.11 given by Hunter

and Schmidt (2004, p. 435). This formula, where x is a single predictor variable,Y is a composite criterion measure derived from the individual criterion measuresy1, y2 . . . yi, n is the number of y criterion measures included in the compositemeasure, and r̄yiyj is the average correlation between y measures, is as follows:

rxY =∑

rxyi√n + n(n − 1)r̄yi yj

.


2000). The initial level of agreement between coding decisions for thismoderator was 95%.

Performance outcomes

We included only studies that reported supervisory ratings of job per-formance. Some studies included facet-level job performance measuressuch as task, contextual, and counterproductive work performance, but toofew, if any, of these studies reported results for contextualized measuresof personality to allow for a meaningful analysis of these criteria.

Scale characteristics

We coded each study based on whether the personality measure usedwas a noncontextualized measure or a contextualized measure of work-specific personality and whether the measure was designed for generalor workplace use. In order to be considered a work-specific, contextual-ized personality measure, a measure must have included explicit languagethat gave the items in that measure a work-specific FOR. We coded apersonality measure as being work-specific if it (a) consisted of a prepon-derance of items that included a specific reference to the workplace in theitems themselves, (b) contained instructions that directed respondents todescribe themselves exclusively in terms of their workplace behaviors, or(c) contained both items and instructions that referenced the workplace.Ninety-nine percent of our initial coding decisions were in agreement forthis moderator.

To determine whether a given personality test was designed for gen-eral or workplace use, we reviewed published papers and test manuals fordetails describing how each measure was developed. In some cases, it wasnecessary to contact the publisher of a given personality measure to obtainthis information. Measures that were intended for use in predicting a widerange of criteria for psychological, medical, or research purposes werecoded as general-purpose measures. Those measures that were based onjob analysis or were designed specifically for use in predicting job perfor-mance criteria were coded as having a workplace development purpose.Initial coding agreement was 96% for this moderator.

Publication status

Dissertations, theses, conference papers, and validity studies providedby test publishers were coded as unpublished studies. Studies found injournals were coded as published studies.


Meta-Analytic Procedure and Artifact Corrections

We analyzed our data using the methods developed by Hunter andSchmidt (2004). This method computes the sample-weighted mean of theobserved correlations and observed standard deviations from the originalstudies and then corrects them for statistical artifacts, including predictorunreliability, criterion unreliability, and range restriction. Because notall studies in our data set included artifact information, we computedseparate artifact distributions for each of the Big Five traits based on thedata available in our data set. Because it is extremely unlikely that theindividuals in our sample were selected top-down based on personalityscores, corrections for direct range restriction were not appropriate for usein this study. Therefore, in order to correct for range restriction we usedthe procedures for correcting for indirect range restriction outlined byHunter and Schmidt (2004). We computed a separate ux value for each BigFive trait by combining data from each of the studies in our data set withthe normative data provided in the test manuals for the various personalityscales that were included in our data set. This procedure resulted in ux

values that ranged from .91 (Conscientiousness and Agreeableness) to .93(Emotional Stability and Openness). These estimates are virtually identi-cal to those reported in previous meta-analyses (Barrick & Mount, 1991;Hurtz & Donovan, 2000). We also computed estimates of predictor relia-bility. We found mean alpha reliabilities that ranged from .78 (Opennessto Experience) to .85 (Emotional Stability). Our estimates are very similarto the estimates reported by Hurtz and Donovan (2000), which werealso based on explicit Big Five measures. Although we did not correctfor unreliability in the predictor, predictor reliabilities were used to testHypothesis 3 and in the process of correcting for indirect range restriction.

In order to correct for unreliability in job performance ratings, wefirst attempted to derive an estimate of mean interrater reliability from thestudies included in our data set. However, only one study reported suchdata. Because interrater reliability estimates were largely unavailable inour data set, we elected to correct for criterion unreliability using a meta-analytic estimate given in previous research. In determining what estimateof interrater reliability to use in our study, we considered the resultsfrom several meta-analyses that reported estimates of mean interraterreliability that were based on large samples. Hunter (1986) estimated meaninterrater reliability to be .60, but this estimate was based on a relativelylimited data set. Based on larger, independent data sets, Rothstein (1990)and Viswesvaran, Ones, and Schmidt (1996) estimated mean interraterreliability to be between .48 and .52. Salgado et al. (2003) also reported ameta-analytically derived reliability estimate of .52 for job performanceratings in a sample of European studies. After reviewing the available


TABLE 1Validity Estimates as Moderated by Personality Scale Type

95% CI

Analysis k N r SDr ρ SDρ Lower Upper

ConscientiousnessOverall 113 19,556 .15 .08 .24 .13 .22 .26

Noncontextualized 91 16,078 .14 .08 .22 .13 .19 .25General 69 12,283 .13 .09 .21 .14 .18 .24Workplace 22 3,795 .18 .05 .28 .08 .25 .31

Contextualized 22 3,478 .19 .00 .30 .00 .30 .30General 7 1,096 .17 .00 .27 .00 .27 .27Workplace 15 2,382 .20 .00 .31 .00 .31 .31

General 76 13,379 .13 .08 .21 .13 .18 .24Workplace 37 6,177 .19 .04 .30 .06 .28 .32

Emotional StabilityOverall 86 13,565 .09 .08 .14 .12 .11 .17


Contextualized 18 2,619 .18 .11 .27 .17 .19 .35General 4 399 .17 .08 .26 .12 .14 .38Workplace 14 2,220 .18 .11 .27 .17 .18 .36


ExtraversionOverall 90 14,568 .07 .10 .11 .16 .08 .14

Noncontextualized 72 11,876 .05 .08 .08 .13 .05 .11General 54 9,062 .06 .08 .09 .13 .06 .12Workplace 18 2,814 .03 .08 .05 .13 −.01 .11



AgreeablenessOverall 94 15,188 .08 .09 .13 .15 .10 .16




continued


TABLE 1 (continued)

95% CI


Openness to ExperienceOverall 80 13,044 .02 .07 .03 .11 .01 .05

Noncontextualized 66 10,866 .01 .06 .02 .09 .00 .04General 51 8,326 −.01 .06 −.02 .09 −.04 .00Workplace 15 2,540 .06 .00 .09 .00 .09 .09

Contextualized 14 2,178 .09 .08 .14 .12 .08 .20General 4 401 .11 .12 .17 .19 −.02 .36Workplace 10 1,777 .08 .07 .12 .11 .05 .19

General 55 8,727 .00 .07 .00 .11 −.03 .03Workplace 25 4,317 .07 .04 .11 .06 .09 .13

Note. k = number of validity coefficients; N = total sample size; r = observed sampleweighted mean validity; SDr = observed sample weighted standard deviation; ρ = meanoperational validity corrected for indirect range restriction; SDρ = corrected standarddeviation; CI = confidence interval.

research on this issue, we chose to use the estimate of .52 provided byViswesvaran et al. (1996) because it was derived from the largest availabledata set, was meta-analytically derived, and has been replicated in anindependent set of primary studies.

Results

Hypothesis 1 predicted that contextualized measures of personalitywould be more valid than noncontextualized measures of personality. Asshown in Table 1, contextualized measures of Conscientiousness weremore valid (ρ = .30, k = 22, N = 3,478) than were noncontextualizedmeasures (ρ = .22, k = 91, N = 16,078), and there was no overlap be-tween the confidence intervals for these estimates (note that the validityestimates discussed in this section are estimates of operational validityand are therefore not corrected for unreliability in the predictor). Con-textualized measures of Emotional Stability also were more valid (ρ =.27, k = 18, N = 2,619) than were noncontextualized measures (ρ =.11, k = 68, N = 10,946) with no overlap in the confidence intervals forthe two estimates. This pattern, which is further depicted in Figure 1,remained consistent for contextualized (ρ = .25, k = 18, N = 2,692) andnoncontextualized (ρ = .08; k = 72, N = 11,876) measures of Extraver-sion, contextualized (ρ = .24, k = 21, N = 3,357) and noncontextualized(ρ = .10, k = 73, N = 11,831) measures of Agreeableness, and contex-tualized (ρ = .14, k = 14, N = 2,178) and noncontextualized (ρ = .02,k = 66, N = 10,866) measures of Openness to Experience. None of the


.40

.35

.30

.25

.20

.15

.10

.05

–.05

.00

Conscientiousness Emotional stability Extraversion

Noncontextualized Contextualized

Agreeableness Opnness toexperience

Figure 1: Comparison of Validity Estimates for Supervisory Ratings ofOverall Job Performance.

confidence intervals for these estimates overlapped. Overall, the resultssuggest that contextualized personality measures are better predictors ofperformance than are noncontextualized measures and provide support forHypothesis 1.

Hypothesis 2 proposed that personality measures designed for use inthe workplace would be more valid than those designed for general use. Asshown in Table 1, the overall validity estimates for personality measuresdesigned for workplace use (ρ = .30, k = 37, N = 6,177) were higher thanthose for measures designed for general-purpose use (ρ = .21, k = 76,N = 13,379) for Conscientiousness. This was also the case for workplace(ρ = .20, k = 30, N = 4,850) and general-purpose (ρ = .11, k = 56,N = 8,715) measures of Emotional Stability, for workplace (ρ = .19,k = 32, N = 5,133) and general-purpose (ρ = .10, k = 62, N = 10,055)measures of Agreeableness, and for workplace (ρ = .11, k = 25, N =4,317) and general-purpose (ρ = .00, k = 55, N = 8,727) measures ofOpenness to Experience. There was no overlap in the confidence intervalsfor these estimates. These findings lend initial support to Hypothesis 2.

However, in order to examine Hypothesis 2 more closely, we disentan-gled the effects of contextualization from the effects of scale developmentby examining the developmental purpose of personality measures withinthe noncontextualized and contextualized personality measures categoriesseparately. Turning first to the results for noncontextualized personality


measures, the results show that scales designed for use in the workplace(ρ = .28, k = 22, N = 3,795) were more valid than those designed forgeneral use (ρ = .21, k = 69, N = 12,283) for Conscientiousness. This wasalso the case for workplace (ρ = .15, k = 17, N = 2,751) and general use(ρ = .08, k = 56, N = 9,080) measures of Agreeableness and workplace(ρ = .09, k = 15, N = 2,540) and general use (ρ = −.02, k = 51, N =8,326) measures of Openness to Experience. Turning next to the resultsfor contextualized measures of personality, only for Conscientiousnesswere measures designed for use in the workplace (ρ = .31, k = 15, N =2,382) more valid than those designed for general use (ρ = .27, k = 7,N = 1,096). In general, the results show that once the FOR of a person-ality measure is taken into account there is no consistent indication thatpersonality measures designed for use in the workplace are more validthan are measures designed for general use. Overall, these results do notsupport Hypothesis 2.

Hypothesis 3 predicted that contextualized measures of personal-ity would have higher internal reliabilities than would noncontextual-ized measures of personality. As shown in Table 2, the mean alphareliability estimates for noncontextualized and contextualized measureswere virtually identical, ranging from .77 to .84 (M = .81) for noncon-textualized measures and from .81 to .87 (M = .84) for contextualizedmeasures. For conscientiousness, we obtained reliability estimates of .83(k = 59, N = 9,202) and .83 (k = 10, N = 1,648) for noncontextualizedand contextualized measures, respectively. Reliability estimates were .84(k = 44, N = 6,849) and .87 (k = 6, N = 789) for Emotional Stability; .81(k = 47, N = 7,326) and .87 (k = 7, N = 943) for Extraversion; .78 (k =48, N = 7,301) and .81 (k = 9, N = 1,527) for Agreeableness; and .77 (k= 45, N = 7,170) and .83 (k = 7, N = 872) for Openness to Experience. Inaddition, within each personality trait there was overlap between the con-fidence intervals surrounding the reliability estimates. Because coefficientalpha is dependent on the number of items a measure contains (Cortina,1993), we examined the possibility that the mean number of items foundin noncontextualized personality measures differed from the number ofitems found in contextualized measures. We found no statistical differ-ence in the mean number of items for the two types of measures. Thissuggests that the validity estimates for contextualized and noncontextual-ized personality scales were not unduly influenced by the number of itemscontained in each scale. Taken as a whole, these results do not supportHypothesis 3.

Hypothesis 4 predicted that the validities reported in publishedstudies of contextualized personality measures will be higher than thevalidities reported in unpublished studies. To test this hypothesis, for eachpersonality trait we compared the validity estimates and 95% confidence


TABLE 2Reliability Estimates for Noncontextualized and Contextualized Measures of

Personality

95% CI

k N Item mean Item SD α SDα Lower Upper

ConscientiousnessOverall 69 10,850 22.79 16.58 .83 .06 .82 .84

Noncontextualized 59 9,202 23.23 17.33 .83 .06 .81 .85Contextualized 10 1,648 20.22 11.79 .83 .06 .79 .87

Emotional StabilityOverall 50 7,638 22.36 16.70 .85 .05 .84 .86

Noncontextualized 44 6,849 23.24 17.48 .84 .05 .83 .85Contextualized 6 789 16.83 9.79 .87 .05 .83 .91

ExtraversionOverall 54 8,269 22.04 16.92 .82 .07 .80 .84


AgreeablenessOverall 57 8,828 20.78 15.59 .78 .08 .76 .80

Noncontextualized 48 7,301 21.29 16.61 .78 .08 .76 .80Contextualized 9 1,527 18.13 8.68 .81 .08 .76 .86

Openness to ExperienceOverall 52 8,042 25.80 17.93 .78 .08 .76 .80


Note. k = number of validity coefficients; N = total sample size; α = observed sampleweighted mean alpha reliability; SDα = observed sample weighted standard deviation ofalpha; CI = confidence interval.

intervals of published and unpublished studies of noncontextualizedand contextualized scales (Dudley, Orvis, Lebiecki, & Cortina, 2006).As shown in Table 3, the results show little to no difference betweenthe validity estimates for noncontextualized measures of EmotionalStability, Extraversion, Agreeableness, and Openness to Experience. Inthose cases where small differences exist, the confidence intervals forthe estimates overlap. This pattern can also be seen for contextualizedmeasures of personality. Specifically, there is little difference betweenvalidity estimates. For contextualized measures, the validity estimatesthat are reported in published studies (ρ = .25, k = 12, N = 2,018) arevery similar to those reported in unpublished studies (ρ = .23, k = 6, N =674) for Extraversion. This is also the case for the results from published(ρ = .24, k = 14, N = 2,602) and unpublished studies (ρ = .27,k = 7, N = 755) for Agreeableness and published


TABLE 3Validity Estimates as Moderated by Publication Status

95% CI


ConscientiousnessOverall

Published 67 11,865 .17 .07 .27 .11 .24 .30Unpublished 46 7,691 .11 .08 .17 .13 .13 .21

NoncontextualizedPublished 52 9,142 .16 .08 .25 .13 .21 .29Unpublished 39 6,936 .10 .08 .16 .13 .12 .20

ContextualizedPublished 15 2,723 .18 .02 .28 .03 .26 .30Unpublished 7 755 .22 .00 .34 .00 .34 .34

Emotional StabilityOverall




ExtraversionOverall




AgreeablenessOverall




continued


TABLE 3 (continued)

95% CI


Openness to ExperienceOverall

Published 47 7,722 .03 .08 .05 .12 .02 .08Unpublished 33 5,322 .01 .06 .02 .09 −.01 .05

NoncontextualizedPublished 38 6,179 .02 .07 .03 .11 .00 .06Unpublished 28 4,687 .00 .05 .00 .08 −.03 .03



(ρ = .12, k = 9, N = 1,543) and unpublished studies (ρ = .17, k = 5, N =635) for Openness to Experience. The exception to this pattern of resultsis seen for measures of Conscientiousness. For contextualized measuresof Conscientiousness, the mean validity reported in published studies wasslightly lower (ρ = .28, k = 15, N = 2,723; 95% CI = .26–.30) thanthe mean validity reported in unpublished studies (ρ = .34, k = 7, N= 755; 95% CI = .34–.34). Taken together, these results do not supportHypothesis 4.

A reviewer suggested that we examine potential differences in thevalidity of contextualized personality measures based on the means bywhich a work-specific FOR is achieved (i.e., based on whether measures(a) consisted of items that made specific references to the workplace, (b)contained instructions that specifically referenced workplace behaviors, or(c) contained both items and instructions that referenced the workplace).The results of this analysis can be found in a supplementary table inAppendix C. The results suggest that the validity of personality measurescan be increased when both instructions and scale items reference work-specific behaviors. However, these results were not consistent across allfive personality traits and should be interpreted with caution given thesmall number of studies in many of the analyses.

Discussion

The purpose of this meta-analysis was to investigate the relative va-lidity of contextualized and noncontextualized measures of self-reportedpersonality. At the overall level of analysis, the validity estimates that we


report are similar to those reported in previous research (Barrick & Mount,1991; Hurtz & Donovan, 2000). Our study builds on prior work by show-ing that contextualized measures of personality are stronger predictors ofjob performance than are noncontextualized measures of personality. Thepattern of our results was consistent across all of the Big Five personalitytraits. In fact, for four of the five Big Five traits—Emotional Stability,Extraversion, Agreeableness, and Openness to Experience—the magni-tude of the validity of contextualized measures was at least twice that ofnoncontextualized measures.

Though we found FOR effects for all five of the personality traits weexamined, it is worth noting that the FOR effect was somewhat smallerfor measures of Conscientiousness. In fact, the validity of contextualizedmeasures of Conscientiousness was only marginally higher than that ofnoncontextualized measures. Hunthausen et al. (2003) proposed that onereason for this may be that the influence of Conscientiousness on behavioris more consistent across various situations than is the influence of otherpersonality traits. A reviewer suggested that another plausible explana-tion for this pattern may be that the nature of Conscientiousness causesnoncontextualized measures of Conscientiousness to be contextualized tosome extent. This occurs because noncontextualized measures of Consci-entiousness tend to include at least some items that could be construedas being specifically linked to work behaviors. For example, consider thefollowing sample items taken from noncontextualized Conscientiousnessscales: “[I] am exacting in my work” (Goldberg, 1999); “At times I havetrouble making myself concentrate on my work” (Schmit et al., 2000).Although the word “job” or “work” could be interpreted to mean a jobat home, work, school, or otherwise, the vast majority of the primarystudies included in this analyses were concurrent studies of incumbentworkers. Therefore, it may be the case that when completing items thatreferred to “jobs,” “work,” or “tasks,” respondents were more inclined todescribe their workplace behavior for these items even though items werenot explicitly contextualized for selection purposes.

We also examined whether the differences in the validities of the twotypes of measures could be accounted for by the purpose for which theywere developed. The results of our moderator analyses suggest that oncethe FOR has been taken into account, personality measures that are de-signed for use in the workplace are not necessarily more valid than arethose designed for general use. Although we acknowledge the importanceof linking predictors to criteria by means of conducting a thorough jobanalysis (Pearlman & Sanchez, 2010), our results suggest that contex-tualized personality measures are better predictors of job performancethan are noncontextualized measures regardless of whether those mea-sures are directly based on a job analysis. Indeed, we found few, if any,


differences in the validities of general-purpose and workplace measures ofpersonality. One possible explanation for these results can be found uponcloser investigation of the specific items that constitute general-purposeand workplace personality measures. For example, consider the followingitems—one taken from a personality measure designed for general use,and the other a sample item from a measure designed for workplace use:“I talk to a lot of different people at parties” and “I find it easy to startconversations with strangers.” These items may be of considerable use inassessing normal adult personality, but it could be argued that they havelittle job relevance outside of specific jobs, such as sales. Thus, althougha personality measure may have been constructed only after a careful jobanalysis, this does not guarantee that all of the items contained in that mea-sure are work specific. In fact, there is sometimes little to no differencebetween items that are found on general-purpose and workplace personal-ity measures. During our coding process, we found that the preponderanceof items contained in noncontextualized measures developed for use in theworkplace were no different than items contained in measures developedfor general use.

In addition, we found no differences between the internal consistencyreliabilities of contextualized and noncontextualized measures, whichsuggests that the higher validities of contextualized measures are notattributable to their reliability. Finally, we explored the extent to whichpublication bias influenced our results. The overall results of our exami-nation were similar to those found in previous research and indicate thatpublished validities for Conscientiousness are slightly higher than arethe unpublished validities (Kepes, McDaniel, Banks, Hurtz, & Donovan,2011). However, publication status did not moderate the validities of anyother Big Five personality trait nor did it account for the differences in thevalidities of contextualized and noncontextualized personality measures.Taken as a whole, our results raise the possibility that the difference invalidities between contextualized and noncontextualized measures is dueto factors other than developmental purposes, reliability differences, orpublication bias.

Implications for Theory and Practice

These results have implications for the theory of conditional dis-positions and add to the growing body of literature that suggests thatcontext-specific criteria are best predicted by context-specific measures(e.g., Bowling & Burns, 2010; Wood & Roberts, 2006). For example,work-specific trait affect has been shown to be more closely linked to jobsatisfaction than is home-specific or noncontextualized affect, whereashome-specific affect is more closely linked to marital satisfaction than


is work-specific or noncontextualized affect (Roberts & Donahue, 1994).Recent meta-analytic results reveal that work-specific locus of control isa better predictor of work outcomes such as burnout and job satisfactionthan is noncontextualized locus of control (Wang et al., 2010). Overall, ourresults are consistent with the FOR effect that has been found in previousresearch (Lievens et al., 2008; Schmit et al., 1995) and support the notionthat personality measures can be made more useful in a selection contextif they are tailored to reflect personality in the workplace.

This study also has several implications for practitioners. First, andperhaps most apparent, practitioners should consider using personalitymeasures that contain a FOR that is specific to the workplace if thepersonality scores are to be used for hiring purposes. We are not suggest-ing that organizations need to develop personality scales that are specificto the jobs in their organizations or to the organization itself. Rather, ourrecommendations echo those of previous researchers (Hunthausen et al.,2003; Morgeson et al., 2007). Simply adding “at work” to existing per-sonality scales may be an efficient way to increase their validity. Second,our results show that contextualized personality measures designed forgeneral use were just as valid as those designed for workplace use. Thissuggests that the use of proprietary personality measures in a selectioncontext may not be necessary. Our results suggest that publicly availablemeasures, such as the International Personality Item Pool (IPIP), maybe effective predictors of performance so long as they are revised toreflect a work-specific FOR. Third, those responsible for designing se-lection procedures should consider including contextualized measures ofpersonality traits that have typically been considered weaker predictorsof performance. Previous meta-analyses (e.g., Barrick & Mount, 1991;Salgado, 1997) concluded that across a broad sampling of jobs, traits suchas Extraversion, Agreeableness, and Openness to Experience were not es-pecially useful predictors of performance. The results of this study suggestthat when contextualized these traits may be more useful for predictingjob performance than was previously believed.

Another benefit that practitioners may realize when using contex-tualized personality measures for selection purposes is an increase inthe positive reaction that applicants have to the hiring process. Appli-cants who experience a negative reaction to a selection procedure maybe more likely to challenge the legality of the hiring process, becomeunmotivated to perform well on selection tests (thus lowering the valid-ity of the tests), or withdraw from the hiring process entirely (Rynes &Connerly, 1993). Historically, applicant reactions to personality tests havebeen generally negative because the tests are perceived as being inva-sive and irrelevant to job performance (Rosse, Miller, & Stecher, 1994).Research has shown that one way to ameliorate this concern is to use


personality tests that contain items that are designed to assess behaviorsand attitudes that are specific to the workplace (Jones, 1991). Applicantsmay perceive such tests as having higher face validity, which reducesthe extent to which applicants are critical of the hiring process itself(Hausknecht, Day, & Thomas, 2004). Holtz, Ployhart, and Dominguez(2005) examined this possibility and found that reactions to contextualizedmeasures were not more positive than were reactions to noncontextualizedmeasures. However, their study was conducted using a student sample.Future research should address this issue among samples of actual jobapplicants.

Limitations and Future Research

Our study is not without limitations. First, it is important to notethat the availability of primary studies that reported validity results forcontextualized measures of personality was limited. As a result, severalof our moderator analyses are based on few studies and the confidenceintervals surrounding some of the estimates presented in our study arequite wide. This limitation is especially relevant to our reliability estimatesfor contextualized measures and our validity estimates for unpublishedcontextualized measures. Due to this limitation, we advise caution wheninterpreting the results for such analyses. Second, in examining reliabilitydifferences between contextualized and noncontextualized measures wecomputed only the coefficient of equivalence (alpha reliabilities) for eachtype of measure, which accounts only for random and specific factor error.Theoretically, it would be more appropriate to compare the coefficient ofequivalence and stability, which accounts for random, specific factor, andtransient error (Le, Schmidt, & Putka, 2009). However, few studies provideestimates of transient error for personality measures (Schmidt, Le, & Ilies,2003), and we could locate no extant studies that estimated transient errorfor contextualized measures of personality.

A further limitation can be inferred from the large standard deviationsfor the validity estimates reported in Table 1. For contextualized measuresof personality, the SDρ for Conscientiousness was .00. For the other fourpersonality traits, SDρ was quite high, ranging from .09 to .21. As a re-sult, the confidence intervals surrounding the validity estimates for thesetraits are quite wide. Overall, this may suggest the existence of potentialmoderators of the validity of contextualized personality measures. We at-tempted to examine several moderators that may be theoretically importantin the context of this meta-analysis, but we found there was an insufficientnumber of primary studies available with which to do so. We attemptedto include validity estimates for facet-level performance ratings such astask, contextual, or counterproductive performance in our analysis. As


mentioned previously, there were extremely few studies that reported suchdata for contextualized personality measures. For task performance, onlytwo studies reported validities for contextualized personality scales. Forcontextual performance, only one study reported validities for contextual-ized personality scales. For counterproductive work behaviors, there wereno studies that reported validity estimates for contextualized scales. As aresult, although we endeavored to include criteria other than overall jobperformance, we determined that this was not feasible. In addition, we didnot include performance ratings that were derived from a source other thansupervisors (e.g., peers, customers, and subordinates) because no studiescontaining a contextualized personality measure provided criterion rat-ings from a source other than supervisors. Also absent from this study isinformation regarding validity estimates obtained from applicant versusincumbent samples because only three studies in our database were basedon applicant samples—and none used contextualized measures. Conse-quently, the generalizability of our findings to applicant populations islimited. More primary studies based on applicant samples are clearlyneeded in this area of research.

Roughly half of the data that comprise our analyses of contextualizedpersonality measures were drawn from primary studies that used the Per-sonal Style Inventory (PSI). If the validity estimates reported for the PSIdiffer systematically from those reported for other contextualized mea-sures, then it may be the case that our results are biased in some way. Weperformed a post hoc analysis to investigate this possibility, which canbe found in Appendix D. We found that for Conscientiousness the meanvalidity estimate reported in studies based on the PSI (ρ = .34, k = 11,N = 1,595) was slightly higher than that reported in other studies (ρ =.27, k = 11, N = 1,883). The confidence intervals for these estimates didnot overlap. For the other Big Five traits, we found some differences inthe validity estimates derived from the PSI and those derived from othermeasures. Although the confidence for these estimates overlapped, thevalidities reported for the PSI were consistently larger than those reportedfor other contextualized measures. Nevertheless, when the PSI was ex-cluded from our analyses, the mean validities for contextualized measuresof personality were higher than those for noncontextualized measures.

Finally, to the extent that the contextualized personality measures in-cluded in our analyses were developed based on empirical item selection(that is, based on their empirical relationship to job performance criteria),the results of our analysis may be biased. We reviewed articles and testmanuals, contacting test publishers when necessary, to obtain informa-tion about the development of each personality measure included in ouranalysis and found that the vast majority of the measures were not de-veloped through empirical item selection. We were unable to obtain this


information for the Page Work Behavior Inventory (PWBI), a contextual-ized personality measure from which we obtained one effect size for eachBig Five trait.

This study sheds light on several other opportunities for future re-search. First, and perhaps most apparently, additional primary studies thatexamine the validity of contextualized personality measures are needed.Specifically, as mentioned previously, few validity estimates are avail-able for criteria such as task, contextual, counterproductive, or objectivelymeasured performance. Though we present validity estimates for overallperformance in this study, previous research has shown that noncontex-tualized measures of personality do not predict all job criteria equally(see table 1 in Ones et al. [2007] for a summary of this research). It isimportant to determine the criteria for which contextualized measures ofpersonality are most valid. In addition, there are several other criteria thathave been linked to noncontextualized personality measures for which nodata exist for contextualized measures. Performance criteria such as lead-ership effectiveness, adaptive performance, or team performance could beexamined. Attitudinal criteria such as job satisfaction and motivation mayalso be related to contextual measures.

Our results are also interesting when placed next to those reported ina recent meta-analysis by Dudley et al. (2006) that showed specific facetsof Conscientiousness were better predictors of certain job outcomes thanwas trait-level Conscientiousness. A comparison of our results with thosereported by Dudley et al. shows that the operational validity of con-textualized, trait-level measures of Conscientiousness is higher than thevalidity of noncontextualized, facet-level measures of Conscientiousness(e.g., achievement, dependability, order, and cautiousness) for predictingoverall job performance. However, it is important to note that Dudleyet al. included only Conscientiousness in their analysis. Thus, althoughthe results of this study suggest that there is still much to be gained bystudying personality at the trait level, researchers are beginning to givemore attention to the validity of personality facets and their potentialfor predicting specific facets of performance criteria (Hough & Dilchert,2010; Sackett, 2010). This raises the possibility that one potentially ef-fective way to increase the validity of personality measures for predictingfacet-level performance criteria might be to combine the two approachesby contextualizing the most valid facet-level personality scales.

Summary

In spite of general acceptance of the notion that personality is avalid predictor of job performance, the debate concerning the usefulnessof self-report measures of personality continues. This study provides a


meta-analytic examination of the relative validity of contextualized andnoncontextualized personality measures, and concludes that contextual-ized measures of personality are more valid predictors of performancethan are noncontextualized measures. These findings have implicationsfor theories of personality and also for the practical implementation ofpersonality measures in employee selection. Personality has been a fruit-ful area for research in the past, and it remains so for the future.

REFERENCES

+Current reference has replaced Hattrup, O’Connell, and Wingate (1998).

∗Abraham JD, Morrison JD, Jr. (2003). Relationship between the Performance Perspec-tive Inventory’s Implementing scale and job performance of incumbents in a dualsecurity/service role. Applied HRM Research, 8, 73–76.

Abraham JD, Morrison JD, Jr. (2009). Performance Perspectives Inventory: PPI technicalmanual, version 8. Tulsa, OK: A&M Psychometrics, LLC.

ACT, Inc. (2007). WorkKeys Talent Assessment user and technical guide. Iowa City, IA:Author.

Allport GW. (1961). Pattern and growth in personality. New York, NY: Holt, Rinehart andWinston.

∗Avis JM. (2001). An examination of the prediction of overall, task, and contextual perfor-mance using three selection measures for a service-type occupation. Unpublisheddoctoral dissertation, University of Southern Mississippi.

∗Bacha RR. (2003). Specifying personality and self-monitoring effects on overall, task,and contextual job performance. Unpublished doctoral dissertation, Kent StateUniversity.

Barrick MR, Mount MK. (1991). The Big Five personality dimensions and job performance:A meta-analysis. PERSONNEL PSYCHOLOGY, 44, 1–26.

∗Barrick MR, Mount MK. (1993). Autonomy as a moderator of the relationships be-tween the Big Five personality dimensions and job performance. Journal of AppliedPsychology, 78, 111–118.

∗Barrick MR, Mount MK. (1996). Effects of impression management and self-deceptionon the predictive validity of personality constructs. Journal of Applied Psychology,81, 261–272.

Barrick MR, Mount MK, Judge TA. (2001). Personality and performance at the beginningof the new millennium: What do we know and where do we go next? InternationalJournal of Selection and Assessment, 9, 9–30.

∗Barrick MR, Mount MK, Strauss JP. (1994). Antecedents of involuntary turnover due to areduction in force. PERSONNEL PSYCHOLOGY, 47, 515–535.

Barrick MR, Stewart GL, Piotrowski M. (2002). Personality and job performance: Test ofthe mediating effects of motivation among sales representatives. Journal of AppliedPsychology, 87, 43–51.

∗Bauer TN, Erdogan B, Liden RC, Wayne SJ. (2006). A longitudinal study of the moderatingrole of extraversion: Leader-member exchange, performance, and turnover duringnew executive development. Journal of Applied Psychology, 91, 298–310.

∗Bergner S, Neubauer AC, Kreuzthaler A. (2010). Broad and narrow personality traitsfor predicting managerial success. European Journal of Work and OrganizationalPsychology, 19, 177–199.


∗Berry CM, Page RC, Sackett PR. (2007). Effects of self-deceptive enhancement onpersonality-job performance relationships. International Journal of Selection andAssessment, 15, 94–109.

∗Bing MN, Burroughs SM. (2001). The predictive and interactive effects of equity sensi-tivity in teamwork-oriented organizations. Journal of Organizational Behavior, 22,271–290.

Bing MN, Whanger, JC, Davison, HK, VanHook, JB. (2004). Incremental validity of theframe-of-reference effect in personality scale scores: A replication and extension.Journal of Applied Psychology, 89, 150–157.

Bowling NA, Burns, GN. (2010). A comparison of work-specific and general personalitymeasures as predictors of work and non-work criteria. Personality and IndividualDifferences, 49, 95–101.

∗Boyes FAJ. (2005). Personality as a predictor of military performance and counterpro-ductive behaviour. Unpublished master’s thesis, Saint Mary’s University, Halifax,Nova Scotia, Canada.

∗Brown TJ, Mowen JC, Donavan DT, Licata JW. (2002). The customer orientation ofservice workers: Personality trait effects on self- and supervisor performance ratings.Journal of Marketing Research, 39, 110–119.

∗Burke LA, Witt LA. (2002). Moderators of the openness to experience-performance rela-tionship. Journal of Managerial Psychology, 17, 712–721.

∗Buttigieg S. (2006). Relationship of a biodata instrument and a Big 5 personality measurewith the job performance of entry-level production workers. Applied HRM Research,11, 65–68.

∗Caligiuri PM. (2000). The Big Five personality characteristics as predictors of expatriate’sdesire to terminate the assignment and supervisor-rated performance. PERSONNEL

PSYCHOLOGY, 53, 67–88.∗Chan D, Schmitt N. (2002). Situational judgment and job performance. Human Perfor-

mance, 15, 233–254.∗Chan FM. (2004). The effects of optimism and the Five-Factor Model of personality

on stress and performance in the work place. Unpublished doctoral dissertation,University of Tennessee.

∗Connolly J. (2001). Assessing the construct validity of a measure of learning agility.Unpublished doctoral dissertation, Florida International University.

∗Conte JM, Gintoft JN. (2005). Polychronicity, Big Five personality dimensions, and salesperformance. Human Performance, 18, 427–444.

∗Cook VD. (2005). An investigation of the construct validity of the Big Five constructof emotional stability in relation to job performance, job satisfaction, and careersatisfaction. Unpublished doctoral dissertation, University of Tennessee.

Cortina JM. (1993). What is coefficient alpha? An examination of theory and applications.Journal of Applied Psychology, 78, 98–104.

Costa PT, Jr., McCrae RR. (1992). Revised NEO Personality Inventory (NEO PI-R) and NEOFive-Factor Inventory (NEO-FFI) professional manual. Odessa, FL: PsychologicalAssessment Resources.

∗Crespo MR. (2007). Intelligence and personality as antecedents to leadership effective-ness and extrinsic career success: The predictive power of individual differences.Unpublished doctoral dissertation, Columbia University.

∗Cutchin GC. (1998). Relationships between the Big-Five personality factors and perfor-mance criteria for in-service high-school teachers. Unpublished doctoral disserta-tion, Purdue University.

Davison HK, Bing MN. (2009). Content validity does matter for the criterion-relatedvalidity of personality tests. Industrial and Organizational Psychology, 2, 501–503.


∗DeGroot T, Kluemper D. (2007). Evidence of predictive and incremental validity ofpersonality factors, vocal attractiveness and the situational interview. InternationalJournal of Selection and Assessment, 15, 30–39.

∗Diefendorff JM, Brown DJ, Kamin AK, Lord RG. (2002). Examining the roles of jobinvolvement and work centrality in predicting organizational citizenship behaviorsand job performance. Journal of Organizational Behavior, 23, 93–108.

∗Diefendorff JM, Richard EM, Gosserand RH. (2006). Examination of situational and at-titudinal moderators of the hesitation and performance relation. PERSONNEL PSY-CHOLOGY, 59, 365–393.

∗Dierdorff EC. (2002). Dispositional influences and individual differences in work perfor-mance: Relationships of goal orientation, growth need strength, and conscientious-ness. Unpublished doctoral dissertation, North Carolina State University.

Digman JM. (1990). Personality structure: Emergence of the five-factor model. AnnualReview of Psychology, 41, 417–440.

Donahue EM, Harary K. (1998). The patterned inconsistency of traits: Mapping the dif-ferential effects of social roles on self-perceptions of the Big Five. Personality andSocial Psychology Bulletin, 24, 610–619.

∗Donavan DT. (1999). Antecedents and consequences of the contact employee’s serviceorientation: From personality traits to service behaviors. Unpublished doctoraldissertation, Oklahoma State University.

Dudley NM, Orvis, KA, Lebiecki JE, Cortina JM. (2006). A meta-analytic investigation ofconscientiousness in the prediction of job performance: Examining the intercorrela-tions and the incremental validity of narrow traits. Journal of Applied Psychology,91, 40–57.

∗English AD. (2003). When personality traits need a frame of reference: Enhancing thepredictive validity of non-cognitive measures. Unpublished doctoral dissertation,Florida Institute of Technology.

∗Farmer SJ. (1999). Enhancing criterion-related validity by assessing context-specifictraits: Implications of conditional dispositions for personnel selection. Unpublisheddoctoral dissertation, Central Michigan University.

Ghiselli EE. (1973). The validity of aptitude tests in personnel psychology. PERSONNEL

PSYCHOLOGY, 26, 461–477.∗Gill CM, Hodgkinson GP. (2007). Development and validation of the Five-Factor Model

Questionnaire (FFMQ): An adjectival-based personality inventory for use in occu-pational settings. PERSONNEL PSYCHOLOGY, 60, 731–766.

Goldberg LR. (1992). The development of markers for the Big-Five factor structure. Psy-chological Assessment, 4, 26–42.

Goldberg LR. (1999). A broad-bandwidth, public-domain, personality inventory measuringthe lower-level facets of several five-factor models. In Mervielde L, Deary I, De FruytF, Ostendorf F (Eds.), Personality psychology in Europe (Vol. 7, pp. 7–28). Tilburg,The Netherlands: Tilburg University Press.

∗Griffin B, Hesketh B. (2004). Why openness to experience is not a good predictor of jobperformance. International Journal of Selection and Assessment, 12, 243–251.

Guion RM, Gottier RF. (1965). Validity of personality measures in personnel selection.PERSONNEL PSYCHOLOGY, 18, 135–164.

∗Halfhill T, Nielsen TM, Sundstrom E, Weilbaecher A. (2005). Group personality com-position and performance in military service teams. Military Psychology, 17, 41–54.

+∗Hattrup, K, O’Connell MS, Labrador JR. (2005). Incremental validity of locus of control

after controlling for cognitive ability and conscientiousness. Journal of Businessand Psychology, 19, 461–481.


Hausknecht JP, Day DV, Thomas SC. (2004). Applicant reactions to selection proce-dures: An updated model and meta-analysis. PERSONNEL PSYCHOLOGY, 57, 639–683.

∗Hayes TL, Roehm HA, Castellano JP. (1994). Personality correlates of success in totalquality manufacturing. Journal of Business and Psychology, 8, 397–411.

Heller D, Ferris DL, Brown D, Watson D. (2009). The influence of work personality on jobsatisfaction: Incremental validity and mediation effects. Journal of Personality, 77,1051–1084.

Heller D, Watson D, Komar J, Min J, Perunovic WQE. (2007). Contextualized personal-ity: Traditional and new assessment procedures. Journal of Personality, 75, 1229–1253.

∗Hense RL III. (2000). The Big Five and contextual performance: Expanding person-environment fit theory. Unpublished doctoral dissertation, University of SouthFlorida.

∗Higgins DM, Peterson JB, Pihl RO, Lee AGM. (2007). Prefrontal cognitive ability, intelli-gence, Big Five personality, and the prediction of advanced academic and workplaceperformance. Journal of Personality and Social Psychology, 93, 298–319.

Hogan R, Hogan J. (2007). Hogan Personality Inventory manual (3rd ed.). Tulsa, OK. HoganAssessment Systems.

Holtz BC, Ployhart RE, Dominguez A. (2005). Testing the rules of justice: The effects offrame-of-reference and pre-test validity information on personality test responsesand test perceptions. International Journal of Selection and Assessment, 13, 75–86.

Hough LM. (1992). The “Big Five” personality variables—construct confusion: Descriptionversus prediction. Human Performance, 5, 139–155.

Hough L, Dilchert S. (2010). Personality: Its measurement and validity for employeeselection. In Farr JL, Tippins NT (Eds.), Handbook of employee selection (pp.299-319). New York, NY: Routledge.

Hough LM, Eaton NK, Dunnette MD, Kamp JD, McCloy RA. (1990). Criterion-relatedvalidities of personality constructs and the effect of response distortion on thosevalidities. Journal of Applied Psychology, 75, 581–595.

Hunter JE. (1986). Cognitive ability: Cognitive aptitudes, job knowledge, and job perfor-mance. Journal of Vocational Behavior, 29, 340–362.

Hunter JE, Schmidt FL. (2004). Methods of meta-analysis: Correcting error and bias inresearch findings (2nd ed.). Thousand Oaks, CA: Sage.

∗Hunthausen JM, Truxillo DM, Bauer TN, Hammer LB. (2003). A field study of frame-of-reference effects on personality test validity. Journal of Applied Psychology, 88,545–551.

Hurtz GM, Donovan JJ. (2000). Personality and job performance: The Big Five revisited.Journal of Applied Psychology, 85, 869–879.

∗Jackson CL, Colquitt JA, Wesson MJ, Zapata-Phelan CP. (2006). Psychological col-lectivism: A measurement validation and linkage to group member performance.Journal of Applied Psychology, 91, 884–899.

∗Jacobs RR, Conte JM, Day DV. (1996). Selecting bus drivers: Multiple predictors, multipleperspectives on validity, and multiple estimates of utility. Human Performance, 9,199–217.

John OP, Donahue EM, Kentle RL. (1991). The Big Five Inventory—Versions 4a and 54.Berkeley, CA: University of California, Berkeley, Institute of Personality and SocialResearch.

Jones JW. (1991). Assessing privacy invasiveness of psychological test items: Job relevantversus clinical measures of integrity. Journal of Business and Psychology, 5, 531–535.


∗Judge TA, Erez A. (2007). Interaction and intersection: The constellation of emotionalstability and extraversion in predicting performance. PERSONNEL PSYCHOLOGY,60, 573–596.

∗Judge TA, LePine JA, Rich BL. (2006). Loving yourself abundantly: Relationship ofthe narcissistic personality to self- and other perceptions of workplace deviance,leadership, and task and contextual performance. Journal of Applied Psychology,91, 762–776.

∗Judge TA, Thoresen CJ, Pucik V, Welbourne TM. (1999). Managerial coping with orga-nizational change: A dispositional perspective. Journal of Applied Psychology, 84,107–122.

Kepes S, McDaniel MA, Banks GC, Hurtz GM, Donovan JJ. (2011, April). Big Five validityand publication bias: Conscientiousness worse than assumed. Poster presented at the26th Annual Conference of the Society for Industrial and Organizational Psychology,Chicago, IL.

∗Kieffer KM, Schinka JA, Curtiss G. (2004). Person-environment congruence and person-ality domains in the prediction of job performance and work quality. Journal ofCounseling Psychology, 51, 168–177.

∗Kim K. (2008). An investigation of the relationship between cultural competenceand expatriate job performance. Unpublished doctoral dissertation, Texas A&MUniversity.

∗Kraus E. (2002). Personality and job performance: The mediating roles of leader-memberexchange quality and action control. Unpublished doctoral dissertation, FloridaInternational University.

∗Lao RY. (2001). Faking in personality measures: Effects on the prediction of job perfor-mance. Unpublished doctoral dissertation, North Carolina State University.

Le H, Schmidt FL, Putka DJ. (2009). The multifaceted nature of measurement artifactsand its implications for estimating construct-level relationships. Organizational Re-search Methods, 12, 165–200.

∗Lee S. (2000). Cross-cultural validity of personality traits for predicting job performanceof Korean engineers. Unpublished doctoral dissertation, The Ohio State University.

Lievens F, De Corte W, Schollaert E. (2008). A closer look at the frame-of-reference effectin personality scale scores and validity. Journal of Applied Psychology, 93, 268–279.

Lounsbury JW, Gibson LW. (2006). Personal Style Inventory: A personality measurementsystem for work and school settings. Knoxville, TN: Resource Associates, Inc.

∗Lounsbury JW, Gibson LW, Hamrick FL. (2004). The development and validation of apersonological measure of work drive. Journal of Business and Psychology, 18,427–451.

∗Loveland JM, Gibson LW, Lounsbury JW, Huffstetler BC. (2005). Broad and narrowpersonality traits in relation to the job performance of camp counselors. Child andYouth Care Forum, 34, 241–255.

∗Lubbers RW. (2003). Self-efficacy and affective well-being among young workers: Exam-ining job quality as an antecedent of employee health and performance outcomes.Unpublished doctoral dissertation, University of Toronto.

∗McCormack L, Mellor D. (2002). The role of personality in leadership: An application ofthe five-factor model in the Australian military. Military Psychology, 14, 179–197.

McDaniel MA, Rothstein HR, Whetzel DL. (2006). Publication bias: A case study of fourtest vendors. PERSONNEL PSYCHOLOGY, 59, 927–953.

Mead GH. (1934). Mind, self, and society from the standpoint of a social behaviorist.Chicago, IL: University of Chicago Press.

∗Minbashian A, Bright JEH, Bird KD. (2009). Complexity in the relationships amongthe subdimensions of extraversion and job performance in managerial occupations.Journal of Occupational and Organizational Psychology, 82, 537–549.


Mischel W. (1973). Toward a cognitive social learning reconceptualization of personality.Psychological Review, 80, 252–283.

∗Moberg PJ. (1996). Social skills, personality, occupational interest, and job performance:A multivariate structural analysis. Unpublished doctoral dissertation, University ofIllinois at Urbana-Champaign.

Morgeson FP, Campion MA, Dipboye RL, Hollenbeck JR, Murphy K, Schmitt N. (2007).Reconsidering the use of personality tests in personnel selection contexts. PERSON-NEL PSYCHOLOGY, 60, 683–729.

∗Morgeson FP, Reider MH, Campion MA. (2005). Selecting individuals in team settings:The importance of social skills, personality characteristics, and teamwork knowl-edge. PERSONNEL PSYCHOLOGY, 58, 583–611.

∗Morrison JD, Jr., Abraham JD. (2003). Relationship between the Performance PerspectiveInventory’s Selling selling scale and job performance of real estate agents. AppliedH.R.M. Research, 8, 85–88.

Mount MK, Barrick MR. (1995). The Big Five personality dimensions: Implications forresearch and practice in human resource management. Research in Personnel andHuman Resources Management, 13, 153–200.

Mount MK, Barrick MR. (2007). Personal Characteristics Inventory. Libertyville, IL:Wonderlic, Inc.

∗Mount MK, Barrick MR, Strauss JP. (1994). Validity of observer ratings of the Big Fivepersonality factors. Journal of Applied Psychology, 79, 272–280.

∗Mount MK, Barrick MR, Strauss JP. (1999). The joint relationship of conscientious-ness and ability with performance: Test of the interaction hypothesis. Journal ofManagement, 25, 707–721.

∗Mount MK, Oh I-S, Burns M. (2008). Incremental validity of perceptual speed and accu-racy over general mental ability. PERSONNEL PSYCHOLOGY, 61, 113–139.

∗Mount MK, Witt LA, Barrick MR. (2000). Incremental validity of empirically keyedbiodata scales over GMA and the five factor personality constructs. PERSONNEL

PSYCHOLOGY, 53, 299–323.Mowen JC, Spears N. (1999). Understanding compulsive buying among college students:

A hierarchical approach. Journal of Consumer Psychology, 8, 407–430.∗Norris GW. (2002). Using measures of personality and self-efficacy to predict work per-

formance. Unpublished doctoral dissertation, The Ohio State University.Ones DS, Dilchert S, Viswesvaran C, Judge TA. (2007). In support of personality assess-

ments in organizational settings. PERSONNEL PSYCHOLOGY, 60, 995–1027.∗Pace VL, Brannick MT. (2010). Improving prediction of work performance through frame-

of-reference consistency: Empirical evidence using openness to experience. Inter-national Journal of Selection and Assessment, 18, 230–235.

Page RC. (2009). Page Work Behavior Inventory: Manual and user’s guide. Minnetonka,MN: Assessment Associates International.

Pearlman K, Sanchez JI. (2010). Work analysis. In Farr JL, Tippins NT (Eds.), Handbookof employee selection. (pp. 73–98). New York, NY: Routledge.

∗Piedmont RL, Weinstein HP. (1994). Predicting supervisor ratings of job performanceusing the NEO Personality Inventory. The Journal of Psychology, 128, 255–265.

∗Plemmons S, Huffman AH, Henning JB, Youngcourt SS. (2007, April). Work ethic asa predictor of job performance: A multidimensional perspective. Poster presentedat the 22nd Annual Conference of the Society for Industrial and OrganizationalPsychology, New York, NY.

∗Postlethwaite BE, Robbins SB, Oh I-S, Casillas A. (2010). Using personality facets,multiple performance criteria, and occupational complexity to better understandpersonality—job performance relationships. Unpublished manuscript.


∗Quick BG. (2003). The demand-control/support model of job strain, neuroticism, andconscientiousness as predictors of job outcomes in juvenile correctional officers.Unpublished doctoral dissertation, Virginia Commonwealth University.

∗Radwinsky RL. (1999). The effect of psychological contracts on the performance oftemporary employees. Unpublished doctoral dissertation, The University of Tulsa.

Reddock CM, Biderman MD, Nguyen NT. (2011). The relationship of reliabilityand validity of personality tests to frame-of-reference instructions and within-person inconsistency. International Journal of Selection and Assessment, 19, 119–131.

Roberts BW, Donahue EM. (1994). One personality, multiple selves: Integrating personalityand social roles. Journal of Personality, 62, 199–218.

Robie C, Born MP, Schmit MJ. (2001). Personal and situational determinants of personalityresponses: A partial reanalysis and reinterpretation of the Schmit et al. (1995) data.Journal of Business and Psychology, 16, 101–117.

∗Robie C, Brown DJ, Bly PR. (2008). Relationship between major personality traits andmanagerial performance: Moderating effects of derailing traits. International Jour-nal of Management, 25, 131–139.

∗Robie C, Ryan AM. (1999). Effect of nonlinearity and heteroscedasticity on the validityof conscientiousness in predicting overall job performance. International Journal ofSelection and Assessment, 7, 157–169.

Robie C, Schmit MJ, Ryan AM, Zickar MJ. (2000). Effects of item context specificity onthe measurement equivalence of a personality inventory. Organizational ResearchMethods, 3, 348–365.

∗Robson S, Abraham J. (2008, April). Observation of performance: Implications for thevalidity of personality. Poster presented at the 23rd Annual Conference of the Societyfor Industrial and Organizational Psychology, San Francisco, CA.

∗Robson SM, Abraham JD, Weiner J. (2010). Characteristics of successful direct supportprofessionals: An examination of personality and cognitive ability requirements.International Journal of Selection and Assessment, 18, 215–219.

∗Rose RM, Fogg LF, Helmreich RL, McFadden TJ. (1994). Psychological predictors ofastronaut effectiveness. Aviation, Space, and Environmental Medicine, 65, 910–915.

Rosenthal R. (1979). The “file drawer problem” and tolerance for null results. PsychologicalBulletin, 86, 638–641.

Rosse JG, Miller JL, Stecher MD. (1994). A field study of job applicants’ reactions topersonality and cognitive ability testing. Journal of Applied Psychology, 79, 987–992.

Rothstein HR. (1990). Interrater reliability of job performance ratings: Growth to asymptotelevel with increasing opportunity to observe. Journal of Applied Psychology, 75,322–327.

Rynes SL, Connerly ML. (1993). Applicant reactions to alternative selection procedures.Journal of Business and Psychology, 7, 261–277.

Sackett PR. (2010). Final thoughts on the selection and assessment field. In Scott JC,Reynolds DH. (Eds.). Handbook of workplace assessment (pp. 757-777). SanFrancisco, CA: Jossey-Bass.

Salgado JF. (1997). The five factor model of personality and job performance in theEuropean Community. Journal of Applied Psychology, 82, 30–43.

Salgado JF, Anderson N, Moscoso S, Bertua C, de Fruyt F, Rolland JP. (2003). A meta-analytic study of general mental ability validity for different occupations in theEuropean Community. Journal of Applied Psychology, 88, 1068–1081.

∗Salgado JF, Rumbo A. (1997). Personality and job performance in financial servicesmanagers. International Journal of Selection and Assessment, 5, 91–100.


Saucier G. (1994). Mini-Markers: A brief version of Goldberg’s unipolar Big-Five markers.Journal of Personality Assessment, 63, 506–516.

Schmidt FL, Le H, Ilies R. (2003). Beyond alpha: An empirical investigation of the effectsof different sources of measurement error on reliability estimates for measures ofindividual differences constructs. Psychological Methods, 8, 206–224.

Schmit MJ, Kihm JA, Robie C. (2000). Development of a global measure of personality.PERSONNEL PSYCHOLOGY, 53, 153–193.

Schmit MJ, Ryan AM, Stierwalt SL, Powell AB. (1995). Frame-of-reference effects on per-sonality scale scores and criterion-related validity. Journal of Applied Psychology,80, 607–620.

Schmitt N. (2004). Beyond the Big Five: Increases in understanding and practical utility.Human Performance, 17, 347–357.

Schmitt N, Gooding RZ, Noe RA, Kirsch M. (1984). Meta-analyses of validity studiespublished between 1964 and 1982 and the investigation of study characteristics.PERSONNEL PSYCHOLOGY, 37, 407–422.

∗Shin H-C. (2006). Main and interaction effects of personality and P-O fit in personnelselection. Unpublished doctoral dissertation, University of Tulsa.

∗Small EE, Diefendorff JM. (2006). The impact of contextual self-ratings and observer rat-ings of personality on the personality-performance relationship. Journal of AppliedSocial Psychology, 36, 297–320.

∗Steffensmeier J. (2008). Situational constraints and personality as antecedents of or-ganizational citizenship behaviors. Unpublished doctoral dissertation, ClemsonUniversity.

∗Stewart GL, Carson KP. (1995). Personality dimensions and domains of service per-formance: A field investigation. Journal of Business and Psychology, 9, 365–378.

∗Strauss JP, Barrick MR, Connerly ML. (2001). An investigation of personality similarityeffects (relational and perceived) on peer and supervisor ratings and the role offamiliarity and liking. Journal Occupational and Organizational Psychology, 74,637–657.

∗Sy T, Tram S, O’Hara LA. (2006). Relation of employee and manager emotional intel-ligence to job satisfaction and performance. Journal of Vocational Behavior, 68,461–473.

∗Tichon MA. (2005). Personnel selection in the transportation sector: An investigation ofpersonality traits in relation to the job performance of delivery drivers. Unpublisheddoctoral dissertation, University of Tennessee.

∗Timmerman TA. (2004). Relationships between NEO-PI-R personality measures and jobperformance ratings of inbound call center employees. Applied HRM Research,9, 35–38.

∗Tracey JB, Sturman MC, Tews MJ. (2007). Ability versus personality: Factors that predictemployee job performance. Cornell Hotel and Restaurant Administration Quarterly,48, 313–322.

∗Tull, KT. (1997). The effects of faking behavior on the prediction of sales performanceusing the Guilford Zimmerman Temperament Survey and the NEO Five FactorInventory. Unpublished doctoral dissertation, University of Akron.

∗Van Scotter JR, Motowidlo SJ. (1996). Interpersonal facilitation and job dedication asseparate facets of contextual performance. Journal of Applied Psychology, 81, 525–531.

∗Vargas V. (2005). Five-factor model of personality, burnout, and performance in childprotective service work. Unpublished doctoral dissertation, Alliant InternationalUniversity.


Viswesvaran C, Ones DS, Schmidt FL. (1996). Comparative analysis of the reliability ofjob performance ratings. Journal of Applied Psychology, 81, 557–574.

Wang Q, Bowling NA, Eschleman KJ. (2010). A meta-analytic investigation of work andgeneral locus of control. Journal of Applied Psychology, 95, 761–768.

∗Weilbaecher AD. (2000). Self-report personality measures of fake good in employmentselection. Unpublished doctoral dissertation, University of Tennessee.

∗Westerman JW, Simmons BL. (2007). The effects of work environment on the personality-performance relationship: An exploratory study. Journal of Managerial Issues, 19,288–305.

∗Williams M. (1999). When is personality a predictor of performance? The moderating roleof autonomy. Unpublished doctoral dissertation, Florida International University.

∗Witt LA. (2002). The interactive effects of extraversion and conscientiousness on perfor-mance. Journal of Management, 28, 835–851.

∗Witt LA, Carlson DS. (2006). The work-family interface and job performance: Moder-ating effects of conscientiousness and perceived organizational support. Journal ofOccupational Health Psychology, 11, 343–357.

Wood D, Roberts BW. (2006). Cross-sectional and longitudinal tests of the personality androle identity structural model (PRISM). Journal of Personality, 74, 779–809.

Wright JC, Mischel W. (1987). A conditional approach to dispositional constructs: The localpredictability of social behavior. Journal of Personality and Social Psychology, 53,1159–1177.

∗Yoon K. (1997). General mental ability and the Big Five personality dimensions: Aninvestigation of the cross-cultural generalizability of their construct and criterion-related validities in Korea. Unpublished doctoral dissertation, University of Iowa.


APP

EN

DIX

ASu

mm

ary

Info

rmat

ion

Reg

ardi

ngth

eC

odin

gof

Pers

onal

ity

Mea

sure

sas

Non

cont

extu

aliz

edor

Con

text

uali

zed

scal

es

Scal

ena

me

Sour

ceD

evel

opm

ent

Cat

egor

yIn

stru

ctio

nalf

ram

eSa

mpl

eite

ms

Big

Five

Inve

ntor

yJo

hn,D

onah

ue,&

Ken

tle(1

991)

Gen

eral

Non

cont

extu

aliz

ed“T

hefo

llow

ing

stat

emen

tsco

ncer

nyo

urpe

rcep

tion

Ise

em

ysel

fas

som

eone

who

ista

lkat

ive.

abou

tyou

rsel

fin

ava

riet

yof

situ

atio

ns.”

Ise

em

ysel

fas

som

eone

who

finds

faul

twith

othe

rs.

Ise

em

ysel

fas

som

eone

who

isfu

llof

ener

gy.

Em

ploy

eeQ

ualit

yIn

vent

ory

Cha

n(2

004)

Gen

eral

Non

cont

extu

aliz

edA

sks

part

icip

ants

the

exte

ntto

whi

chth

eyag

ree

ordi

sagr

eew

ithea

ch

Itis

easy

for

me

tofit

inw

ithth

epe

ople

arou

ndm

e.st

atem

ent.

Peop

lew

hokn

owm

ew

ell

wou

ldde

scri

bem

eas

disc

iplin

ed.

Ibe

com

eov

erw

helm

edby

emot

ions

.

Five

-Dim

ensi

onal

Hig

gins

etal

.(20

07)

Gen

eral

Non

cont

extu

aliz

ed“W

hatp

oint

onth

esc

ale

Act

ive.

Tem

pera

men

tbe

stde

scri

bes

you?

”Sh

y.In

vent

ory

Talk

ativ

e.

cont

inue

d


APP

EN

DIX

A(c

ontin

ued)

Scal

ena

me

Sour

ceD

evel

opm

ent

Cat

egor

yIn

stru

ctio

nalf

ram

eSa

mpl

eite

ms

Five

Fact

orM

odel

Gill

&H

odgk

inso

nG

ener

alC

onte

xtua

lized

Scal

ean

chor

sw

ere

revi

sed

Sens

itive

.Q

uest

ionn

aire

(200

7)to

refle

cta

wor

kse

tting

.C

oope

rativ

e.“1

=N

otlik

em

eat

alla

tw

ork

...5

=V

ery

like

me

atw

ork”

(pp.

738–

739)

.

Punc

tual

.

Gol

dber

g’s

mar

kers

Gol

dber

g(1

992)

Gen

eral

Non

cont

extu

aliz

ed“D

escr

ibe

your

self

asyo

uA

nxio

us.

for

the

Big

-Fiv

ear

ege

nera

llyC

oope

rativ

e.fa

ctor

stru

ctur

eor

typi

cally

...”

Inte

llect

ual.

Glo

balP

erso

nalit

yIn

vent

ory

Schm

it,K

ihm

,&R

obie

(200

0)G

ener

alN

onco

ntex

tual

ized

Res

pond

ents

are

aske

dto

indi

cate

how

clos

ely

stat

emen

tsde

scri

beth

em.

Und

erce

rtai

nci

rcum

stan

ces

itis

OK

tobr

eak

the

rule

s.Pe

ople

can

serv

eas

exce

llent

tool

sfo

rge

tting

wha

tyou

wan

tor

need

.I

don’

tmin

dch

angi

ngpl

ans

atth

ela

stm

inut

e. cont

inue

d


APP

EN

DIX

A(c

ontin

ued)

Scal

ena

me

Sour

ceD

evel

opm

ent

Cat

egor

yIn

stru

ctio

nalf

ram

eSa

mpl

eite

ms

Hog

anPe

rson

ality

Inve

ntor

yH

ogan

&H

ogan

(200

7)G

ener

alN

onco

ntex

tual

ized

“Thi

sas

sess

men

tcon

tain

sa

seri

esof

stat

emen

ts.

Ius

ually

try

tote

llpe

ople

wha

tthe

yw

antt

ohe

ar.

Rea

dea

chon

e,de

cide

Iam

nota

good

team

play

er.

how

you

feel

abou

tit,

and

then

mar

kyo

uran

swer

.”I

don’

thol

dgr

udge

s.

Inte

rnat

iona

lG

oldb

erg

(199

9)G

ener

alN

onco

ntex

tual

ized

“Des

crib

eyo

urse

lfas

you

Am

the

life

ofth

epa

rty.

Pers

onal

ityIt

emge

nera

llyar

eno

w,n

otas

Pay

atte

ntio

nto

deta

ils.

Pool

you

wis

hto

bein

the

futu

re.”

Wor

ryab

outt

hing

s.

Min

i-M

arke

rsSa

ucie

r(1

994)

Gen

eral

Non

cont

extu

aliz

ed“D

escr

ibe

your

self

asyo

uB

ashf

ul.

gene

rally

are

now

,not

asO

rgan

ized

.yo

uw

ish

tobe

inth

efu

ture

.”Ta

lkat

ive.

Mow

en&

Spea

rsM

owen

&Sp

ears

(199

9)G

ener

alN

onco

ntex

tual

ized

Part

icip

ants

indi

cate

toFe

elba

shfu

lwith

othe

rs.

exte

ntto

whi

chO

rgan

ized

.st

atem

ents

desc

ribe

them

.K

ind

toot

hers

.

cont

inue

d


APP

EN

DIX

A(c

ontin

ued)

Scal

ena

me

Sour

ceD

evel

opm

ent

Cat

egor

yIn

stru

ctio

nalf

ram

eSa

mpl

eite

ms

NE

O(i

nclu

des

NE

O-P

I,N

EO

-PI-

R,

Cos

ta&

McC

rae

(199

2)G

ener

alN

onco

ntex

tual

ized

Res

pond

ents

indi

cate

the

exte

ntto

whi

chth

eyag

ree

ordi

sagr

eew

ith

Frig

hten

ing

thou

ghts

som

etim

esco

me

into

my

head

.N

EO

-FFI

)st

atem

ents

.I

don’

tget

muc

hpl

easu

refr

omch

attin

gw

ithpe

ople

.I

have

ave

ryac

tive

imag

inat

ion.

Page

Wor

kB

ehav

ior

Inve

ntor

yPa

ge(2

009)

Wor

kpla

ceC

onte

xtua

lized

Pers

onal

com

mun

icat

ion

with

the

auth

orof

the

PWB

Iin

dica

ted

that

the

Ifin

dit

very

easy

tost

art

conv

ersa

tions

with

stra

nger

s.in

stru

ctio

nsre

ferr

edto

aw

ork-

spec

ific

cont

ext.

Ite

ndto

feel

nerv

ous

indi

fficu

ltsi

tuat

ions

.I

reco

ver

quic

kly

from

inte

nse

stre

ssor

pres

sure

.

Perf

orm

ance

Pers

pect

ives

Abr

aham

&M

orri

son

(200

9)W

orkp

lace

Non

cont

extu

aliz

ed“D

escr

ibe

your

self

asyo

uge

nera

llyar

eno

w,n

otas

Don

’tm

ind

bein

gth

ece

nter

ofat

tent

ion.

Inve

ntor

yyo

uw

ish

tobe

inth

efu

ture

.”B

elie

veth

atpl

anni

ngah

ead

mak

esth

ings

turn

out

bette

r.A

mab

leto

see

the

best

ina

situ

atio

n.

cont

inue

d


APP

EN

DIX

A(c

ontin

ued)

Scal

ena

me

Sour

ceD

evel

opm

ent

Cat

egor

yIn

stru

ctio

nalf

ram

eSa

mpl

eite

ms

Pers

onal

Cha

ract

eris

tics

Inve

ntor

y

Mou

nt&

Bar

rick

(200

7)W

orkp

lace

Non

cont

extu

aliz

ed“R

ead

each

stat

emen

tand

choo

seth

eon

eth

atbe

stde

scri

bes

you.

”

Ilik

ein

itiat

ing

conv

ersa

tions

with

peop

leI

dono

tkno

w.

Ipr

efer

read

ing

abo

okto

wat

chin

gte

levi

sion

.O

ther

sha

vede

scri

bed

me

asbe

ing

too

lax

atw

ork.

Pers

onal

Styl

eIn

vent

ory

Lou

nsbu

ry&

Gib

son

(200

6)W

orkp

lace

Con

text

ualiz

ed“M

osto

fth

ePS

Iite

ms

are

wri

tten

from

the

Ilik

eto

keep

my

wor

kne

atan

dw

ello

rgan

ized

.pe

rspe

ctiv

eof

wor

k”(p

.3).

IfI

have

argu

men

twith

som

eone

atw

ork,

Ilik

eto

beth

efir

stto

apol

ogiz

eaf

terw

ards

.

Rev

ised

Gol

dber

gD

eGro

ot&

Klu

empe

rG

ener

alC

onte

xtua

lized

“Des

crib

eyo

urse

lfas

you

Anx

ious

.(2

007)

typi

cally

are

atw

ork.

”C

oope

rativ

e.In

telle

ctua

l.

cont

inue

d


APP

EN

DIX

A(c

ontin

ued)

Scal

ena

me

Sour

ceD

evel

opm

ent

Cat

egor

yIn

stru

ctio

nalf

ram

eSa

mpl

eite

ms

Rev

ised

IPIP

Farm

er(1

999)

Gen

eral

Con

text

ualiz

edPa

rtic

ipan

tsw

ere

aske

dto

indi

cate

how

accu

rate

lyI

amco

ncer

ned

abou

tmy

cow

orke

rs.

each

stat

emen

tdes

crib

edth

em.

Im

ake

peop

leat

wor

kfe

elat

ease

.I

seld

omge

tem

otio

nala

tw

ork.

Rev

ised

NE

OE

nglis

h(2

003)

;H

unth

ause

net

al.

(200

3)

Gen

eral

Con

text

ualiz

edPa

rtic

ipan

tsw

ere

give

nve

rbal

and

wri

tten

inst

ruct

ions

to“t

hink

Itr

yto

beco

urte

ous

toev

eryo

neI

mee

twhi

lew

orki

ng.

abou

thow

they

beha

veat

wor

kw

hen

resp

ondi

ngto

each

stat

emen

t”(p

.547

).

Atw

ork,

Ike

epm

ybe

long

ings

neat

and

clea

n.

Hal

fhill

etal

.(20

05)

Gen

eral

Con

text

ualiz

edIt

ems

wer

ead

ded

orre

vise

d“t

ore

fer

toa

mili

tary

Itr

yto

beco

urte

ous

toev

eryo

neI

mee

tatw

ork.

wor

ken

viro

nmen

t”(p

.47)

.In

my

mili

tary

wor

ken

viro

nmen

t,I

ama

chee

rful

,hig

h-sp

irite

dpe

rson

.

cont

inue

d


APP

EN

DIX

A(c

ontin

ued)

Scal

ena

me

Sour

ceD

evel

opm

ent

Cat

egor

yIn

stru

ctio

nalf

ram

eSa

mpl

eite

ms

Rev

ised

Smal

l&D

iefe

ndor

ffG

ener

alC

onte

xtua

lized

Inst

ruct

ions

wer

ere

vise

dto

Bas

hful

.M

ini-

Mar

kers

(200

6)re

flect

aw

ork

setti

ng.

Org

aniz

ed.

“Des

crib

eyo

urse

lfas

you

are

atw

ork”

(p.3

05).

Talk

ativ

e.

Wor

kKey

sTa

lent

AC

T(2

007)

Wor

kpla

ceN

onco

ntex

tual

ized

“Rea

dea

chst

atem

enta

ndI

amno

tver

ycr

eativ

e.A

sses

smen

tin

dica

teho

ww

elli

tde

scri

bes

you

byfil

ling

Itis

hard

for

me

tore

adso

cial

cues

.in

the

appr

opri

ate

oval

prec

edin

gea

chst

atem

ent.”

It’s

okay

tobe

diso

rgan

ized

.


APP

EN

DIX

BSu

mm

ary

ofM

eta-

Ana

lyti

cD

atab

ase

Scal

eD

esig

nr

Pred

icto

rre

liabi

lity

Stud

yR

efer

ence

Sour

ceSa

mpl

ede

sign

scal

eU

seFO

Rn

AC

ES

EX

OA

CE

SE

XO

Abr

aham

&M

orri

son

(200

3)1

11

PPI

21

55.1

0.1

4−.

08−.

25−.

07.7

8.8

8.8

5.7

9.8

4A

vis

(200

1)2

11

NE

O1

117

3.1

6.0

9.1

8.2

0.0

6–

––

––

Bac

ha(2

003)

21

1N

EO

11

150

.08

.29

.16

.09

–.9

0.9

1.9

1.8

5–

Bar

rick

&M

ount

(199

3)1

11

PCI

21

146

.01

.25

.01

.14

.06

.67

.89

.83

.85

.85

Bar

rick

&M

ount

(199

6)1

22

PCI

21

147

.00

.26

.18

−.04

−.05

.82

.87

.86

.85

.82

Bar

rick

&M

ount

(199

6)1

22

PCI

21

139

.06

.27

.15

.02

.07

.82

.87

.86

.85

.82

Bar

rick

etal

.(19

94)

11

1PC

I2

110

9–

.25

––

––

.85

––

–B

auer

etal

.(20

06)

11

2M

M1

167

––

–.2

5–

––

–.8

2–

Ber

gner

etal

.(20

10)

11

1N

EO

11

128

.14

.11

.11

.16

.12

.70

.83

.86

.73

.71

Ber

ryet

al.(

2007

)1

11

PWB

I2

226

1.0

3.1

3.1

7.2

1.1

6.9

2.9

2.9

3.9

5.9

4B

ing

&B

urro

ughs

(200

1)1

11

NE

O2

112

1.1

5.2

3–

––

.68

.79

––

–B

ing

&B

urro

ughs

(200

1)1

12

PSI

22

162

.32

.21

––

–.7

2.8

0–

––

Boy

es(2

005)

31

1N

EO

11

254

.05

.14

.04

−.05

−.05

.72

.87

.85

.78

.72

Boy

es(2

005)

31

1N

EO

11

106

−.16

.19

.11

.19

.07

.79

.85

.88

.84

.73

Bro

wn

etal

.(20

02)

11

1M

S1

124

9−.

07.1

8.0

7.0

7−.

10.8

5.7

3.8

8.8

6.8

3B

urke

&W

itt(2

002)

11

1PC

I2

111

4.0

7.2

3.2

1−.

03.1

2.7

9.7

1.8

0.8

6.7

0B

uttig

ieg

(200

6)1

13

IPIP

11

180

.05

.08

.15

−.02

.01

.87

.87

.90

.90

.86

Cal

igiu

ri(2

000)

11

1H

PI1

194

.19

.34

.10

−.06

−.05

––

––

–C

han

(200

4)2

11

EQ

I1

120

1.1

9.2

0.0

9.1

5.1

2.9

3.9

2.8

9.8

6.8

3

cont

inue

d

488 PERSONNEL PSYCHOLOGYA

PPE

ND

IXB

(con

tinue

d)

Scal

eD

esig

nr

Pred

icto

rre

liabi

lity

Stud

yR

efer

ence

Sour

ceSa

mpl

eD

esig

nsc

ale

Use

FOR

nA

CE

SE

XO

AC

ES

EX

O

Cha

n&

Schm

itt(2

002)

11

1N

EO

11

160

.21

.08

.20

.30

.36

.67

.81

.87

.84

.69

Con

nolly

(200

1)2

11

IPIP

11

107

.09

−.07

.17

−.15

.07

.84

.84

.88

.90

.77

Con

te&

Gin

toft

(200

5)1

11

MM

11

174

.06

.05

−.02

.25

.00

.85

.83

.75

.80

.83

Coo

k(2

005)

22

3PS

I2

225

0.1

9.2

3.3

4.2

1.1

0–

––

––

Cre

spo

(200

7)2

11

GPI

21

630

–.1

4–

––

––

––

–C

utch

in(1

998)

21

1N

EO

11

126

−.13

.12

−.10

−.07

−.05

.84

.89

.93

.89

.90

DeG

root

&K

luem

per

(200

7)1

11

GB

G1

115

4.0

1.1

4–

.13

–.9

1.9

2–

.88

–D

eGro

ot&

Klu

empe

r(2

007)

11

1G

BG

12

154

−.03

.19

–.2

5–

.91

.92

–.9

1–

Die

fend

orff

etal

.(20

02)

11

1G

BG

11

130

.11

.06

––

–.9

1.9

2–

––

Die

fend

orff

etal

.(20

06)

11

1M

M1

117

7–

.20

––

––

.79

––

–D

ierd

orff

(200

2)2

11

NE

O1

115

8–

.06

––

––

.86

––

–D

onav

an(1

999)

21

1M

S1

121

2−.

02.1

0−.

22.1

9.0

1.9

1.8

1.9

0.9

0.8

4E

nglis

h(2

003)

21

1N

EO

11

73.1

9.1

7.0

6.1

6−.

08.8

5.8

9.8

6.8

2.6

9E

nglis

h(2

003)

21

1N

EO

12

71.2

5.2

5.3

3.1

8.2

2.8

3.8

9.8

2.8

1.6

0Fa

rmer

(199

9)2

11

IPIP

11

101

.02

−.08

.04

––

––

––

–Fa

rmer

(199

9)2

11

IPIP

12

81.2

1.1

8−.

05–

––

––

––

Gill

&H

odgk

inso

n(2

007)

11

1FF

MQ

22

223

.19

.24

.14

.09

−.08

––

––

–G

ill&

Hod

gkin

son

(200

7)1

11

FFM

Q2

216

8−.

012

.01

−.11

.29

−.05

––

––

–G

ill&

Hod

gkin

son

(200

7)1

11

FFM

Q2

213

5.1

0.3

0.1

6−.

07.0

4–

––

––

Gri

ffin

&H

eske

th(2

004)

11

1IP

IP1

155

––

––

.09

––

––

.89

Gri

ffin

&H

eske

th(2

004)

11

1IP

IP1

113

1–

––

–.0

9–

––

–.8

7G

riffi

n&

Hes

keth

(200

4)1

11

NE

O1

128

––

––

.05

––

––

.88

Hal

fhill

etal

.(20

05)

11

1N

EO

12

422

.22

.12

––

–.8

0.8

3–

––

Hat

trup

etal

.(20

05)

11

1Sp

ecia

l1

212

1–

.13

––

––

.72

––

–

cont

inue

d


APP

EN

DIX

B(c

ontin

ued)

Scal

eD

esig

nr

Pred

icto

rre

liabi

lity

Stud

yR

efer

ence

Sour

ceSa

mpl

eD

esig

nSc

ale

Use

FOR

nA

CE

SE

XO

AC

ES

EX

O

Hay

eset

al.(

1994

)1

11

HPI

11

130

.01

.29

.16

−.09

−.18

––

––

–H

ense

III

(200

0)2

11

NE

O1

115

2.0

1.0

9.0

3.0

6−.

11.8

0.8

0.8

5.7

6.7

5H

iggi

nset

al.(

2007

)1

11

FDT

I1

196

.09

.23

.06

.14

.13

––

––

–H

iggi

nset

al.(

2007

)1

11

FDT

I1

180

.07

.08

.01

−.08

.03

––

––

–H

unth

ause

net

al.(

2003

)1

11

NE

O1

210

4−.

02.3

1.1

2.2

6.2

7–

––

––

Hun

thau

sen

etal

.(20

03)

11

1N

EO

11

102

−.03

.10

.06

−.10

−.12

––

––

–Ja

ckso

net

al.(

2006

)1

11

BFI

11

124

.14

.24

−.05

.02

−.14

––

––

–Ja

cobs

etal

.(19

96)

11

1H

PI1

157

4−.

02.0

0.0

1−.

07−.

02–

––

––

Judg

e&

Ere

z(2

007)

11

1B

FI1

112

2.0

9.2

9.1

3.1

0–

.71

.77

.83

.70

–Ju

dge

etal

.(19

99)

11

1N

EO

11

514

––

––

.06

––

––

.68

Judg

eet

al.(

2006

)1

11

BFI

11

131

−.03

.11

.06

.13

−.02

.82

.80

.81

.85

.83

Kie

ffer

etal

.(20

04)

11

1N

EO

11

514

.06

.07

−.06

.00

.05

––

.86

–.7

4K

im(2

008)

21

1IP

IP1

133

8−.

05.2

4.0

3.0

6.0

9.7

2.7

0.7

8.8

3.7

3K

raus

(200

2)2

11

IPIP

11

94−.

04−.

01.1

1.1

6.1

4.6

2.7

8.8

1.8

8.7

6L

ao(2

001)

21

1N

EO

11

395

–−.

10–

––

–.8

7–

––

Lee

(200

0)2

11

NE

O1

131

5−.

02.1

1−.

01−.

05−.

05.8

2.9

0.8

9.8

9.8

4L

ouns

bury

etal

.(20

04)

11

1PS

I2

210

5−.

12.1

3.0

9−.

13–

––

––

–L

ouns

bury

etal

.(20

04)

11

1PS

I2

218

8.1

5.1

9.1

9.0

8.0

6–

––

––

Lou

nsbu

ryet

al.(

2004

)1

11

PSI

22

154

.42

.30

.48

.36

––

––

––

Lou

nsbu

ryet

al.(

2004

)1

11

PSI

22

238

.26

.19

.14

.25

.25

––

––

–L

ovel

and

etal

.(20

05)

11

2PS

I2

214

5.2

8.2

7.1

8.2

7–

––

––

–L

ubbe

rs(2

003)

21

2G

BG

11

195

–.2

9–

––

–.8

7–

––

McC

orm

ack

&M

ello

r(2

002)

11

1N

EO

11

99.0

0.2

6.0

9−.

17.0

3–

––

––

cont

inue

d


APP

EN

DIX

B(c

ontin

ued)

Scal

eD

esig

nr

Pred

icto

rre

liabi

lity

Stud

yR

efer

ence

Sour

ceSa

mpl

eD

esig

nSc

ale

Use

FOR

nA

CE

SE

XO

AC

ES

EX

O

Min

bash

ian

etal

.(20

09)

11

1N

EO

11

179

–.1

5–

.10

––

.90

–.8

8–

Mob

erg

(199

6)2

11

NE

O1

125

1.1

0.1

4.1

7−.

06−.

08.8

8.9

0.9

2.9

0.9

1M

orge

son

etal

.(20

05)

11

1PC

I2

190

.15

.24

.17

.20

––

––

––

Mor

riso

n&

Abr

aham

(200

3)1

11

PPI

21

40–

.29

–.3

5–

––

––

–M

ount

etal

.(19

94)

11

1G

BG

11

105

.05

––

–.0

9.7

9–

––

.75

Mou

ntet

al.(

1999

)1

11

PCI

21

121

–.2

9–

––

––

––

–M

ount

etal

.(20

00)

11

1PC

I2

114

6.0

9.2

7.0

7.0

8.1

4–

––

––

Mou

ntet

al.(

2000

)1

11

PCI

21

222

.11

.21

.09

−.07

.09

––

––

–M

ount

etal

.(20

08)

11

1PC

I2

113

3.1

2.2

3−.

09.0

5.0

2–

––

––

Nor

ris

(200

2)2

11

NE

O1

112

3.2

0−.

07.1

2–

–.7

8.8

0.8

3–

–Pa

ce&

Bra

nnic

k(2

010)

11

1N

EO

12

83–

––

–.1

7–

––

–.8

6Pa

ce&

Bra

nnic

k(2

010)

11

1N

EO

11

83–

––

–−.

01–

––

–.8

8Pi

edm

ont&

Wei

nste

in(1

994)

11

1N

EO

11

207

−.13

.19

.12

.07

.04

––

––

–Pl

emm

ons

&H

uffm

an(2

007)

41

1B

FI1

111

5–

.34

––

––

.86

––

–Po

stle

thw

aite

etal

.(20

10)

51

1W

TA2

122

6.0

7−.

01.0

7−.

08−.

02–

––

––

Post

leth

wai

teet

al.(

2010

)5

11

WTA

21

444

.11

.11

.04

−.02

.01

––

––

–Po

stle

thw

aite

etal

.(20

10)

51

1W

TA2

122

3.2

1.2

5.1

9.0

7.1

7–

––

––

Qui

ck(2

003)

21

1N

EO

11

284

.05

.06

.14

.05

−.12

.63

.80

.77

.60

.57

Rad

win

sky

(199

9)2

11

IPIP

11

43.4

2.0

2.3

2–

–.8

2.7

9.8

6–

–R

obie

&R

yan

(199

9)1

11

PC

I1

114

6–

.24

––

––

––

––

Rob

ie&

Rya

n(1

999)

11

1N

EO

11

999

–.0

8–

––

––

––

–

cont

inue

d


APP

EN

DIX

B(c

ontin

ued)

Scal

eD

esig

nr

Pred

icto

rre

liabi

lity

Stud

yR

efer

ence

Sour

ceSa

mpl

eD

esig

nSc

ale

Use

FOR

nA

CE

SE

XO

AC

ES

EX

O

Rob

ie&

Rya

n(1

999)

11

1N

EO

11

200

–.2

5–

––

––

––

–R

obie

etal

.(20

08)

11

1G

PI2

114

4−.

13.1

5.0

4.3

2.2

2.7

0.7

2.6

5.8

2.6

7R

obso

n&

Abr

aham

(200

8)4

11

PPI

21

92.0

5.0

1−.

04.0

9.0

8–

––

––

Rob

son

etal

.(20

10)

12

2PP

I2

114

6.2

6.2

0.1

2.0

5.0

3.7

5.8

3.7

9.6

8.8

0R

ose

etal

.(19

94)

11

1N

EO

11

65.2

9−.

17.2

1−.

15−.

28–

––

––

Salg

ado

&R

umbo

(199

7)1

11

NE

O1

112

5−.

11.3

0.2

4.1

0−.

14.5

8.7

4.7

6.7

2.5

8Sh

in(2

006)

21

1H

PI1

121

0–

.08

.20

.25

––

––

––

Smal

l&D

iefe

ndor

f(2

006)

11

1M

M1

214

3.1

2.1

7.2

4−.

06−.

09.8

2.8

0.7

7.8

1.7

5Sm

all&

Die

fend

orf

(200

6)1

11

MM

11

143

.16

.21

.17

−.07

−.17

.83

.79

.79

.81

.74

Stef

fens

mei

er(2

008)

21

1H

PI1

112

9−.

12−.

01–

––

––

––

–St

ewar

t&C

arso

n(1

995)

11

1G

BG

11

105

.19

.33

–−.

18–

.65

.75

–.7

5–

Stra

uss

etal

.(20

01)

11

1G

BG

11

157

–.1

5.0

1.0

9–

–.8

4.7

4.7

2–

Syet

al.(

2006

)1

11

BFI

11

187

.20

−.09

.03

.32

.01

.82

.87

.85

.86

.75

Tic

hon

(200

5)2

11

PSI

22

39.3

4.3

2.1

6.1

2–

––

––

–T

imm

erm

an(2

004)

11

1N

EO

11

203

.16

.16

−.01

.00

−.03

.73

.87

.84

.80

.75

Tra

cey

etal

.(20

07)

11

1N

EO

11

64–

−.11

––

––

.82

––

–T

race

yet

al.(

2007

)1

11

NE

O1

117

7–

.34

––

––

.82

––

–T

ull(

1997

)2

11

NE

O1

119

7−.

04−.

01.0

3.1

3−.

14–

––

––

Van

Scot

ter

&M

otow

idlo

(199

6)1

11

NE

O1

150

8.1

4.2

2–

.10

–.7

3.7

9–

.71

–V

arga

s(2

005)

21

1PC

I2

116

3.0

7.0

5.1

1.0

8.1

2–

––

––

Wei

lbae

cher

(200

0)2

11

PSI

22

91−.

23.1

2−.

02.0

3.0

6.6

4.7

4.8

7.8

1.8

2W

eilb

aech

er(2

000)

21

1PS

I2

278

.29

.19

.24

.12

.09

.74

.82

.89

.84

.80

cont

inue

d


APP

EN

DIX

B(c

ontin

ued)

Scal

eD

esig

nr

Pred

icto

rre

liabi

lity

Stud

yR

efer

ence

Sour

ceSa

mpl

eD

esig

nSc

ale

Use

FOR

nA

CE

SE

XO

AC

ES

EX

O

Wei

lbae

cher

(200

0)2

11

PSI

22

145

.19

.24

.30

.12

.11

.71

.78

.85

.84

.80

Wes

term

an&

Sim

mon

s(2

007)

11

1N

EO

11

106

.12

.40

.14

.08

.03

––

––

–W

illia

ms

(199

9)2

11

NE

O1

194

−.01

.08

−.15

−.04

.01

.68

.81

.86

.77

.73

Witt

(200

2)1

11

PCI

21

144

–.3

4–

−.10

––

.74

–.8

5–

Witt

&C

arls

on(2

006)

11

1IP

IP1

113

6–

.31

––

––

.78

––

–Y

oon

(199

7)2

11

NE

O1

120

−.13

.51

−.10

−.10

−.38

.77

.88

.77

.85

.80

Yoo

n(1

997)

21

1N

EO

11

41.1

9.1

7.3

7.2

6−.

17.7

8.8

9.9

3.7

7.8

7Y

oon

(199

7)2

11

NE

O1

169

.07

.13

.14

−.05

−.03

.79

.92

.92

.86

.80

Yoo

n(1

997)

21

1N

EO

11

79.1

2.2

1.1

1.1

7.1

3.7

8.9

2.8

8.8

6.8

0Y

oon

(199

7)2

11

NE

O1

197

.00

.09

.05

.03

−.14

.82

.91

.90

.88

.83

Yoo

n(1

997)

21

1N

EO

11

120

.11

−.03

−.01

.03

−.14

.79

.91

.87

.82

.83

Yoo

n(1

997)

21

1N

EO

11

136

−.06

.19

.16

.13

−.01

.77

.88

.90

.86

.83

Not

e.Fo

rso

urce

,1=

artic

le,2

=do

ctor

aldi

sser

tatio

n,3

=m

aste

r’s

thes

is,4

=un

publ

ishe

dco

nfer

ence

pape

r,5

=un

publ

ishe

dte

chni

calr

epor

t;fo

rsa

mpl

e,1

=in

cum

bent

s,2

=ap

plic

ants

;for

stud

yde

sign

,1=

conc

urre

nt,2

=pr

edic

tive,

3=

unab

leto

dete

rmin

e;fo

rsca

le,B

FI=

Big

Five

Inve

ntor

y,C

CSQ

=C

usto

mer

Con

tact

Styl

esQ

uest

ionn

aire

,EQ

I=

Em

ploy

eeQ

ualit

yIn

vent

ory,

FDT

I=

Five

-Dim

ensi

onal

Tem

pera

men

tIn

vent

ory,

FFM

Q=

Five

Fact

orM

odel

Que

stio

nnai

re,G

BG

=G

oldb

erg,

GPI

=G

loba

lPe

rson

ality

Inve

ntor

y,H

PI=

Hog

anPe

rson

ality

Inve

ntor

y;IP

IP=

Inte

rnat

iona

lPe

rson

ality

Item

Pool

,M

M=

Sauc

ier’

sM

ini-

Mar

kers

,M

S=

Mow

enan

dSp

ears

;N

EO

=N

EO

-PI,

NE

O-P

I-R

,or

NE

O-F

FI,

OPQ

=O

ccup

atio

nal

Pers

onal

ityQ

uest

ionn

aire

,PC

I=

Pers

onal

Cha

ract

eris

tics

Inve

ntor

y,PP

I=

Perf

orm

ance

Pers

pect

ives

Inve

ntor

y,PS

I=

Pers

onal

Styl

eIn

vent

ory,

PWB

I=

Page

Wor

kB

ehav

ior

Inve

ntor

y,W

TA=

Wor

kKey

sTa

lent

Ass

essm

ent;

for

use,

1=

gene

ralp

urpo

se,2

=w

orkp

lace

purp

ose;

for

FOR

,1=

nonc

onte

xtua

lized

,2=

cont

extu

aliz

ed;n

=sa

mpl

esi

ze;A

=A

gree

able

ness

,C=

Con

scie

ntio

usne

ss,E

S=

Em

otio

nalS

tabi

lity,

EX

=E

xtra

vers

ion,

O=

Ope

nnes

sto

Exp

erie

nce;

allr

elia

bilit

ies

are

inte

rnal

cons

iste

ncy

(alp

ha)

relia

bilit

ies.


APPENDIX CValidity Estimates for Contextualized Measures of Personality as Moderated by

Contextualization Method

95% CI


ConscientiousnessInstructions only 3 558 .16 .00 .25 .00 .25 .25Items only 18 2,816 .19 .00 .30 .00 .30 .30Instructions and items 1 104 .31 – .48 – .48 .48

Emotional StabilityInstructions only 2 404 .19 .00 .29 .00 .29 .29Items only 15 2,111 .18 .13 .27 .20 .17 .37Instructions and items 1 104 .12 – .18 – .18 .18

ExtraversionInstructions only 3 558 .15 .10 .23 .16 .05 .41Items only 14 2,030 .15 .10 .23 .16 .15 .31Instructions and items 1 104 .26 – .40 – .40 .40

AgreeablenessInstructions only 3 558 .04 .00 .06 .00 .06 .06Items only 17 2,695 .18 .13 .29 .21 .19 .39Instructions and items 1 104 −.02 – −.03 – −.03 −.03

Openness to ExperienceInstructions only 2 404 .07 .10 .11 .16 −.11 .33Items only 11 1,670 .08 .07 .12 .11 .05 .19Instructions and items 1 104 .27 – .41 – .41 .41



APPENDIX DValidity Estimates for Contextualized Measures: Personal Style Inventory Versus

All Other Measures

95% CI


ConscientiousnessPersonal Style Inventory 11 1,595 .22 .00 .34 .00 .34 .34Other 11 1,883 .17 .03 .27 .05 .24 .30

Emotional StabilityPersonal Style Inventory 10 1,433 .23 .10 .35 .15 .26 .44Other 8 1,186 .12 .09 .18 .14 .08 .28

ExtraversionPersonal Style Inventory 10 1,433 .17 .09 .26 .14 .17 .35Other 8 1,259 .15 .10 .23 .16 .12 .34

AgreeablenessPersonal Style Inventory 11 1,595 .20 .14 .32 .22 .19 .45Other 10 1,762 .10 .09 .16 .15 .07 .25

Openness to ExperiencePersonal Style Inventory 6 990 .13 .00 .20 .00 .20 .20Other 8 1,188 .06 .10 .09 .16 −.02 .20


A MATTER OF CONTEXT: A META-ANALYTIC INVESTIGATION OF...

Documents

Transcript of A MATTER OF CONTEXT: A META-ANALYTIC INVESTIGATION OF...