Faking in personnel selection: Does it matter and can we do anything about it? Eric D. Heggestad...
-
Upload
garrett-hibbs -
Category
Documents
-
view
216 -
download
3
Transcript of Faking in personnel selection: Does it matter and can we do anything about it? Eric D. Heggestad...
Faking in personnel Faking in personnel selection:selection:
Does it matter and can we do Does it matter and can we do anything about it?anything about it?
Eric D. HeggestadEric D. HeggestadUniversity of North Carolina - CharlotteUniversity of North Carolina - Charlotte
Education Testing Service Mini-Education Testing Service Mini-ConferenceConference
Oct 13Oct 13th th & 14& 14thth 20062006
Four Questions About Four Questions About Faking in Personnel Faking in Personnel Selection ContextsSelection Contexts
1.1. Can people fake?Can people fake?
2.2. Do applicants fake? Do applicants fake?
3.3. Does faking matter?Does faking matter?— I will talk about one projectI will talk about one project
4.4. What do we do about it?What do we do about it?— I will talk about one projectI will talk about one project
Does faking matter?Does faking matter?
Effects on Validity and Effects on Validity and SelectionSelection
Mueller-Hanson, Heggestad, & Thornton Mueller-Hanson, Heggestad, & Thornton (2003)(2003)
Ss completed personality and criterion Ss completed personality and criterion measures in lab settingmeasures in lab setting
Personality measurePersonality measure— Achievement Motivation InventoryAchievement Motivation Inventory
Criterion measureCriterion measure— A speeded ability test with no time limitA speeded ability test with no time limit— Could leave when they wanted, opportunity for Could leave when they wanted, opportunity for
normative feedbacknormative feedback
GroupsGroups— Honest Honest ((nn = 240) = 240) vs. faking vs. faking ((nn = 204) = 204)
Means & Standard Means & Standard DeviationsDeviations
PredictorPredictor
CriterionCriterion
Faking Faking GroupGroup
Honest Honest GroupGroup
Effect Effect SizeSize
214.7 225.6 0.41
40.5 40.1 -0.05
Criterion-Related Criterion-Related ValidityValidity
Faking Faking GroupGroup
Honest Honest GroupGroup
.17* .05
* p < .05
Upper thirdUpper third .20*
Lower thirdLower third .26*
.07
.45*
Full Groups
But Validity is Only But Validity is Only Skin DeepSkin Deep
Important to look at selectionImportant to look at selection Groups were combined and various Groups were combined and various
selection ratios examinedselection ratios examined
Variables examinedVariables examined Percent of selectees from each groupPercent of selectees from each group Performance of those selectedPerformance of those selected
Effects on SelectionEffects on SelectionPercent hired at various selection ratiosPercent hired at various selection ratios
0
10
20
30
40
50
60
70
90 80 70 60 50 25 20 15 10
HonestFaking
Selection Ratio (%)Selection Ratio (%)
Per
cent
of
Sel
ecte
esP
erce
nt o
f S
elec
tees
Note: Honest made up 54% Note: Honest made up 54% of sampleof sample
Effects on SelectionEffects on SelectionGroup performance at various selection Group performance at various selection
ratiosratios
36
37
38
39
40
41
42
43
44
45
46
90 80 70 60 50 25 20 15 10
HonestFaking
Selection Ratio (%)Selection Ratio (%)
Per
form
ance
Per
form
ance
.56.56.50.50.31.31.23.23.18.18.15.15.08.08.09.09.07.07
ConclusionsConclusions
Faking appears to have…Faking appears to have… An impact on the criterion-related An impact on the criterion-related
validity of our predictorvalidity of our predictor— Most noticeably at the high end of the distributionMost noticeably at the high end of the distribution
An impact on the quality of decisionsAn impact on the quality of decisions— Low performing fakers more likely to be selected in Low performing fakers more likely to be selected in
top-down contextstop-down contexts
What do we do about What do we do about faking?faking?
What Do We Do About What Do We Do About Faking?Faking?
Approach 1: Detection and CorrectionApproach 1: Detection and Correction
Tries to correct faking that has already Tries to correct faking that has already occurredoccurred
Score correctionsScore corrections— Not successful (Ellingson, Sackett & Hough, 1999; Not successful (Ellingson, Sackett & Hough, 1999;
Schmitt & Oswald, 2006)Schmitt & Oswald, 2006)
IRT workIRT work RetestingRetesting
What Do We Do About What Do We Do About Faking?Faking?
Approach 2: PreventionApproach 2: Prevention
Many prevention strategiesMany prevention strategies WarningsWarnings Subtle itemsSubtle items Multidimensional forced-choice (MFC) Multidimensional forced-choice (MFC)
response formatsresponse formats
What is an MFC What is an MFC Format?Format?
Dichotomous quartet formatDichotomous quartet format Item contains four statementsItem contains four statements Each statement represents a different traitEach statement represents a different trait 2 statements positively worded, 2 statements positively worded,
2 statements negatively worded2 statements negatively worded Indicate “Most Like Me” and “Least Indicate “Most Like Me” and “Least
Like Me” Like Me”
Example MFC ItemExample MFC Item
Avoid difficult reading material (-)
Only feel comfortable with friends (-)
Believe that others have good intentions (+)
Make lists of things to do (+)
XXXX
XXXX
Most Most Like MeLike Me
Least Least Like MeLike Me
MFC FormatsMFC Formats
Appears to be faking resistant Appears to be faking resistant (Christiansen et al., 1998; Jackson et al., 2000)(Christiansen et al., 1998; Jackson et al., 2000)
Example from Jackson et al. (2000)Example from Jackson et al. (2000) Likert-type format effect size = .95Likert-type format effect size = .95 MFC format effect size = .32MFC format effect size = .32
However….However….
Normative vs. IpsativeNormative vs. Ipsative MFC measures typically provide MFC measures typically provide partiallypartially ipsativeipsative measurement measurement
Selection settings require normative Selection settings require normative assessmentassessment
Also, evaluations have focused on group Also, evaluations have focused on group level analyseslevel analyses
Forced-Choice as Forced-Choice as Prevention? Prevention? Heggestad, Heggestad,
Morrison, Reeve & McCloy (2006)Morrison, Reeve & McCloy (2006)
Two studiesTwo studies Study 1 – Do MFC measures provide Study 1 – Do MFC measures provide
normative trait information?normative trait information?
Study 2 – Are MFC measures resistant to Study 2 – Are MFC measures resistant to faking at individual level?faking at individual level?
Study 1 Study 1 Do MFC measures provide Do MFC measures provide
normative information?normative information?
Participants Participants (n= 307)(n= 307) completed three completed three measures under honest instructionsmeasures under honest instructions
NEO-FFINEO-FFI IPIP Likert measure IPIP Likert measure IPIP MFC measureIPIP MFC measure
— Conducted three data collections to create this Conducted three data collections to create this measuremeasure
Study 1 Study 1 Do MFC measures provide Do MFC measures provide
normative information?normative information?
Logic: If MFC provides normative Logic: If MFC provides normative information, then correspondence information, then correspondence between …between …
IPIP-Likert and IPIP-MFC scales should IPIP-Likert and IPIP-MFC scales should be quite goodbe quite good
Each IPIP measure and the NEO-FFI Each IPIP measure and the NEO-FFI should be similar should be similar
Study 1 Study 1 Do MFC measures provide Do MFC measures provide
normative information?normative information?
.81.81
.87.87
.75.75
.75.75
.83.83
.68.68
.67.67
.76.76
.70.70
.81.81
.59.59
.58.58
.65.65
.64.64
.71.71
IPIP Likert IPIP Likert IPIP MFCIPIP MFC
NEO NEO IPIP LikertIPIP Likert
NEO NEO IPIP MFCIPIP MFC
StabilityStability
ExtroversionExtroversion
OpennessOpenness
AgreeablenessAgreeableness
Conscientious.Conscientious.
CorrelationsCorrelations
Study 1 Study 1 Do MFC measures provide Do MFC measures provide
normative information?normative information?
We also defined correspondence as mean We also defined correspondence as mean percentile differences across the percentile differences across the measuresmeasures
n
tileFORM
tileFORM |%%| 21
Study 1 Study 1 Do MFC measures provide Do MFC measures provide
normative information?normative information?
14.0014.00
11.3811.38
15.2215.22
16.3916.39
12.6112.61
18.2918.29 21.1321.13
18.6118.61 20.4920.49
15.2815.28 18.5818.58
17.6317.63 19.3119.31
14.0714.07 16.9616.96
IPIP Likert IPIP Likert IPIP MFCIPIP MFC
NEO NEO IPIP LikertIPIP Likert
NEO NEO IPIP MFCIPIP MFC
StabilityStability
ExtroversionExtroversion
OpennessOpenness
AgreeablenessAgreeableness
Conscientious.Conscientious.
Percentile RankPercentile Rank
Study 1 Study 1 Do MFC measures provide Do MFC measures provide
normative information?normative information?
ConclusionsConclusions MFC seems to do a reasonable job of MFC seems to do a reasonable job of
capturing normative trait informationcapturing normative trait information— People can be compared directly!People can be compared directly!
Study 2 Study 2 Are MFC measures Are MFC measures
resistant to faking at individual level?resistant to faking at individual level?
Participants Participants (n= 282)(n= 282) completed three completed three measuresmeasures
NEO-FFI NEO-FFI Honest instructions Honest instructions IPIP Likert IPIP Likert Faking instructions Faking instructions IPIP MFC IPIP MFC Faking instructions Faking instructions
Replication of Previous Replication of Previous FindingsFindings
IPIP LikertIPIP Likert IPIP MFCIPIP MFC
StabilityStability
ExtroversionExtroversion
OpennessOpenness
AgreeablenessAgreeableness
Conscientious.Conscientious.
0.750.75
0.650.65
0.360.36
0.650.65
1.231.23
0.610.61
0.330.33
0.130.13
0.070.07
1.201.20
Effect SizesEffect Sizes
Study 2 Study 2 Are MFC measures Are MFC measures
resistant to faking at individual level?resistant to faking at individual level?
Logic: If MFC is resistant to faking at Logic: If MFC is resistant to faking at the individual level, then…the individual level, then…
NEO-FFI NEO-FFI (honest)(honest) IPIP-MFC IPIP-MFC (like honest)(like honest)
andand NEO-FFI NEO-FFI (honest)(honest) IPIP-Likert IPIP-Likert (fakeable)(fakeable)
IPIP-MFC IPIP-MFC IPIP-Likert IPIP-Likert
Study 2 Study 2 Are MFC measures Are MFC measures
resistant to faking at individual level?resistant to faking at individual level?
.62.62
.61.61
.59.59
.48.48
.68.68
.37.37
.37.37
.53.53
.50.50
.40.40
.26.26
.36.36
.55.55
.52.52
.39.39
NEO NEO IPIP MFCIPIP MFC
IPIP Likert IPIP Likert IPIP MFCIPIP MFC
NEO NEO IPIP LikertIPIP Likert
StabilityStability
ExtroversionExtroversion
OpennessOpenness
AgreeablenessAgreeableness
Conscientious.Conscientious.
CorrelationsCorrelations
Study 2 Study 2 Are MFC measures Are MFC measures
resistant to faking at individual level?resistant to faking at individual level?
20.2320.23
21.0921.09
20.4420.44
24.3324.33
18.0518.05
25.2925.29
24.2324.23
21.8521.85
21.5421.54
23.4723.47
28.8728.87
26.1226.12
20.6920.69
22.8222.82
23.7523.75
IPIP Likert IPIP Likert IPIP MFCIPIP MFC
NEO NEO IPIP LikertIPIP Likert
NEO NEO IPIP MFCIPIP MFC
StabilityStability
ExtroversionExtroversion
OpennessOpenness
AgreeablenessAgreeableness
Conscientious.Conscientious.
Percentile RankPercentile Rank
Study 2 Study 2 Are MFC measures Are MFC measures
resistant to faking at individual level?resistant to faking at individual level?
Conclusion Conclusion MFC not a solution to fakingMFC not a solution to faking
— Can fake specific scalesCan fake specific scales— Not faking resistant at individual levelNot faking resistant at individual level
Summary and Summary and ConclusionConclusion
Faking does impact scoresFaking does impact scores Changes the nature of the scoreChanges the nature of the score Not likely to have a big effect on CRVNot likely to have a big effect on CRV Could have notable implications for Could have notable implications for
selectionselection
Dichotomous quartet response format Dichotomous quartet response format does not offer a viable remedydoes not offer a viable remedy