Disparate impact reduction 463

Lost in Translation: Disparate ImpactReduction Strategies and LegalStakeholders

ERICA N. DREW AND CHOCKALINGAM VISWESVARANFlorida International University

In their coverage of employment discrim-ination issues within personnel selection,Lindsey, King, Dunleavy, McCausland, andJones (2013) aptly discuss how researchin the area of disparate impact has sug-gested and evaluated the merit of varioustechniques to reduce the occurrence ofdifferential hiring rates across groups(e.g., Ployhart & Holtz, 2008; Ryan &Tippins, 2004). Later, Lindsay et al. pro-vide various recommendations for howindustrial–organizational (I–O) psychol-ogists can advance research and practicein this area. In our opinion, to reach the‘‘ultimate goal of eradicating employmentdiscrimination’’ (Lindsey et al., p. 391),at least in terms of disparate impact,stakeholders beyond the I–O field mustbe educated about the value of thesestrategies. In this commentary we summa-rize the results of a national survey thatdemonstrates attorneys may not be awareof the usefulness of different strategiesdeveloped by I–O psychologists to reducethe occurrence of disparate impact. Weagree with Lindsay et al. that we needto inform the EEOC of our findings—butargue that we need to go beyond the EEOCand make concerted efforts to convey ourfindings to the legal community in general.

Individuals within the legal commu-nity are key decision makers in instances

Correspondence concerning this article should beaddressed to Erica N. Drew.E-mail: edrew001@fiu.edu

Address: Department of Psychology, FloridaInternational University, 11200 SW 8th Street, Miami,FL 33199

where disparate impact has occurred or issuspected, rendering attorneys and judgesexpressly important stakeholders to con-sider. In a recent article, Gutman (2012)stated, ‘‘any large-scale method of assess-ment that results in [disparate] impact willlikely be challenged, as is any attempt tomitigate [disparate] impact with alternativemethods or content manipulation’’ (p. 700).The question then becomes whether otherstakeholders in the legal process understandthe purpose of these strategies, and further,does the use of these strategies mitigateadverse legal outcomes for organizationsunder investigation? Some research sug-gests no. For example, Wingate and Thorton(2004) examined federal judges perceptionsof I–O expert witness testimony and foundthat judges were relatively unfamiliar withthe field of I–O, which impacted their abil-ity to appreciate testimony demonstratingthe appropriateness of selection strategiesused. Further, no known empirical researchexists to explain how organizational imple-mentation of disparate impact reductionstrategies influences an attorney’s decisionto pursue various legal outcomes.

In the survey, we sought to estimate theperceived legal defensibility of the disparatereduction strategies discussed in the Lindseyet al. focal article. Specifically, we wereinterested in how various disparate impactreduction strategies affected: (a) monetarycompensation requests (MCR) in settlementnegotiations and (b) desire to pursue litiga-tion (litigiousness). In the survey, attorneysindicated how these organizational efforts

464 E.N. Drew and C. Viswesvaran

in selection system design would affectMCR and litigiousness on a five-point Likertscale where 1 = significantly decrease,3 = neither increase nor decrease, and5 = significantly increase.

Practical experience in labor and em-ployment law and/or civil rights law was arequirement for participation. Participationemail requests were sent to AmericanBar Association (ABA, 2012) registeredemployment and labor law blog contribu-tors (n = 150), employment and labor lawfirms (n = 722), and professional contacts(n = 26). Responses were obtained from102 practicing attorneys, resulting in an11% response rate. Many participantsopted out of responding to demographicquestions, but based on those who reporteddemographic information, the majorityof respondents were male (64%) andCaucasian (82.8%). Ages ranged from24 to 65 (M = 42.76), similar to attorneypopulation characteristics (ABA, 2012).The majority of attorneys reported practicalexperience in areas of employment/laborlaw (82%), and civil rights law (33%). Asignificant portion of attorneys indicatedinvolvement in a Title VII dispute (71%), themajority of which represented employers(65%). Many attorneys (53%) indicatedthey had ‘‘litigated an employment disputeregarding employment selection.’’1 Again,the majority represented employers (64%).

To develop the survey items, wesearched the I–O literature on differentdisparate impact reduction strategies; forexample, we considered strategies summa-rized and suggested by Ryan and Tippins(2004, e.g., reduction of verbal abilityrequirements, using a multiple hurdleapproach where screening tools with less

1. Litigated refers to whether or not the attorneyparticipated in trying a case before a judge withinthe adversarial arena. Many attorneys wish to avoidcosts associated with litigation and often move tosettle the case out of court with the two opposingparties. Absence of participation in litigation doesnot imply that the attorneys who participated in thisstudy did not deal with employment discriminationissues but rather they had yet to resolve an issuewithin the courts.

disparate impact potential were used earlyin the selection process and those withgreater potential for disparate impact later inthe process). Table 1 provides the completeitem list. A panel of two personnel selectionexperts and three employment discrimi-nation legal experts reviewed the items tocheck for wording, accuracy, redundancy,and comprehensiveness. In addition, aclass of third-year law students taking anemployment discrimination course pilotedthe survey to ensure adequate coverage ofissues and appropriateness of item wording.

As can be seen in Table 1, many of thesestrategies correspond to those discussed inLindsey et al. across their three categories:(a) what constructs to measure (alterna-tive selection methods, broaden KSAO’sassessed), (b) how to measure constructs(alternative modes of presentation, removalof subgroup biased items), and (c) howto use selection tools to make decisions(banding techniques, use of cut scores,differential weighting of predictors). Inaddition, Lindsay et al. point out that tohire minorities, they must apply for thejob. Thus, we also considered a methodof targeted recruiting (i.e., increasing andretaining the number of minorities in theapplicant pool) as a strategy.

Overall, many of the recommendeddisparate impact reduction strategies wererated as only moderately influential on legaloutcomes of interest; some had no impact atall. One finding in particular clearly demon-strates the need for I–O researchers andpractitioners to increase visibility and valuerecognition of disparate impact reductionstrategies within the legal community.Though Lindsey et al. recommend that weshould ‘‘stop attempting to use minoritypreferences’’ within hiring procedures (andwe tend to agree), banding with minoritypreference ranked higher than a standardbanding strategy, as well as alternativemodes of presentation. Though banding iscontroversial (Pyburn, Ployhart, & Kravitz,2008), the literature generally supports theability of video or multimedia formats todecrease the potential for disparate impactas compared to paper-and-pencil tests (e.g.,

Disparate impact reduction 465

Table 1. Rank Order, Means, and Standard Deviations of Disparate Impact ReductionStrategies

M SD Rank M SD Rank

Organizational evidence item

Monetarycompensationrequests Litigiousness

(a) Made efforts to increase and retain thenumber of qualified minority and femaleapplicants in the hiring pool.

2.07 0.88 1 1.96 0.83 2

(b) Cutoff scores used were reasonable andconsistent with normal expectations ofacceptable job performance.

2.14 0.87 2 1.95 0.85 1

(c) Items were written to be free of content thatwould be more familiar or less familiar to aparticular subgroup.

2.16 0.87 3 2.06 0.85 3

(d) Used specific measures of ability ratherthan general measures of ability.

2.35 0.71 4 2.32 0.69 4

(e) Removed items that demonstrated differentscores for subgroups.

2.47 1.21 5 2.38 1.28 5

(f) Removed test components that had highpotential for disparate impact but were alsothe most valid predictors of jobperformance.

2.54 0.92 6 2.57 0.96 7

(g) Verbal ability requirements were reducedto an extent supported by the job analysis.

2.62 0.77 7 2.58 0.78 6

(h) Used alternative measurement methods. 2.67 0.84 8 2.74 0.87 8(i) Used test-score banding with minority

preference within bandwidth.2.77 1.01 9 2.82 1.11 9

(j) Used alternative modes of test itempresentation.

2.84 0.62 10 2.92 0.63 10

(k) Used screening devices with less disparateimpact early in the selection process andthose with greater potential for disparateimpact later in the process.

2.88 0.84 11 2.95 0.87 11

(l) Used test-score banding instead of selectingemployees top down.

2.94 0.73 12 2.99 0.75 12

(m) Gave more weight to contextualperformance predictors than to taskperformance.

3.16 0.77 13 3.18 0.83 13

Note. N = 89–108. Ratings provided on a five-point Likert scale. Lower scores demonstrate lower monetarycompensation, and litigiousness.

Chan & Schmitt, 1997; Pulakos & Schmitt,1996). Thus, it is surprising that bandingwith minority preference fared better, andit reflects a gap between our research baseand attorney evaluations.

In summary, though the strategies wedevelop may be effective at decreasinglikelihood of differential hiring rates, the

legal defensibility of these strategies hingesupon how the legal community, includingregulatory agencies, perceives them. Ifthe purpose and significance of thesemethods are not communicated, our effortsmay be wasted. We applaud Lindsayet al. for a comprehensive review of the50 years of discrimination research by I–O

466 B.E. Liberman

psychologists but caution that we need tomake more efforts to communicate thesefindings to key stakeholders.

References

American Bar Association (2012). Lawyer demograph-ics. Retrieved from http://www.americanbar.org/

Chan, D., & Schmitt, N. (1997). Video-based versuspaper-and-pencil method of assessment in situa-tional judgment tests: Subgroup differences in testperformance and face validity perceptions. Journalof Applied Psychology, 82, 143–159.

Gutman, A. (2012). Legal constraints on personnelselection decisions. In N. Schmitt (Ed.), Oxfordhandbook of personnel assessment and selection(pp. 686–720). New York, NY: Oxford UniversityPress, Inc.

Lindsey, A., King, E., Dunleavy, E., McCausland, T., &Jones, K. (2013). What we know and don’t: Erad-icating employment discrimination 50 years afterthe Civil Rights Act. Industrial and Organizational

Psychology: Perspectives on Science and Practice,6(4), 391–413.

Ployhart, R. E., & Holtz, B. C. (2008). Thediversity–validity dilemma: Strategies for reduc-ing racioethnic and sex subgroup differencesand adverse impact in selection. Personnel Psy-chology, 61, 153–172. doi: 10.1111/j.1744-6570.2008.00109.x

Pulakos, E. D., & Schmitt, N. (1996). An evaluationof two strategies for reducing adverse impact andtheir effects on criterion-related validity. HumanPerformance, 9, 241–258.

Pyburn, K. M., Jr., Ployhart, R. E., & Kravitz, D. A.(2008). The diversity–validity dilemma: Overviewand legal context. Personnel Psychology, 61,143–151.

Ryan, A. M., & Tippins, N. T. (2004). Attracting andselecting: What psychological research tells us.Human Resource Management, 43, 305–318. doi:10.1002/hrm.20026

Wingate, P. H., & Thorton, G. C., III (2004). Indus-trial/organizational psychology and the federaljudiciary: Expert witness testimony and the Daubertstandards. Law and Human Behavior, 28, 97–114.