Investigating Test Bias

BCGi: Adverse Impact & Test Validation Book Series

What is “Test Bias”?

• Employers or test publishers that have the sample size necessary to

conduct a criterion-related validity study can typically complete a

test fairness study

• Test fairness reveals whether a test operates similarly between

various groups with respect to both the test and job performance

metrics

• If a test is identified as “fair,” it still might exhibit have adverse

impact (the two concepts are not the same)

• Conducting a fairness study requires following some rather exacting

and mechanical steps, but if done correctly, it can provide

important insights regarding the inner working of the test

www.BCGinstitute.org

Why is this Topic Important to HR/EEO Professionals?

• Why do I need to know about this topic? – Valid tests can still be unfair

– Tests that are statistically unfair can be a liability

– Conducting a fairness study is easier than one might think…

– Unfair tests are more harmful than tests that have adverse impact but are “fair”

• What are the key essentials I need to know about this topic? – Conducting a fairness study requires having reliable data for

both the test and job performance criteria

– Adverse impact and fairness are different concepts

• What are the consequences surrounding these issues? – A test that is “valid” may not meet the Uniform Guidelines

“fairness” requirements

– The “start up” cost of a validation case can be $30k to $80k

www.BCGinstitute.org

Investigating Test Bias:

Presentation Overview

• Overview

• The Uniform Guidelines Definition of “Test Fairness”

• The “Cleary” Model

• Lautenschlager and Mendoza’s Reframing of Test Bias Analysis

• Carrying Out Lautenschlager and Mendoza’s Approach in SPSS

• Subgroup Scatterplots

• Final Recommendations

www.BCGinstitute.org

Resources

• Adverse Impact and Test Validation: A Practitioner's Handbook by Daniel A. Biddle, Ph.D.

– Purchase online at www.BCGinstitute.org

• Adverse Impact and Test Validation Book Series Webinars

– Recordings available online for all BCGi Platinum Members

– Webinar slides available online to all BCGi Standard & Platinum

Members

• BCGi Membership

– Free Standard Membership

– Premium Platinum Membership

www.BCGinstitute.org

About Our Sponsor: Biddle Consulting Group (BCG)

BCG is an HR firm dedicated to providing the highest products and services related to

Equal Employment Opportunity (EEO), Affirmative Action and Employee Selection.

• BCG’s Consulting Services

― Affirmative Action Plan Outsourcing

― Compensation Analysis

― EEO/AA Litigation Support (Plaintiff and Defendant)

― Job Analysis

― Test Development and Test Validation

• BCG’s Software Products

― Adverse Impact Toolkit™

― AutoAAP® affirmative action plan development software

― AutoGOJA® job analysis software

― C4 call center testing software

― COMPARE™ compensation analysis software

― CritiCall® dispatcher/call-taker personnel selection software

― ENCOUNTER soft skills video situational judgment testing software

― OPAC® office skills testing software

― TVAP™ Test Validation & Analysis Program software

www.Biddle.com