Test cells for investigating emission and susceptibility ...
Investigating Test Bias (Overview)
6
Click here to load reader
-
Upload
biddle-consulting-group -
Category
Business
-
view
681 -
download
0
description
The Uniform Guidelines categorize a test as “unfair” whenever “…members of one race, sex, or ethnic group characteristically obtain lower scores on a selection procedure than members of another group, and the differences in scores are not reflected in differences in a measure of job performance” (Section 8A). This section continues to explain that tests that meet this definition of unfairness “may unfairly deny opportunities to members of the group that obtains the lower scores.” Since the framing of the Uniform Guidelines, most practitioners have used an earlier conceptualization of test fairness called the “Cleary Rule.” This rule was developed by Cleary in 1968 where she stated that test bias can be evaluated by testing two hypotheses with respect to the linear relation between test scores and a criterion measure: (1) Equality of slopes, and then (2) equality of intercepts (given that slopes are equal). Cleary used the mathematics of the Analysis of Covariance (ANCOVA) to test these hypotheses. Learn more about the BCG Institute for Workforce Development by going to www. BCGInstitute.org Visit http://bcginstitute.org/?AIBookSeries to learn about the Adverse Impact and Test Validation webinar series based on Dr. Biddle’s book.
Transcript of Investigating Test Bias (Overview)
Investigating Test Bias
BCGi: Adverse Impact & Test Validation Book Series
What is “Test Bias”?
• Employers or test publishers that have the sample size necessary to
conduct a criterion-related validity study can typically complete a
test fairness study
• Test fairness reveals whether a test operates similarly between
various groups with respect to both the test and job performance
metrics
• If a test is identified as “fair,” it still might exhibit have adverse
impact (the two concepts are not the same)
• Conducting a fairness study requires following some rather exacting
and mechanical steps, but if done correctly, it can provide
important insights regarding the inner working of the test
www.BCGinstitute.org
Why is this Topic Important to HR/EEO Professionals?
• Why do I need to know about this topic? – Valid tests can still be unfair
– Tests that are statistically unfair can be a liability
– Conducting a fairness study is easier than one might think…
– Unfair tests are more harmful than tests that have adverse impact but are “fair”
• What are the key essentials I need to know about this topic? – Conducting a fairness study requires having reliable data for
both the test and job performance criteria
– Adverse impact and fairness are different concepts
• What are the consequences surrounding these issues? – A test that is “valid” may not meet the Uniform Guidelines
“fairness” requirements
– The “start up” cost of a validation case can be $30k to $80k
www.BCGinstitute.org
Investigating Test Bias:
Presentation Overview
• Overview
• The Uniform Guidelines Definition of “Test Fairness”
• The “Cleary” Model
• Lautenschlager and Mendoza’s Reframing of Test Bias Analysis
• Carrying Out Lautenschlager and Mendoza’s Approach in SPSS
• Subgroup Scatterplots
• Final Recommendations
www.BCGinstitute.org
Resources
• Adverse Impact and Test Validation: A Practitioner's Handbook by Daniel A. Biddle, Ph.D.
– Purchase online at www.BCGinstitute.org
• Adverse Impact and Test Validation Book Series Webinars
– Recordings available online for all BCGi Platinum Members
– Webinar slides available online to all BCGi Standard & Platinum
Members
• BCGi Membership
– Free Standard Membership
– Premium Platinum Membership
www.BCGinstitute.org
About Our Sponsor: Biddle Consulting Group (BCG)
BCG is an HR firm dedicated to providing the highest products and services related to
Equal Employment Opportunity (EEO), Affirmative Action and Employee Selection.
• BCG’s Consulting Services
― Affirmative Action Plan Outsourcing
― Compensation Analysis
― EEO/AA Litigation Support (Plaintiff and Defendant)
― Job Analysis
― Test Development and Test Validation
• BCG’s Software Products
― Adverse Impact Toolkit™
― AutoAAP® affirmative action plan development software
― AutoGOJA® job analysis software
― C4 call center testing software
― COMPARE™ compensation analysis software
― CritiCall® dispatcher/call-taker personnel selection software
― ENCOUNTER soft skills video situational judgment testing software
― OPAC® office skills testing software
― TVAP™ Test Validation & Analysis Program software
www.Biddle.com
