Using Selection Procedures: Cutoffs, Banding, and Ranking (Overview)
6
Click here to load reader
-
Upload
biddle-consulting-group -
Category
Business
-
view
703 -
download
0
description
A perfectly valid selection procedure can be invalidated through improper use. Validation has to do with the interpretation of scores. A valid selection procedure produces scores that can be informative in both absolute and relative terms. A person who scores 90% on a written test absolutely answered about 9 out of each set of 10 questions correctly. In an absolute sense, they answered just about every test item correctly. But what if they scored in the lowest 10% of all test takers (i.e., about 90% of the applicants scored higher)? This paints a completely different picture. Relative to the other applicants, they scored very low. Interpretation at this point can be difficult: was it the test, was it the relative abilities of the test takers, or are there other factors at play? Scores on a selection procedure should be used in such a fashion that the validation evidence supports the way the selection procedure interpreted them. If classifying applicants into two groups – qualified and unqualified – is the end goal, the test should be used on a pass/fail basis (i.e., an absolute classification based on achieving a certain level on the selection procedure). If the objective is to make relative distinctions between substantially equally qualified applicants, then banding is the approach that should be used. Ranking should be used if the goal is to make decisions on an applicant-by-applicant basis (making sure that the requirements for ranking are addressed). If an overall picture of each applicant’s combined mix of KSAPCs is desired, then a weighted and combined selection process should be used. For each of these procedures, different types of validation evidence should be gathered to justify the corresponding manner in which the scores will be interpreted. Learn more about the BCG Institute for Workforce Development by going to www. BCGInstitute.org Visit http://bcginstitute.org/?AIBookSeries to learn about the Adverse Impact and Test Validation webinar series based on Dr. Biddle’s book.
Transcript of Using Selection Procedures: Cutoffs, Banding, and Ranking (Overview)
Using Selection Procedures: Cutoff
Scores, Banding, Ranking, and
Weighting
BCGi: Adverse Impact & Test Validation Book Series
Cutoff Scores, Banding, Ranking, and Weighting
• All tests are not created equal…
• The same is true with job requirements…
• Therefore, tests should be used according to how the
skills and abilities are measured by the test and required
on the job
• Deciding whether to use cutoffs, banding, ranking, or
weighting procedures is important in both litigation and
non-litigation settings
• The federal Uniform Guidelines and relevant cases
provide clear guidance on these issues…
www.BCGinstitute.org
Why is this Topic Important to HR/EEO Professionals?
• Why do I need to know about this topic?
– The specific use and interpretation of test scores is validated, not “a test”
– Arbitrary cutoffs can be liabilities—a 70% cutoff may or may not be valid or defensible
– If not done properly, banding or ranking can lead to categorizing candidates into incorrect qualification groups
– If not done properly, weighting and combining tests results can give misleading results
• What are the key essentials I need to know about this topic? – Developing a custom cutoff requires a 3-4 hour workshop with SMEs
– When AI is present, federal EEO laws will trump state or CSB rules
– Banding is a valid way of grouping candidates into usable score categories
– Rank ordering on tests requires legal and professional justification
• What are the consequences surrounding these issues? – Using test scores in different ways will change AI outcomes
– A “perfectly valid” test can easily lose in litigation if it’s not used correctly
– The “start up” cost of a validation case can be $30k to $80k
www.BCGinstitute.org
Presentation Overview
• Overview & Background
• Developing Valid Cutoff Scores
• Banding
• Ranking
• Weighting Selection Procedures Into Combined
Scores
• Final Recommendations
www.BCGinstitute.org
Resources
• Adverse Impact and Test Validation: A Practitioner's Handbook by Daniel A. Biddle, Ph.D.
– Purchase online at www.BCGinstitute.org
• Adverse Impact and Test Validation Book Series Webinars
– Recordings available online for all BCGi Platinum Members
– Webinar slides available online to all BCGi Standard & Platinum
Members
• BCGi Membership
– Free Standard Membership
– Premium Platinum Membership
www.BCGinstitute.org
About Our Sponsor: Biddle Consulting Group (BCG)
BCG is an HR firm dedicated to providing the highest products and services related to
Equal Employment Opportunity (EEO), Affirmative Action and Employee Selection.
• BCG’s Consulting Services
― Affirmative Action Plan Outsourcing
― Compensation Analysis
― EEO/AA Litigation Support (Plaintiff and Defendant)
― Job Analysis
― Test Development and Test Validation
• BCG’s Software Products
― Adverse Impact Toolkit™
― AutoAAP® affirmative action plan development software
― AutoGOJA® job analysis software
― C4 call center testing software
― COMPARE™ compensation analysis software
― CritiCall® dispatcher/call-taker personnel selection software
― ENCOUNTER soft skills video situational judgment testing software
― OPAC® office skills testing software
― TVAP™ Test Validation & Analysis Program software
www.Biddle.com
