Criminal Justice Basic Abilities Test (CJBAT) Update -Law Enforcement Officer

-Corrections Officer

Florida Criminal Justice Standards and Training

Commission

Industrial/Organizational Solutions, Inc.

August 6, 2008

Chad Legel & Mark Tawney

Agenda About I/O Solutions Update on CJBAT Testing in 07-08

Test objective (changes from pre-Oct. 07)

Test performance update Pass rates, scores, reliability

Test validation studies Likelihood of passing SOCE based on

CJBAT score New test forms

Wrap-up

About I/O Solutions

National public safety test publisher and consulting firm based in Chicago

Staff of ten consultants trained in Industrial/Organizational Psychology

Provider of Florida CJBAT Program to approximately 30 FL law enforcement and correctional academies since 2001

Maintain a research and development function devoted to validating the CJBAT (on-going) and continually improving the computer-based test administration system

CJBAT Test Objectives

Assess core basic abilities as defined by FDLE

Provide an easily administered testing program

More rigorously screen abilities to ensure requisite ability levels

Establish compelling test validity evidence

CJBAT assesses the 8 mandated abilities using 120 questions covering 8 ability areas

CJBAT is available in paper and computer-based formats (with strong focus on customer service)

Careful research was done to establish an appropriate cut-off score that screens out unqualified candidates

Research demonstrates a strong relationship between CJBAT and academy/SOCE success

CJBAT Performance

CJBAT Test/Form

Sample Size Mean

Pass Rate

Reliability (Alpha)

LEO 1 4515 79.35 62 0.93

LEO 2 2220 77.15 51 0.92

LEO 3 960 77.05 52 0.92

CO 1 3499 76.06 66 0.95

CO 2 1404 73.08 58 0.93

CO 3 1061 74.89 52 0.93

LEO Ability Area Average Score Std. Dev.

Written Comprehension 87.7 12.39

Information Ordering 86.84 14.08

Memorization 86.64 8.88

Spatial Orientation 81.87 21.47

Inductive Reasoning 78.72 20.51

Deductive Reasoning 71.7 25.13

Problem Solving 71.7 24.39

Written Expression 69.41 15.43

Major difference in sample composition: LEO = 19% female, CO = 50% female

CO Ability Area Average Score Std. Dev.

Written Comprehension 85.23 13.02

Information Ordering 84.29 18.54

Memorization 82.61 11.2

Inductive Reasoning 74.35 23.5

Written Expression 73.86 16.6

Spatial Orientation 70.91 25.69

Deductive Reasoning 70.52 27.28

Problem Solving 66.69 27.79

CJBAT Validation 2 criterion-related validation studies demonstrating the

relationship between the CJBAT and important criteria CJBAT correlated with academy performance (final

academy score) Data collected from Broward Sheriff, FHP, George Stone,

Gulf Coast, Lake Tech, Manatee Tech and Pinellas Sheriff CJBAT correlated with state certification scores (SOCE)

Data collected from all academies using CJBAT

General Guidelines for Interpreting Validity Coefficients (US Dept. of Labor)

Validity Value Interpretation

above .35 very beneficial

.21 - .35 likely to be useful

.11 - .20 depends on circumstances

below .11 unlikely to be useful

CJBAT Test/Form Sample Size Academy Correlation

LEO 59 0.88

CO 107 0.83

Meta-Analytic 166 0.85

CJBAT Test/Form Sample Size SOCE Correlation*

LEO 1 413 0.48

LEO 2 143 0.5

LEO 3 75 0.45

CO 1 272 0.54

CO 2 109 0.75

CO 3 74 0.71

Meta-Analytic 1086 0.54*correlations corrected for range restriction

SOCE Success Based on CJBAT Score

CJBAT Test Score Range SOCE Pass SOCE Fail Passing Rate

90+ 120 2 98.36

85-89 129 12 91.49

79-84 71 16 81.61

78- * 115 42 73.25

90+ 121 3 97.58

85-89 77 9 89.53

80-84 44 11 80

72-79 29 15 65.91

71- * 20 21 48.78

*went on to pass the CJBAT in subsequent attempt(s)

LEO (attempt 1)

CO (attempt 1)

New CJBAT Forms

CJBAT LEO & CO second practice exam released

LEO 4 and CO 4 available October 2008 Field-tested on incumbent LEO and CO

academy participants LEO N= 111

Broward County CC, Santa Fe CC, Withlachoochee

CO N= 36 Chipola, Manatee County Sheriff,

Washington Holmes

CJBAT Wrap-Up CJBAT is screening candidates

effectively (identifying a sizable subset of the sample that does not possess necessary abilities)

Academy and SOCE validation studies demonstrate that test is HIGHLY predictive of success in the academy and on SOCE

We are quite confident in the CJBAT’s defensibility based on its high validity levels

I/O Solutions is interested in continuing to learn from our clients how the CJBAT is functioning locally

Questions?