CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis
description
Transcript of CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis
![Page 1: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/1.jpg)
CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis
Ed Byrnes, Ph.D.Eastern Washington University
Michael Lawson, M.S.University of California at Davis
![Page 2: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/2.jpg)
Evaluation Questions
1. What is the CPS recidivism rate for Birth & Beyond families in comparison to similar families who were involved with CPS?
2. How does the time to onset of CPS recidivism differ between Birth & Beyond families and similar families who were involved with CPS?
![Page 3: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/3.jpg)
Evaluation Design• Quasi Experimental• External Comparison Group• Fixed Follow Up Periods• Data Sources
– B & B Database– Sacramento County CPS Database
• Between Groups Comparability– Multiple Points of Comparison
• Outcomes– CPS Recidivism and Onset of Recidivism
• Client Predictors of Outcomes– Demographic Characteristics– Assessment Results
![Page 4: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/4.jpg)
Birth and Beyond Group
• 384 B & B Families
• All Had A Prior CPS Referral
• All Had At Least One B & B Home Visit
• All Had A First Home Visit On Or Before July 11, 2006 For One Year Follow Up
![Page 5: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/5.jpg)
CPS Comparison Group• 327 Families• NOT B & B Clients• All Had A First CPS Referral Between January
1, 1999 and July 11, 2006• Sample Trimming
– Children’s Ages– Most Severe CPS Allegation– Children’s Language and Race
• Stratified Random Sampling– Most Severe CPS Allegation– Children’s Gender, Language and Race
![Page 6: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/6.jpg)
Between Groups Comparability• No Significant Differences in
– Children’s Race– Children’s Primary Language – Children’s Gender
• Very Unlikely to Influence Results• Children’s Ages Differed Significantly
– CPS Comparison Group Had More Neonates– CPS Comparison Group Was Then Older– Children’s Age and Group Membership Correlated
at .25, a Weak Relationship– Significance An Artifact of Sample Size of 711
Families• If Children’s Ages Have Any Influence on
Results It Is Quite Small
![Page 7: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/7.jpg)
Between Groups Comparability
Child's Race x BB & CPS Groups
42%
28% 31%33%42%
25%
0%
25%
50%
75%
100%
White Black Latino
Race
Pc
t o
f G
rou
p
CPS Comparison BB Group
![Page 8: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/8.jpg)
Between Groups ComparabilityChild's Language x BB & CPS Groups
90%
10%
97%
3%0%
25%
50%
75%
100%
English Spanish
Language
Pc
t o
f G
rou
p
CPS Comparison BB Group
![Page 9: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/9.jpg)
Between Groups Comparability
Child's Gender x BB & CPS Groups
52% 48%45%55%
0%
25%
50%
75%
100%
Female Male
Gender
Pc
t o
f G
rou
p
CPS Comparison BB Group
![Page 10: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/10.jpg)
Between Groups Comparability
Child's Age x BB & CPS Groups
0
1
2
3
4
0 25 50 75 100
Percentile
Ch
ild's
Ag
e
CPS Comparison BB Group
![Page 11: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/11.jpg)
CPS Recidivism• Data From CPS Database
• All Referrals With A Definitive Finding– Substantiated, Inconclusive and Unfounded– Reflect Contact With The CPS System
• One Year Follow Up Period– From First Home Visit for B & B Group– From First CPS Referral for Comparison
Group– Contrast Voluntary Additional Services with
Usual CPS Processing
![Page 12: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/12.jpg)
CPS Recidivism• Of the 384 B & B families, 140 (36%) had a new
CPS referral during the one year follow up period
• Of the 327 CPS Comparison Group families 159 (49%) had a new CPS referral during the one year follow up period
• This difference was statistically significant• B & B participation versus regular CPS services
was weakly correlated with CPS Recidivism (Phi = .12)
• Participating in B & B home visiting influenced CPS recidivism in the desired direction
![Page 13: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/13.jpg)
CPS Recidivism1 Year CPS Recidivism
49%
36%
0%
25%
50%
75%
100%
CPS Comparison BB Group
Group
Pc
t w
/ Ne
w C
PS
Re
ferr
al
![Page 14: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/14.jpg)
CPS Recidivism Onset
• Data From CPS Database
• Same Criteria As CPS Recidivism
• One Year Follow Up Period
• Survival Analysis
• Time to First CPS Referral– From First Home Visit for B & B Group– From First CPS Referral for Comparison
Group
![Page 15: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/15.jpg)
CPS Recidivism Onset• Average time elapsed from the beginning of
services to their first CPS referral during the one year follow up period– For B & B families was 279 days (sd = 6.5)– For CPS Comparison Group families was 256 days
(sd = 7.45)• This difference was statistically significant• Relationship between group membership and
time to the onset of CPS recidivism was moderate-to-strong– Magnitudes of the test statistics
• Participating in B & B home visiting influenced the time to the onset of CPS recidivism in the desired direction
![Page 16: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/16.jpg)
CPS Recidivism Onset
CPS Recidivism Onset
49%
36%
0%
25%
50%
75%
100%
30 60 90 120 150 180 210 240 270 300 330 360
Days From Start
Cu
mu
lati
ve
CP
S
Re
cid
ivis
m R
ate
CPS Comparison BB Group
![Page 17: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/17.jpg)
CPS Recidivism Onset
• B & B and CPS Comparison Groups have similar patterns of cumulative CPS recidivism rates through the first 150 days
• After 150 days the cumulative CPS recidivism curves diverge
• The relationship between participating in B & B home visiting and a delayed onset of CPS recidivism is more beneficial as time progresses
![Page 18: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/18.jpg)
Predicting CPS Outcomes• Additional Evaluation Questions
– What demographic, service and assessment characteristics of cases are strongly related to having a new CPS referral within 150 days of the first home visit?
– What demographic, service and assessment characteristics of cases are strongly related to having a new CPS referral within one year of the first home visit?
• Members of the CPS comparison group never engaged in B & B services they could not be included in these analyses
![Page 19: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/19.jpg)
Predicting CPS Outcomes• Logistic Regression Primer
– Predict the occurrence of an event
• Risk and Protection– Can be quantified using Logistic Regression
• Odds Ratios– Value of 1 means no relationship between a
factor and an event– Value greater than 1 means a risk factor for
an event– Value less than 1 means a protective factor
against an event
![Page 20: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/20.jpg)
Predicting CPS Outcomes• Outcome Variables
– New CPS Referral Within 150 Days– New CPS Referral Within 1 Year– Same Definitions As Recidivism and Onset
• Predictor Variables– Demographics– Services– Assessment
![Page 21: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/21.jpg)
Predicting CPS Outcomes• Model Building Process• Two Stages
– Model by Variable Category for Each Outcome– Final Model for Each Outcome
• Criteria For Final Model Inclusion– Statistical Significance– Odds Ratio Greater Than 2
• Evaluating Models– Goodness of Fit, Improved Case Classification,
Outliers In The Solution v. Over-identification– Sample Size– Explained Variance: How Much Variability In The
Outcome Can Be Attributed To Variables In The Model – 20% or higher is acceptable
![Page 22: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/22.jpg)
Predicting CPS Outcomes• Predicting 150 Day CPS Recidivism• No Service Variables Met Final Model Criteria• Predictors At Intake
– Being a first time parent– Being pregnant– Being under the age of 25– Being a member of the Black race– Being a member of the White race– Being a member of the Hispanic race– Meeting the criteria for clinical depression– Having a high total score on the APPI (indicating
more skillful parenting)– Having a low score on the MSSI (indicating a low
level of social support)
![Page 23: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/23.jpg)
Predicting CPS Outcomes• Predicting 150 Day CPS Recidivism• 85 Cases With Complete Data• 36% Explained Variance• Met Other Model Quality Criteria• Risk Factors
– Pregnant at the Time of Intake– Being Black– Having Low Levels of Social Support
• Protective Factor– Being Under Age 25
![Page 24: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/24.jpg)
Predicting CPS OutcomesRisk & Protection for 150 Day CPS Referral
13.2
0.1
3.22.0
-5.0
0.0
5.0
10.0
15.0
Pregnant atIntake
Under 25 Black* Low SocialSupport*
Factor
Od
ds
Ra
tio
RISK
PROTECTION
* = Not Statistically Significant
![Page 25: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/25.jpg)
Predicting CPS Outcomes• Predicting 1 Year CPS Recidivism• No Service Variables Met Final Model Criteria• Predictors At Intake
– Being fluent in English– Being pregnant– Having a low total score on the APPI (indicating less
skillful parenting)– Experiencing severe domestic violence
• Predictors From Follow Up Assessment– Having a high score on the CAGE questionnaire
(indicating a higher likelihood of alcohol dependence)
– Experiencing severe domestic violence
![Page 26: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/26.jpg)
Predicting CPS Outcomes• Predicting 1 Year CPS Recidivism• 93 Cases With Complete Data• 25% Explained Variance• Met Other Model Quality Criteria• Risk Factors
– Pregnant at the Time of Intake– Having a Higher Final CAGE Questionnaire
Score– Having Low Parenting Skills at Intake
• No Protective Factors
![Page 27: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/27.jpg)
Predicting CPS OutcomesRisk & Protection for 1 Year CPS Referral
4.5
2.2
8.2
-5.0
0.0
5.0
10.0
15.0
Pregnant at Intake Low Total APPI Score* Final CAGE Points
Factor
Od
ds
Ra
tio
RISK
PROTECTION
* = Not Statistically Significant
![Page 28: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/28.jpg)
Dynamic Risk
• Assessment Results Are Not Very Stable Over Time– Pretest – Posttest Correlations Are Weak To
Modest
• Risk and Protective Factors That B & B Clients Experience Are Dynamic Rather Than Static– B & B Home Visitors Must View Assessment
As An Ongoing Process
![Page 29: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/29.jpg)
Dynamic Risk
Chronbach's Alpha and Test-Retest Reliability
Internal Consistency Test-Retest Reliability
pre post r
AAPI-2 0.76 0.81 .48 - .67
MSSI 0.71 0.70 .41 - .51
CES-D 0.82 0.84 .32 - .34
![Page 30: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/30.jpg)
CPS Recidivism Results
• B & B participation was associated with lower aggregate CPS recidivism rates
• B & B participation was associated with a delayed onset of CPS recidivism
• Given the broad ecologies of child maltreatment, these results can be viewed as promising
![Page 31: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/31.jpg)
Discussion Questions• What challenges have you encountered in
conducting quasi experimental evaluation research, and how have you addressed these?
• Since California has a county administered child welfare system what efforts can we make to encourage uniformity of data for greater generalizability of results?
![Page 32: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/32.jpg)
Discussion Questions• What are the trade offs between using
matched (case control) designs versus designs that rely on inclusion and exclusion algorithms for comparison groups?
• Since California and it's counties, like so many other states and locales, are facing deep budget shortfalls, how can we continue to promote evidence based practice as a priority?
![Page 33: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/33.jpg)
Dissemination Plan
• Incorporate feedback about our study from this symposium into a final manuscript.
• Submit the manuscript for publication.
• Peer reviewed journals being considered include:– Children and Youth Services Review;– Research on Social Work Practice.
![Page 34: CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis](https://reader035.fdocuments.in/reader035/viewer/2022081603/56815a2a550346895dc76b43/html5/thumbnails/34.jpg)
Follow Up Contact
Ed Byrnes, Ph.D.
Email: [email protected]
(509) 455-3422
Michael Lawson, M.S.
Email: [email protected]