Evaluation of the Fighting Back Initiative by Kay E. Sherwood Presented by Maddie Velez Presented by...
-
Upload
clifton-baldwin -
Category
Documents
-
view
212 -
download
0
Transcript of Evaluation of the Fighting Back Initiative by Kay E. Sherwood Presented by Maddie Velez Presented by...
![Page 1: Evaluation of the Fighting Back Initiative by Kay E. Sherwood Presented by Maddie Velez Presented by Maddie Velez.](https://reader036.fdocuments.in/reader036/viewer/2022072006/56649d135503460f949e7f1e/html5/thumbnails/1.jpg)
Evaluation of the Fighting Back Initiativeby Kay E. Sherwood
Evaluation of the Fighting Back Initiativeby Kay E. Sherwood
Presented by Maddie Velez
Presented by Maddie Velez
![Page 2: Evaluation of the Fighting Back Initiative by Kay E. Sherwood Presented by Maddie Velez Presented by Maddie Velez.](https://reader036.fdocuments.in/reader036/viewer/2022072006/56649d135503460f949e7f1e/html5/thumbnails/2.jpg)
What is Fighting Back?What is Fighting Back?
A community-based drug abuse prevention
program
A community-based drug abuse prevention
program
![Page 3: Evaluation of the Fighting Back Initiative by Kay E. Sherwood Presented by Maddie Velez Presented by Maddie Velez.](https://reader036.fdocuments.in/reader036/viewer/2022072006/56649d135503460f949e7f1e/html5/thumbnails/3.jpg)
Why study this Initiative?
Why study this Initiative?
Shows the importance of taking context into evaluations
Raises questions about how community interventions are conceptualized and evaluated
Provides a warning about the manageability of large-scale, comprehensive evaluations.
Shows the importance of taking context into evaluations
Raises questions about how community interventions are conceptualized and evaluated
Provides a warning about the manageability of large-scale, comprehensive evaluations.
![Page 4: Evaluation of the Fighting Back Initiative by Kay E. Sherwood Presented by Maddie Velez Presented by Maddie Velez.](https://reader036.fdocuments.in/reader036/viewer/2022072006/56649d135503460f949e7f1e/html5/thumbnails/4.jpg)
Background: Duration and Scale Contribute to Complexity
Background: Duration and Scale Contribute to Complexity◊ A 12 year initiative◊ Original stakeholders differed greatly from the stakeholders involved 12 years later
◊ Reduction of intervention sites went from 15 to 5
◊ First evaluation team replaced after 2 years.
◊ High staff / leadership turnover◊ Original key leader retires◊ Few examples of credible, successful evaluations that truly measured the interventions impact
◊ A 12 year initiative◊ Original stakeholders differed greatly from the stakeholders involved 12 years later
◊ Reduction of intervention sites went from 15 to 5
◊ First evaluation team replaced after 2 years.
◊ High staff / leadership turnover◊ Original key leader retires◊ Few examples of credible, successful evaluations that truly measured the interventions impact
![Page 5: Evaluation of the Fighting Back Initiative by Kay E. Sherwood Presented by Maddie Velez Presented by Maddie Velez.](https://reader036.fdocuments.in/reader036/viewer/2022072006/56649d135503460f949e7f1e/html5/thumbnails/5.jpg)
The Foundation Takes on Substance abuse
The Foundation Takes on Substance abuse
◊ Robert Wood Johnson heads up the Foundation’s first efforts in the area of substance abuse.
◊ First grant was made to Vanderbilt University for $26.4 million in 1988
◊ Foundation explores addressing the national problems of substance abuse and dependence
◊ Robert Wood Johnson heads up the Foundation’s first efforts in the area of substance abuse.
◊ First grant was made to Vanderbilt University for $26.4 million in 1988
◊ Foundation explores addressing the national problems of substance abuse and dependence
![Page 6: Evaluation of the Fighting Back Initiative by Kay E. Sherwood Presented by Maddie Velez Presented by Maddie Velez.](https://reader036.fdocuments.in/reader036/viewer/2022072006/56649d135503460f949e7f1e/html5/thumbnails/6.jpg)
Continued…Continued…
◊ July 1988…the goal became “by pulling together into a single unified effort, communities can begin to solve the pressing problem of drug and alcohol abuse.”
◊ The expectation…”to reduce the demand for illegal drugs and alcohol in the funded communities.”
◊ Project STAR and ALERT◊ Poly abuse - combination of mental health problems and substance abuse occurring
◊ July 1988…the goal became “by pulling together into a single unified effort, communities can begin to solve the pressing problem of drug and alcohol abuse.”
◊ The expectation…”to reduce the demand for illegal drugs and alcohol in the funded communities.”
◊ Project STAR and ALERT◊ Poly abuse - combination of mental health problems and substance abuse occurring
![Page 7: Evaluation of the Fighting Back Initiative by Kay E. Sherwood Presented by Maddie Velez Presented by Maddie Velez.](https://reader036.fdocuments.in/reader036/viewer/2022072006/56649d135503460f949e7f1e/html5/thumbnails/7.jpg)
New Leadership: Kathryn EdmundsonNew Leadership: Kathryn Edmundson◊ New evaluation agenda: Could you organize to create political will for change at the local level and get it to add up to a national-level movement?
◊ An element of racism and elitism in the law enforcement
◊ Expected outcomes
◊ New evaluation agenda: Could you organize to create political will for change at the local level and get it to add up to a national-level movement?
◊ An element of racism and elitism in the law enforcement
◊ Expected outcomes
![Page 8: Evaluation of the Fighting Back Initiative by Kay E. Sherwood Presented by Maddie Velez Presented by Maddie Velez.](https://reader036.fdocuments.in/reader036/viewer/2022072006/56649d135503460f949e7f1e/html5/thumbnails/8.jpg)
Evaluation I: Lost time, Money, and
Credibility
Evaluation I: Lost time, Money, and
Credibility◊ 1990-1994◊ The first evaluation team replaced, 4 years, $4.6 million, and a baseline
◊ Division between stakeholders missed changes
◊ Augment between the 2nd evaluation team and foundation staff regarding lack of baseline data.
◊ 1990-1994◊ The first evaluation team replaced, 4 years, $4.6 million, and a baseline
◊ Division between stakeholders missed changes
◊ Augment between the 2nd evaluation team and foundation staff regarding lack of baseline data.
![Page 9: Evaluation of the Fighting Back Initiative by Kay E. Sherwood Presented by Maddie Velez Presented by Maddie Velez.](https://reader036.fdocuments.in/reader036/viewer/2022072006/56649d135503460f949e7f1e/html5/thumbnails/9.jpg)
A 1996 WatershedA 1996 Watershed
◊ Become unified with an emphasis on prevention, early intervention, treatment, and aftercare.
◊ NPO (National Program Office) moved to Boston University School of Public Health.
◊ NPO joined another foundation funded program called “Joined Together”, with new director David Rosenbloom.
◊ Board of Trustees makes a recommendation to give the program Fighting Back more time.
◊ Preliminary analysis indicates data that during mid-implementation Fighting Back had no effect.
◊ Become unified with an emphasis on prevention, early intervention, treatment, and aftercare.
◊ NPO (National Program Office) moved to Boston University School of Public Health.
◊ NPO joined another foundation funded program called “Joined Together”, with new director David Rosenbloom.
◊ Board of Trustees makes a recommendation to give the program Fighting Back more time.
◊ Preliminary analysis indicates data that during mid-implementation Fighting Back had no effect.
![Page 10: Evaluation of the Fighting Back Initiative by Kay E. Sherwood Presented by Maddie Velez Presented by Maddie Velez.](https://reader036.fdocuments.in/reader036/viewer/2022072006/56649d135503460f949e7f1e/html5/thumbnails/10.jpg)
A National Program Office Change
A National Program Office Change
◊ Fighting Back reduces # of sites eligible for new funding.
◊ Measure most substance abuse within the communities to be able to do something measurable at community level.
◊ Increasing treatment and treatment capacity an important goal.
◊ Fighting Back reduces # of sites eligible for new funding.
◊ Measure most substance abuse within the communities to be able to do something measurable at community level.
◊ Increasing treatment and treatment capacity an important goal.
![Page 11: Evaluation of the Fighting Back Initiative by Kay E. Sherwood Presented by Maddie Velez Presented by Maddie Velez.](https://reader036.fdocuments.in/reader036/viewer/2022072006/56649d135503460f949e7f1e/html5/thumbnails/11.jpg)
1994-2000 Evaluation II
1994-2000 Evaluation II
◊ Consensus 2nd evaluation team does an credible job with difficult circumstances.
◊ 1st Evaluators spend $4.6 million dollars with little to show for it.
◊ Fighting Back Program and evaluation staff is moving forward w/out replacement dollars.
◊ Consensus 2nd evaluation team does an credible job with difficult circumstances.
◊ 1st Evaluators spend $4.6 million dollars with little to show for it.
◊ Fighting Back Program and evaluation staff is moving forward w/out replacement dollars.
![Page 12: Evaluation of the Fighting Back Initiative by Kay E. Sherwood Presented by Maddie Velez Presented by Maddie Velez.](https://reader036.fdocuments.in/reader036/viewer/2022072006/56649d135503460f949e7f1e/html5/thumbnails/12.jpg)
Relying on Survey DataRelying on Survey Data
◊ Phone surveys throughout the community.◊ Management Information Systems (MIS).◊ Ethnographic Studies.◊ Community Indicators◊ Four Research questions were identified by the 2nd evaluation team.
◊ Strong correlations between strategies and outcomes.
◊ Community Indicators◊ School survey data difficult to use.
◊ Phone surveys throughout the community.◊ Management Information Systems (MIS).◊ Ethnographic Studies.◊ Community Indicators◊ Four Research questions were identified by the 2nd evaluation team.
◊ Strong correlations between strategies and outcomes.
◊ Community Indicators◊ School survey data difficult to use.
![Page 13: Evaluation of the Fighting Back Initiative by Kay E. Sherwood Presented by Maddie Velez Presented by Maddie Velez.](https://reader036.fdocuments.in/reader036/viewer/2022072006/56649d135503460f949e7f1e/html5/thumbnails/13.jpg)
The Price of Relying on Survey Data
The Price of Relying on Survey Data
◊ 1996 residue of distrust◊ Saxe’s research team became known as the “national evaluation
◊ Community has been seen as the “human subject”
◊ National evaluation offer no alternative to outcomes perspective
◊ High emotions surrounding analysis emerged accusations
◊ Bickman claims bias evaluations; Eval. Team are required to point out potential problems in the interventions
◊ 1996 residue of distrust◊ Saxe’s research team became known as the “national evaluation
◊ Community has been seen as the “human subject”
◊ National evaluation offer no alternative to outcomes perspective
◊ High emotions surrounding analysis emerged accusations
◊ Bickman claims bias evaluations; Eval. Team are required to point out potential problems in the interventions
![Page 14: Evaluation of the Fighting Back Initiative by Kay E. Sherwood Presented by Maddie Velez Presented by Maddie Velez.](https://reader036.fdocuments.in/reader036/viewer/2022072006/56649d135503460f949e7f1e/html5/thumbnails/14.jpg)
The Evaluation’s Ability to Explain
The Evaluation’s Ability to Explain
◊ Evaluation illustrate all central problems for evaluation
◊ Saxe wanted to undertake a more extensive implementation analysis, foundation unwilling to pay for it
◊ Fighting Back site activities revised after an initial publication in 1997
◊ Knickman claims the foundation had the wrong goals; He felt that there was a need for shorter-term goals
◊ Evaluation illustrate all central problems for evaluation
◊ Saxe wanted to undertake a more extensive implementation analysis, foundation unwilling to pay for it
◊ Fighting Back site activities revised after an initial publication in 1997
◊ Knickman claims the foundation had the wrong goals; He felt that there was a need for shorter-term goals
![Page 15: Evaluation of the Fighting Back Initiative by Kay E. Sherwood Presented by Maddie Velez Presented by Maddie Velez.](https://reader036.fdocuments.in/reader036/viewer/2022072006/56649d135503460f949e7f1e/html5/thumbnails/15.jpg)
Measuring and Interpreting Outcomes
Measuring and Interpreting Outcomes
◊ Key disagreements remain a piece of the national evaluation that focuses on the use of household survey data
◊ 3 waves of surveys- 1995, 1997, 1999
◊ Jellinek described early thinking on the evaluation
◊ Presentation of Results - A second area of disagreement
◊ Key disagreements remain a piece of the national evaluation that focuses on the use of household survey data
◊ 3 waves of surveys- 1995, 1997, 1999
◊ Jellinek described early thinking on the evaluation
◊ Presentation of Results - A second area of disagreement
![Page 16: Evaluation of the Fighting Back Initiative by Kay E. Sherwood Presented by Maddie Velez Presented by Maddie Velez.](https://reader036.fdocuments.in/reader036/viewer/2022072006/56649d135503460f949e7f1e/html5/thumbnails/16.jpg)
The Continuing Debate and the Foundation’s
Takeaway
The Continuing Debate and the Foundation’s
Takeaway◊ Knickman and Morris presented a summary of the Fighting Back experience to the foundation’s board in 4/’04
◊ Knickman focused on the fundamentals of complexity and the lessons about realistic scale for expected outcomes
◊ Teams were formed ◊ Substance abuse- D.A.R.E. and treatment reform
◊ Knickman and Morris presented a summary of the Fighting Back experience to the foundation’s board in 4/’04
◊ Knickman focused on the fundamentals of complexity and the lessons about realistic scale for expected outcomes
◊ Teams were formed ◊ Substance abuse- D.A.R.E. and treatment reform