Los Angeles County Evaluation System: An Outcomes Reporting Program

2008

Site Report Presentation to Contract Program Auditors

Purpose of this Presentation

To provide a brief overview of the Los Angeles County Evaluation System (LACES).To review the content and purpose of the Site Reports.Review those content areas that are the focus of the evaluation.

Overview of LACES

LACES is a partnership between ADPA, Treatment Providers, and UCLA/ISAP to conduct an evaluation - LACES is not an external study.

Treatment programs are contractually required to comply with the evaluation efforts.

Evaluate the alcohol and other drug treatment system.On-going evaluation, not a temporary study.

First… Some Questions…

How many programs are your responsible for monitoring?

How often do you visit the programs?

What’s a Site Report?

Have you heard of a site report?

Have you seen a site report?

Have you had the opportunity to compare the information in the site reports with ADAM?

A Site Report Is…

Brief: six pages, double-sided.Information is gathered from the admission & discharge LACPRS.Information is included on your site and for all sites of the same program type.

What is the Purpose of the Site Report?

To provide information concerning the key performance/outcome areas.To provide feedback to sites concerning how they are performing with regards to the focus areas of the evaluation.To provide feedback on how other similar programs are performing.

Contents of the Site Report

Executive SummaryIncludes overall participant admission and discharge numbers.Includes information on key outcomes.This will be the focus of this presentation as it includes the outcome information that will be used to evaluate programs.

Full ReportIncludes detailed information on all participants.Includes more details on changes in key outcomes.

Executive Summary

Single page, stand alone document.Covers the admission to discharge changes for your site compared to all sites of the same program type.Key Areas include

“Participant Flow” chart.“Performance Measures.”

Important Performance & Outcome Measures

Completion of treatment/Positive Compliance.Abstinence from/Reduction in drug/alcohol use.Exit InterviewsLength of Stay/Enrollment in Treatment (2 benchmarks)

30 Days = Engagement.90 Days = Retention.

All important aspects of the treatment plan.

Why Completion of Treatment?

Most outside of this field are interested in completion figures.Positive Compliance

Completed treatment.Left treatment prior to completion with satisfactory progress.

Others have adopted this mindset, but many have not.

Why Reduction/Abstinence?

Silly picture, for a silly question.

That is our business.If we cannot demonstrate abstinence or reductions in use, funders will move the money to those who can.

Interviews

Responses to all discharge LACPRS questions.One of the most important of the evaluation foci.Information from both admission and discharge is required to measure treatment effectiveness.No exit interviews = No measurement of changes that occurred during treatment.

Why Engagement & Retention?

First 30 days are critical to the outcome of treatment.Research shows that without an adequate amount of time in treatment, few improvements are found.Funding organizations are beginning to take note of engagement and retention rates.

Note on Documentation

Accuracy is extremely important.Length of stay/enrollment.Example – length of stay/enrollment.

• In 2004-2005, average length of stay was 105 days.

• 40% were in treatment for 30+ days.

If providers are not inputting discharge date accurately – what other information is being entered into LACPRS inaccurately?

Max LOS/LOE by Funding Source

All funding sources support 30- and 90-day length of stay/enrollment benchmarks.

FUNDING Outpatient LOE

Residential LOS

Day Care LOE

CalWORKS 365 Days 365 Days 365 Days

Drug Court 730 Days 90 Days N/A

Female Offender

N/A 180 Days N/A

General Relief 180 Days 180 Days 180 Days

PPN 180 Days 180 Days N/A

Prop 36 300 Days 90 Days 204 Days

Drug Medi-Cal No max length - approval is need for 180+ days.

Average Provider Performance

During the last 3 fiscal years, average performance in these areas was…

Does this make sense to you?

PERFORMANCE AREA Outpatient

Residential

Day Care

Reduction in Primary Drug Use

75.7% 97.7% 77.9%

30-Day Retention 71.8% 64.1% 68.9%

90-Day Retention 48% 38.1% 40.6%

Exit Interviews 48.6% 73.7% 47%

Roles of the Contract Monitor

Answer questions regarding site reports.Review reports for accurate information.Provide suggestions on how to improve outcomes – to come.Seek assistance from ADPA or UCLA for responses to questions, concerns, or additional assistance.

How to Improve Exit Interview Rates

Some ways to increase rates of exit interviews include

Informing the participant of the exit interview when admitted.Include completion of the exit interview as part of the individualized treatment plan.Remind the participant when they are nearing completion of their plan of the exit interview.

Improve Engagement & Retention?

Engage them from first contact• Make same day appointments if possible.

Appointment cards• To encourage forgetful clients.

Phone calls• To remind clients of appointments.• To follow-up with clients who may have

dropped out.

Other Suggestions?

Challenges

LACPRSProvides a lot of good information.Revisions and monitoring of information will be needed to ensure quality & validity.

Site ReportsAlso requires revisions to format & content.Not getting past Executive Directors’ eyes.

Your Views on the Site Reports

What do you think about the site reports?What have you heard from your providers?Do you have suggestions or recommended changed to the reports?

Future of the Site Reports

The Site Reports will be used to asses individual sites in the key areas described in this presentation.Over time peer performance standards will be developed.Programs that fall short of the standards will be offered information, training and other forms of assistance in order to bring their site’s performance up to the standard.

The End