FUNCTIONAL FAMILY THERAPY (FFT)

DEFINITIONS, MEASUREMENTS, & CALCULATIONS FOR INSPIRE DATA HIGHLIGHTS REPORT

1 1/13/2012

This glossary explains how outcomes presented in the INSPIRE Data Highlights Report from INSPIRE are defined and calculated. Calculations use data entered

into INSPIRE as well as data entered into the Client Services System (CSS; a web-based data collection and documentation system utilized by all FFT sites

world-wide and managed by FFT Inc.). Variables derived from INSPIRE are indicated in red, while those from the CSS are indicated in blue.

Outcome Definition Measurement & Calculation Population Served Number of youth who were open and active

This measure captures the total number of youth that received any FFT within the reporting period.

The number of youth meeting the following criteria: 1. Date of first session is before or during the reporting period. 2. Discharge date, if it exists, falls on the first day of the reporting period or

later.

Number of new youth opened this reporting period

Number of referred youth that entered the FFT program (had a first session) during this reporting period.

Number of youth for whom the first session date falls within the current reporting period. The goal of this measure is to capture the number of new youth who are enrolled in FFT within the reporting period.

Percent of enrolled youth who were at risk of placement

Percent of youth that began treatment during this reporting period that are at imminent risk of out-of-home placement if they do not participate in or successfully complete the FFT program. Includes youth enrolled in the FFT program in lieu of out-of-home placement, as a last chance at avoiding placement, stepping down from a placement, or expected to be placed out-of-home if they do not make progress in the FFT program. Does not include youth for whom there is not a clear and immediate placement risk (e.g., youth who could be placed if their behavior problems continue worsen over time or who would likely

Provider indicates yes or no to the question “Would this youth have been placed out of home if they were not accepted to participate in the FFT program?” Provider is instructed to respond based on information available at the point of enrollment and that the preferred data source is the referral source and/or agency with the authority to make this type of placement decision, although the FFT program supervisor or intake staff makes the final determination regarding whether the youth is at imminent risk of placement. (cont. on next page) Calculation of percent includes only youth with a first session date during the reporting period, for whom the question is answered. Formula is:

(# of youth at risk for placement) (# of youth at risk + # of youth not at risk)

Rev. 1/13/2012 2

be referred to another home/community treatment program if they do not succeed in FFT). Out-of-home placement refers to placement in a dependency system, foster care system, juvenile justice system, drug and alcohol system, or mental health system.

Number of new parents/caregivers served

For new youth enrolled in FFT this reporting period, the number of parents/caregivers participating in FFT. Based on the number of parents/caregivers involved at the start of treatment.

For reporting periods through Dec. 31, 2011: The number of parents/parent figures completing a Family Self Report (FSR) the first time the FSR is administered. (Note: If the FSR is not administered within the first month, parent/caregiver information is considered missing and therefore the count will be 0 for that youth. If the number of new parents/caregivers served is lower than expected, it may be because FSRs are not being completed as expected.) For reporting periods from Jan. 1, 2012 forward: The number of parents/caregivers is entered on the INSPIRE enrollment screen. For youth with a first session date during the reporting period (i.e., newly enrolled youth), the number of parents/caregivers is counted.

Percent of youth from each referral source

Of youth referred during the current reporting period, the percent that were referred by each source.

Percent is based on total number of youth with a referral date in this reporting period, for whom a referral source is indicated. May include cases that were assigned a therapist but were never opened (i.e., never had a first session). Only one referral source is counted for each youth. The CSS choices for referral source and their categorization for the INSPIRE report is as follows:

Juvenile Justice= probation + parole + diversion

Children & Youth = child welfare

Mental Health Services = mental health

Schools = school

Other = self-referral + internal agency referral + other Formula example:

(# of youth from referral source A) (total # of youth from referral sources

A + B + C + D + E)

Rev. 1/13/2012 3

Implementation Quality Assurance

Dissemination Adherence ratings in the desired range

The number of therapists with a GTR for the present reporting period, who had a Dissemination Adherence rating of at least 4, indicating degree to which the therapist completes FFT documentation in a timely fashion and follows FFT dissemination protocol (e.g., appropriate spacing of sessions, flexible scheduling, responsive to community partners).

As part of the FFT protocol, a Global Therapist Rating is completed at least every 4 months for each therapist and is based on ratings by the supervisor. Dissemination Adherence Ratings range from 0 to 6. If a therapist has more than one GTR for the reporting period, the average is used.

Fidelity ratings in the desired range

The number of therapists with a GTR for the present reporting period, who had a Fidelity Rating of at least 3, indicating clinical competence (e.g., sophistication of interventions, tailoring interventions to family) and clinical adherence to the model (applying the model as intended and “doing the right thing at the right time”). Note: It is expected that sites in Phase 1 and new therapists will not meet the threshold for Fidelity Ratings.

As part of the FFT protocol, a Global Therapist Rating is completed at least every 4 months for each therapist and is based on weekly ratings by the supervisor. Fidelity Ratings range from 0 to 6. If a therapist has more than one GTR for the reporting period, the average is used.

Rev. 1/13/2012 4

Discharged Youth

Number of youth discharged

Total number of youth discharged from the FFT program during this reporting period. Includes successful, unsuccessful, and administrative discharges.

Number of youth with a discharge date that falls within the given reporting period.

Successful discharges (number and percent)

Successful discharge is defined as completing all treatment phases of FFT and having a positive outcome.

Therapist indicates if youth completed all phases of FFT (yes or no). If yes, therapist indicates whether the outcome was positive, non-significant, or worse. Positive outcome is generally defined as completing all three phases and receiving ratings of three or above on an outcome measure completed by the youth, other family members, and the therapist. The 6-item outcome measure asks about changes in the family, family communication, family conflict, adolescent behavior, parenting skills, parental supervision. Ratings of 3 or above indicate that things are some, a lot, or very better. Therapist may override the ratings to indicate a positive outcome if clinical evidence supports such a decision. (cont. on next page) Youth is counted as successful if discharge date falls within the reporting period and the following criteria are met: 1. Youth completed FFT. 2. Youth had a positive outcome.

Percent is based on total number of discharges (i.e., successful, unsuccessful, and administrative discharges). Formula is:

(# of successfully discharged youth) (total # of discharged youth)

Unsuccessful discharges (number and percent)

Unsuccessful discharges are those that do not meet the criteria for successful discharge and were not closed for administrative reasons.

Therapist indicates if youth completed all phases of FFT (yes or no). If yes, therapist indicates whether the outcome was positive, non-significant, or worse (see above). If youth did not complete FFT, the therapist indicates if the youth dropped out or was never seen. If the youth dropped out, the therapist then selects the reason from a list of several possibilities such as “administrative discharge,” “placed out of home,” “quit after first session,” etc. Youth is counted as unsuccessful if discharge date falls within the reporting period and either:

Rev. 1/13/2012 5

1. The youth completed FFT but outcome was worse or no significant

change, or 2. Treatment was not completed and the reason for closure was not

“administrative discharge.” Percent is based on total number of discharges (i.e., successful, unsuccessful, and administrative discharges). Formula is:

(# of unsuccessfully discharged youth) (total # of discharged youth)

Administrative discharges (number and percent)

Number of youth whose cases were closed for administrative reasons including: didn't meet FFT criteria, incarcerated for pre-referral reasons only, funding was terminated, moved out of area, and treatment was ended by the referral source or family because of overlapping services. Overall, administrative discharges are cases closed prior to completion for non-clinical reasons or reasons beyond the program’s control.

Therapist indicates if youth completed all phases of FFT (yes or no). If yes, therapist indicates whether the outcome was positive, non-significant, or worse (see above). If youth did not complete FFT, the therapist selects the reason from a list of several possibilities. Youth is counted as an administrative discharge if discharge date falls within the reporting period, treatment was not completed, and the therapist indicates that the reason for closure is “administrative discharge.” Percent is based on total number of discharges (i.e., successful, unsuccessful, and administrative discharges) during the reporting period. Formula is:

(# of administratively discharged youth) (total # of discharged youth)

Average length of stay for successful discharges

The average length of treatment, in months, for youth who were successfully discharged from FFT during the reporting period.

Length of treatment is measured as the number of days from first session date to discharge date. Youth who were successfully discharged during the reporting period are included in the calculation.

Average length of stay for unsuccessful discharges

The average length of treatment, in months, for youth who were unsuccessfully discharged from FFT during the reporting period.

Length of treatment is measured as the number of days from first session date to discharge date. Youth who were unsuccessfully discharged during the reporting period are included in the calculation.

Rev. 1/13/2012 6

Youth satisfied with FFT (number and percent)

Number and percent of youth discharged (successfully or unsuccessfully) during this reporting period that reported satisfaction with their experience in the FFT program.

Therapist indicates yes or no for the question “Was the youth satisfied with his/her experience in the FFT program?” Program is instructed that the data source can be surveys, exit interviews, or another approved tool completed at the time of discharge from the FFT program, and to leave the question blank if they do not know the answer. If a survey is used, sites are encouraged to establish a consistent threshold/score for determining whether the respondent is satisfied. Calculation includes youth successfully and unsuccessfully discharged during the reporting period for whom the question is answered. Formula is:

(# of youth satisfied) _ (# of youth satisfied + # of youth not satisfied)

Youth whose parents/caregivers were satisfied with FFT (number and percent)

Number and percent of youth discharged (successful and unsuccessful) during this reporting period who had a parent/caregiver reporting satisfaction with their experience in the FFT program.

Therapist indicates yes or no for the question “Were the parent(s)/caregiver(s) satisfied with their experience in the FFT program?” Program is instructed that the data source can be surveys, exit interviews, or another approved tool completed at the time of discharge from the FFT program, and to leave the question blank if they do not know the answer. If a survey is used, sites are encouraged to establish a consistent threshold/score for determining whether the respondent is satisfied. Calculation of percent includes successfully and unsuccessfully discharged youth for whom the question is answered. Formula is:

(# of youth with satisfied caregiver) (# of youth with satisfied caregiver + # of youth with unsatisfied caregiver)

Rev. 1/13/2012 7

Outcomes at Discharge

Number of youth included in analysis

Outcomes at discharge are based on the number of youth who were successfully or unsuccessfully discharged during the reporting period. Administrative discharges are not included. For each outcome, the number of youth in the analysis depends on the number of youth for whom information was provided. The table below the graph, showing the number improving and number reporting, highlights the number of youth in the analysis for each outcome.

Percent who remained drug-free

The percent of youth discharged this reporting period (successfully or unsuccessfully) who were identified with a drug or alcohol problem at enrollment and were drug-free at discharge, as indicated by negative drug screens. Youth who were not referred due to concerns about substance use or who did not have drug screens during the final three months of FFT are not included in the analysis.

Youth meets the following criteria: 1. Therapist answers yes to the enrollment question “Was youth referred

due to concerns about his/her drug and alcohol abuse?” Therapist is instructed that tobacco use not be included and that preferred source of information is the referral source or a member of the youth’s interagency service planning team.

2. The youth had at least one drug screen with a date that falls within the last 3 months of treatment, for which the results were available to the therapist and were entered into INSPIRE.

3. The results of all drug screens during the last three months of treatment were negative.

Calculation of percent includes successfully or unsuccessfully discharged youth that were referred due to concerns about substance use and had at least one drug screen in the last three months of treatment. Formula is:

(# of youth with no positive drug screens) _ (# of youth with positive drug screens +

# of youth with no positive drug screens) “ N/A” indicates that no youth met the conditions for inclusion in the calculation (e.g., no youth had substance use concerns at referral, had drug screens in the last 3 months of treatment, and were successfully or unsuccessfully discharged).

Rev. 1/13/2012 8

Percent with no new criminal offenses

Percent of youth discharged this reporting period (successfully or unsuccessfully) who were not charged with a misdemeanor or felony, or adjudicated delinquent or criminally convicted, for a new offense that occurred while in the FFT program. Charges, adjudications, and convictions for offenses that occurred prior to FFT and non-criminal violations of the terms of probation are not counted.

At client discharge, therapist answered Yes to the statement: “Youth has had no new law violation since FFT treatment began.” Programs were instructed to answer No (thereby indicating a new offense) if the youth was charged with a misdemeanor or felony, or adjudicated delinquent or criminally convicted, for a new offense (other than non-criminal violation of the terms of probation) that occurred while in treatment. Calculation of percent includes all successfully and unsuccessfully discharged youth for whom the question is answered. Formula is:

(#with no new charges) (# with no new charges + # with new charges)

Percent living in the community

Percent of youth discharged this reporting period (successfully or unsuccessfully) who were living in the community at discharge (i.e., had not been removed from the home and placed prior to completion of FFT). Youth incarcerated or placed solely for an event that occurred prior to enrollment in FFT are not included in this analysis, since they are considered administrative discharges.

Therapist indicates if youth completed all phases of FFT (yes or no). If no, therapist selects the reason from a list of several possibilities. Youth is counted as living in community if:

1) youth completed treatment or 2) youth did not complete treatment and reason for non-completion is

not incarceration or placement out of home.

Calculation of percent includes all successfully and unsuccessfully discharged youth. Formula is:

(# of youth living in community) (# of successful + unsuccessful discharges)

Rev. 1/13/2012 9

Percent of families with improved functioning

Percent of youth discharged this reporting period (successfully or unsuccessfully) whose families showed improvement in functioning, as indicated by ratings from the youth, caregivers, and therapist.

The Outcome Measure is completed by the therapist (for all cases) as well as the youth and parents/caregivers (for cases that complete all phases of treatment). Ratings are given for 6 items asking about changes in the family, family communication, family conflict, adolescent behavior, parenting skills, parental supervision. Improved family functioning is defined as an average rating of 3 or above, which indicates that things are some, a lot, or very better . Calculation of percent includes all successfully and unsuccessfully discharged youth. Formula is:

(# of youth with improved family functioning) (# of successful + unsuccessful discharges)

Percent with improved school attendance

Of youth discharged this reporting period (successfully or unsuccessfully) who were referred due to concerns about school attendance, the percent that demonstrated improvement while in the FFT program.

Youth meets the following criteria: 1. Therapist indicated at enrollment that the youth was referred due to

concerns about school attendance. Preferred source of information is referral source or a member of the youth’s interagency service planning team.

2. Therapist responds “Improved” to the question, “Did the youth show improvement in school attendance during his/her time in FFT?” Therapist answers based on his/her clinical judgment.

Calculation of percent includes all successfully or unsuccessfully discharged youth that were referred due to concerns about school attendance, for whom the question about improvement is answered. Formula is:

(# of youth with improved school attendance) (# of youth with improved + no change + worse)

Rev. 1/13/2012 10

Percent with improved school performance

Of youth discharged this reporting period (successfully or unsuccessfully) who referred due to concerns about academic performance, the percent that demonstrated improvement while in the FFT program.

Youth meets the following criteria: 1. Therapist indicated at enrollment that the youth was referred due to

concerns about school performance (academic). Preferred source of information is referral source or a member of the youth’s interagency service planning team.

2. Therapist responds “Improved” to the question, “Did the youth show improvement in school attendance during his/her time in FFT?” Therapist answers based on his/her clinical judgment.

Calculation of percent includes all successfully or unsuccessfully discharged youth that were referred due to concerns about school performance, for whom the question about improvement is answered. Formula is:

(# of youth with improved school performance) (# of youth with improved + no change + worse)

Rev. 1/13/2012 11

Outcomes 6 Months Post-Discharge

Number of youth for whom data was available

The number of youth who were successfully or unsuccessfully discharge, for whom 6-month follow-up data was collected during this reporting period. For each follow-up outcome, the number of youth in the analysis depends on the number of youth for whom information was provided. The table below the graph, showing the number improving and number reporting, highlights the number of youth in the analysis for each outcome.

The following criteria are met: 1. Youth was successfully or unsuccessfully discharged (see above) 2. Follow-up was completed and data entered into INSPIRE. Source(s) of

data may include interviews, surveys, official records, etc . Information may be provided by the youth, a caregiver, a non-parent relative, or a community agency such as a probation officer or case worker.

3. Date of follow-up interview is within 5-7 months (149-211 days) from the discharge date.

No new charges or out-of-home placements

Percent of youth who did not have any of the following reported at 6-months post-discharge: 1) new misdemeanor or felony charges, 2) adjudicated delinquent or criminally convicted for a new offense, or 3) placement out of the home. Includes both delinquency placements and other placements for youth behavior or mental health.

A youth is counted as having no new charges or placements if the following criteria are met: 1. A NO is indicated for “Has the youth been charged with a misdemeanor

or felony, or adjudicated delinquent or criminally convicted, for a new offense since discharge/last interview?” (non-criminal probation violations are excluded) AND

2. Either a. NO is indicated for “Has the youth experienced any behavior issues

since discharge/last interview?” b. YES is indicated for “Has the youth experienced any behavior

issues” but the respondent reports that the youth has not been placed for any behavior problems.

Calculation of percent includes all youth with a 6-month post-discharge interview this reporting period for whom either 1) all relevant information is available or 2) not all relevant questions are answered, but a new offense or placement is indicated. Formula is:

(# with no new charges or placement) _ (# with no new charges or placement +

# with a new charge or placement)

Rev. 1/13/2012 12

No drug use

Percent of youth with no drug or alcohol use reported at 6-months post-discharge. Tobacco use is not considered when determining whether the youth is using substances.

Youth is counted as drug-free if: 1. The respondent answers YES to the question “Has the youth

experienced any behavior issues since discharge/last interview?” but does not report the presence of drug or alcohol use OR

2. The respondent indicates that the youth has not had any behavior issues since discharge.

Calculation of percent includes all youth with a 6-month post-discharge interview this reporting period for whom the question about behavior issues since discharge is answered. Formula is:

(# with no drug use indicated) _ (# with behavior issues + # without behavior issues)

No school problems

Percent of youth with no school problems reported at 6-months post-discharge.

Youth is counted if the respondent answers NO when asked “Has the youth experienced any issues in school since discharge/last interview?” Calculation of percent includes all youth with a 6-month post-discharge interview this reporting period for whom the question is answered. Formula is:

(# with no school problems) (# with school problems + # without school problems)

When the value for an outcome is “n/a,” it indicates that either 1) the outcomes is not applicable (e.g., the number of youth successfully discharged is not applicable if no youth were discharged during the reporting period) or 2) data needed to calculate the outcome was missing or not available.

This document has been prepared by the EPISCenter. The EPISCenter is a project of the Prevention Research Center, College of Health and Human Development, Penn State University, and is funded by the Pennsylvania Commission on Crime and Delinquency and the Pennsylvania Department of Public Welfare as a component of the Resource

Center for Evidence-Based Prevention and Intervention Programs and Practices.

The EPISCenter can be contacted at 814-863-2568 or via email at EPISCenter@psu.edu.