PBHCI National Evaluation

RAND Corporation

&

National Council for Community Behavioral Healthcare

Welcome!

Audrey Burnam, PhD

Scharf Dec -10

The RAND Corporation

• Nonprofit institution that helps improve policy and decisionmaking through research and analysis.

• RAND is tasked with completing an independent and objective national-level evaluation of PBHCI

– Funded jointly by SAMHSA and ASPE

National Council

Jeff Capobianco, MA

Scharf Dec -10

The National Council

• The National Council for Community Behavioral Healthcare (National Council) has won a competitive grant funded jointly by SAMHSA and HRSA to establish the National Training and Technical Assistance Center for Primary and Behavioral Healthcare Integration.

• The National Council is working closely with RAND and will provide technical assistance to grantees to insure they are able to successfully provide the data required for the evaluation.

RAND’s Tasks

…back to Audrey Burnam, PhD

Scharf Dec -10

Overview of RAND’s Tasks

Conduct a national evaluation of the PBHCI program that answers 3 questions:

1. Outcomes: Does integrating primary and behavioral health care lead to improvements in mental and physical health of persons with SMI and/or substance use disorders?

2. Process: Is it possible to integrate the services provided by primary care providers and community-based behavioral health agencies?

3. Models: Which models and/or respective model features of integrated primary and behavioral health care lead to better mental and physical health outcomes?

Scharf Dec -10

Goals of This Webinar

• Part I:

– Achieve common understanding of:

• The framework for the national evaluation

• Data that support the national evaluation

• Procedures for transmitting data to RAND

• Part II:

– Data collection procedures

PART I: Framework for the National Evaluation

Deborah Scharf, PhD

Scharf Dec -10

Structure-Process-Outcomes Evaluation Model

STRUCTURE OF CAREWhat services are

available?

Core (required) components:• Screening/referral/treatment

• Registry/tracking system• Care management

• Prevention/wellness approach

Examples of optional components:• Co-location of nurse

practitioners/PCPs• Identification of primary care

supervising physician• Embedded nurse care managers

• Use of EBPs• Use of SBIRT

• Other

PROCESS OF CARETo what degree are services

implemented?

Examples: • Numbers/proportions of SMI

patients screened/referred/treated & for what

• Number of records/types of information tracked in registry

•Number of SMI patients connected with care manager &

for what

• Number of individualized wellness plans developed and

utilized

OUTCOMES OF CAREDoes it make a difference?

Improved overall health status of clients:

• Physical health outcomes (e.g., BMI, blood pressure, blood glucose or HbA1c, lipid profile)

• Mental health outcomes

Environment/Administrative Context(e.g., size, auspice, location, health IT)

Independent Variables Dependent Variables

Scharf Dec -10

Data Sources (I)

1. TRAC

2. Quarterly reports

3. Clinical registries

*Staff may be asked to participate in interviews or surveys

*Some sites may be invited to participate in other evaluation activities

Scharf Dec -10

Data Sources (II)

1. TRAC Entered directly into TRAC system

2. Quarterly Reports Uploaded to PBHCI SharePoint site

3. Clinical Registries a) Uploaded to PBHCI SharePoint site

or

b) Transmitted directly from a data extraction system

TRAC: CMHS Transformation Accountability

Susie Lovejoy, MS

Scharf Dec -10

Transformation Accountability (TRAC)

• In response to the 1993 Government Performance and Results Act (GPRA) requirements CMHS developed the TRAC system.

• TRAC is the web-based system through which all SAMHSA CMHS grants report performance measurement data. 

• Measures include: client level outcomes, infrastructure development indicators, measures of mental health promotion and prevention activities, and satisfaction with technical assistance services.

Scharf Dec -10

TRAC (II)

• All of you (grantees) have been trained on the Infrastructure, Prevention, & Promotion (IPP) portion of TRAC.  

• Grantees funded in Sept 2009 (cohort I) have been trained on the client-level outcomes measures for people receiving direct services through the grant. 

• Grantees funded in 2010 (cohorts II and III) will be trained in February to collect and enter the remaining data into TRAC

Scharf Dec -10

TRAC (III)

• Grantees must collect and report data on behavioral health outcomes for each person receiving services from the grant (NOMS).

• Grantee staff interview consumers and enter data into TRAC at:

– Entry into integrated care– Every 6 mos while enrolled– At discharge from the program 

Scharf Dec -10

TRAC (IV)

• For PBHCI, grantees must also log physical healthcare indicators in TRAC

• Required indicators: Height, weight, BP, glucose or A1C, cholesterol, triglycerides

• Optional indicators: Waist circumference, Breath CO (for smoking status)

Scharf Dec -10

TRAC (V)

• Intake, 6 months, discharge:

– Height– Weight– Blood pressure– Waist

circumference*– Breath CO*

• Intake, annually, discharge:

– Glucose or A1C– Triglycerides– Cholesterol– Successful 8h fast*

*Recommended

Note: 6 month indicators are collected quarterly but entered bi-annually

Scharf Dec -10

TRAC (IV)

• TRAC data can be downloaded– This can help you avoid double-entry

• Collect and enter the physical healthcare data in the same time windows as you collect and enter the TRAC client-level interviews.

• This TRAC change will be in place ~Feb 2011

PBHCI Secure SharePoint Site

Regan Main, BA

Scharf Dec -10

PBHCI Secure SharePoint Site

• SharePoint is the primary method of transmitting quarterly reports and clinical registry data to SAMHSA and RAND

Scharf Dec -10

SharePoint (II)

• Use the SharePoint site because it is secure– Meets HIPAA data protection requirements

• Each grant program has a password-protected folder

– Uploading data is easy!

Scharf Dec -10

SharePoint (III)

Log-on to SharePoint using the link below:

http://sharepointext.rand.org/sites/pbhci/default.aspx

*Watching this presentation from the archives? See separate file with additional SharePoint slides

Quarterly Reports

Nicole Eberhart, PhD

Scharf Dec -10

Quarterly Reports

• Required quarterly by every grantee – 1 report per program

• Uploaded to your folder in SharePoint

• Quarterly reports are Word documents– RAND provides a template– Blank templates are on SharePoint

Scharf Dec -10

Quarterly Report: Labeling

VERY IMPORTANT!

• Top of report:– Your grant number– Name of your site and program– Name of the Project Director– Name of the person to contact (if not the Project Director)

• Label the report for the dates covered and the date submitted

• Put the quarter number and year

• Label electronic attachments with something that includes the abbreviated site name, Quarter #, and Year.

• Example: ANN ARBOR CMHC 1st QTR 1/31/10

Scharf Dec -10

Quarterly Report: General Issues

• Aim for 3-5 pages

• Answer each question with regards to both

– Core program features• screening/assessment• treatment and referral• registry/tracking• care management• prevention/early intervention/wellness• referral/follow up

– Optional features, e.g.:• use of SBIRT and/or other evidence-based practices• primary care supervising physician • embedded nurse care manager, etc.

Scharf Dec -10

Quarterly Report: Content (I)

• Describe your program’s accomplishments

• Describe any changes in staffing since the last quarter

– Clinical discipline – Role – Duties of grant-supported new staff – Changes in % effort – Attach resumes and any formal correspondence made

between you and SAMHSA about staffing changes

• Describe the involvement of consumers and families in your project

Scharf Dec -10

• Barriers in implementing your program, solutions you generated, or your plans to resolve them

• Delays in the program as you originally proposed it, changes in content or timing

• Infrastructure activities, staff involved, and associated expenditures

• Wellness-related education and programming, staff involved, associated expenditures

Quarterly Report: Content (II)

Scharf Dec -10

• Progress with data collection (software, clinical registry, TRAC, qualitative data) and plans to monitor, analyze, and/or share the data

• Specific program components supported by grant monies

• Funding sources for key primary care services provided via your program, and sustainability beyond the official grant period

Quarterly Report: Content (III)

Scharf Dec -10

Quarterly Report: Content (IV)

• Which patients are eligible to receive PBHCI services

– All SMI patients? New SMI patients? SMI patients with physical health conditions? SMI patients receiving psychotropic medication?

• Staff involvement in PBHCI group activities or with SAMHSA Project Officer

– Group conference calls, grantee meetings, site visits and interaction with the Project Officer or Technical Assistance Provider(s).

• Other sources of funding, and proportion of PBHCI consumers who are also participating other funded programs

Scharf Dec -10

Glossary

• Assessment: Actions taken following a positive screen to provide a more comprehensive or in-depth picture of an individual patient’s specific physical health condition(s) and to determine the best service plan to address them

• Early intervention: Interventions that are appropriate for, and specifically target, people displaying the early signs and symptoms of a physical health condition.

• Prevention: Interventions that occur before the initial onset of a physical health condition to prevent the development of that condition.

• Qualitative data collection methods: Interviews, focus groups, etc.

• Screening: Refers to a preliminary procedure, such as a test or examination (e.g., blood pressure for hypertension, cholesterol for coronary artery disease), to detect the most characteristic sign(s) of a physical health condition that may require urgent attention or further investigation.

• Wellness-related education and programming activities: Activities that promote wellness in mind, body, and spirit. They may include educational activities (e.g., nutrition education, physical activity education), physical activities (e.g., yoga classes, walking groups), and psychosocial activities (e.g., peer support programs, health-focused social functions).

Clinical Registries

Deborah Scharf, PhD

Scharf Dec -10

Clinical Registries

• Two purposes:

1. Improve clinical care

2. Record consumer-level clinical interactions

• Data system for consumers with a specific type of disease diagnosed and treated at a practice*

– Allows care team members to proactively manage consumers with chronic diseases*

• Grantees must maintain a clinical registry

*Iowa Dept of Public Health, 2010

Scharf Dec -10

Benefits of a Registry*

• Enable the provider to ensure that all their patients are getting proper care

• Track progress of high-risk patients

• Identify the need for follow-up services

• Increase quality of care and improve patient outcomes

• Coordinate care and identify gaps

• Incorporate consensus guidelines for disease management

*Iowa Dept of Public Health, 2010

Scharf Dec -10

Evaluation Uses of a Clinical Registry

• Facilitate data collection

– Registries can and should include all key client-level data elements

• Standardize client-level data collection

– Data collection can be aligned with flow of integrated clinical care

Scharf Dec -10

Attempt To Balance Simplicity & Complexity for Serving Multiple Purposes

• Meets minimal reporting expectations• Provides baseline data w/out longitudinal, clinically relevant follow-up information

• Exceeds sites’ data collection resources/ capacities• Results in incomplete and/or unreliable data collection across sites

Overly Simplistic “Reasonable” Middle Ground Overly Complex

• Facilitates sites’ efforts to provide high-quality clinical care• Assists sites in meeting SAMHSA’s reporting requirements• Supports meaningful evaluation

Scharf Dec -10

Clinical Registry Items Extracted for the Evaluation

Each consumer’s total contacts including these services:

Physical Health

Mental Health

Substance Use

Wellness

Screening / Assessment X X X

Referral X X X X

Treatment Planning X X X

Medication Mgt. X X X

Counseling X X

EBPs X X

List EBPs / Activities X X X

Hospitalizations X X X

Scharf Dec -10

Clinical Registry Items Extracted for the Evaluation (II)

• Each consumer’s total contacts with:– Care managers– Primary care providers (MDs, LPNs, PAs)– Psychiatrists or psychiatric nurses– Counselors (LSW, psychologists, substance

abuse counselors, others)– Peer specialists– Other specialists (dentist, nutritionist, others)– Wellness programs

Scharf Dec -10

Clinical Registry Items Extracted for the Evaluation (III)

• IMPORTANT!– All data fields to be extracted from clinical

registries should be whole numbers• i.e., 1, 2, 3, … 99

– Even lists of EBPs should be coded as whole numbers

– RAND will provide you with a list of EBPs and wellness activities associated codes

• Example EBP: CBT = 1; MI = 2• Example wellness activity: Diabetes

education = 1; Smoking cessation = 2

Scharf Dec -10

Registry: For Cohort I

• Clinical registry data is taking the place of numeric data previously reported in quarterly reports

• Data are now collected at the patient-level – Not in aggregate (i.e., not at the program-level)

Scharf Dec -10

If You Already Have a Clinical Registry…

• Make sure it contains all necessary fields– Review these materials, pg 12 of your RFA, and talk to

your contact at the Technical Assistance Center

• Establish a plan for:– Extracting all necessary fields – Making all data fields numeric, whole numbers, and

consistent with PBHCI requirements

• Your registry will likely contain data that we do not collect for the evaluation

Scharf Dec -10

If You Do Not Have a Clinical Registry…

• Look carefully at the data to be extracted from PBHCI registries

• Work with the Technical Assistance Center to select a registry tool that will meet your clinical and research needs

Scharf Dec -10

When Choosing a Registry, Consider…

• To what extent will using the registry improve the quality of your program’s integrated care?

• To what extent is the organization of the registry aligned with the flow of integrated care at your site?

• To what extent will the registry help you automate your data collection and reporting?

THANK YOU!