FY17 18 MEDI-CAL SPECIALTY MENTAL HEALTH EXTERNAL … and... · MHP Location — Santa Clara MHP...

FY17–18 MEDI-CAL SPECIALTY MENTAL HEALTH

EXTERNAL QUALITY REVIEW

SANTA CLARA MHP FINAL REPORT

Behavioral Health Concepts, Inc.

5901 Christie Avenue, Suite 502

Emeryville, CA 94608

[email protected]

www.caleqro.com

855-385-3776

Prepared for:

California Department of

Health Care Services (DHCS)

Review Dates:

January 23-25, 2018

- 2 -

Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18

TABLE OF CONTENTS SANTA CLARA MHP SUMMARY OF FINDINGS .............................................................................. 5

Introduction ................................................................................................................................................................................ 5

Access ............................................................................................................................................................................................ 5

Timeliness.................................................................................................................................................................................... 6

Quality ........................................................................................................................................................................................... 6

Outcomes ..................................................................................................................................................................................... 6

INTRODUCTION...................................................................................................................................... 8

Validation of Performance Measures ............................................................................................................................... 8

Performance Improvement Projects ................................................................................................................................ 9

MHP Health Information System Capabilities .............................................................................................................. 9

Validation of State and County Consumer Satisfaction Surveys ........................................................................... 9

Review of Recommendations and Assessment of MHP Strengths and Opportunities ................................ 9

PRIOR YEAR REVIEW FINDINGS, FY16-17 .................................................................................. 11

Status of FY16–17 Review of Recommendations ..................................................................................................... 11

Changes in the MHP Environment and Within the MHP—Impact and Implications ................................ 15

PERFORMANCE MEASUREMENT .................................................................................................... 17

Total Beneficiaries Served ................................................................................................................................................. 18

Penetration Rates and Approved Claim Dollars per Beneficiary ....................................................................... 18

High-Cost Beneficiaries ....................................................................................................................................................... 22

Timely Follow-up After Psychiatric Inpatient Discharge ...................................................................................... 23

Diagnostic Categories .......................................................................................................................................................... 24

Performance Measures Findings—Impact and Implications .............................................................................. 25

PERFORMANCE IMPROVEMENT PROJECT VALIDATION ....................................................... 27

Santa Clara MHP PIPs Identified for Validation ........................................................................................................ 27

Clinical PIP—Welcome to Treatment Group.............................................................................................................. 29

Non-clinical PIP—2018 Contract Renewal Process ................................................................................................ 31

PIP Findings—Impact and Implications ...................................................................................................................... 32

PERFORMANCE AND QUALITY MANAGEMENT KEY COMPONENTS ................................... 33

Access to Care ......................................................................................................................................................................... 33

Timeliness of Services ......................................................................................................................................................... 35

Quality of Care ........................................................................................................................................................................ 36

Key Components Findings—Impact and Implications .......................................................................................... 41

CONSUMER AND FAMILY MEMBER ............................................................................................... 45

FOCUS GROUPS ..................................................................................................................................... 45

Consumer/Family Member Focus Group 1 ................................................................................................................ 45

Consumer/Family Member Focus Group 2 ................................................................................................................ 46

Consumer/Family Member Focus Group Findings—Implications ................................................................... 47

- 3 -


INFORMATION SYSTEMS REVIEW ................................................................................................. 49

Key Information Systems Capabilities Assessment (ISCA) Information Provided by the MHP ........... 49

Summary of Technology and Data Analytical Staffing ........................................................................................... 50

Current Operations ............................................................................................................................................................... 50

Priorities for the Coming Year ......................................................................................................................................... 51

Major Changes Since Prior Year ...................................................................................................................................... 51

Other Significant Issues ...................................................................................................................................................... 52

Plans for Information Systems Change ........................................................................................................................ 53

Current Electronic Health Record Status ..................................................................................................................... 53

Personal Health Record ...................................................................................................................................................... 54

Medi-Cal Claims Processing .............................................................................................................................................. 54

Information Systems Review Findings—Implications .......................................................................................... 55

SITE REVIEW PROCESS BARRIERS ................................................................................................. 57

CONCLUSIONS ....................................................................................................................................... 58

Strengths and Opportunities ............................................................................................................................................ 58

Recommendations................................................................................................................................................................. 61

ATTACHMENTS .................................................................................................................................... 62

Attachment A—On-site Review Agenda ...................................................................................................................... 63

Attachment B—Review Participants ............................................................................................................................. 64

Attachment C—Approved Claims Source Data ......................................................................................................... 69

Attachment D—PIP Validation Tool .............................................................................................................................. 70

- 4 -


LIST OF TABLES AND FIGURES Table 1: MHP Medi-Cal Enrollees and Beneficiaries Served in CY16, by Race/Ethnicity ................... 18

Table 2: High-Cost Beneficiaries ................................................................ Error! Bookmark not defined.

Table 3: PIPs Submitted by MHP ............................................................................................................. 27

Table 4: PIP Validation Review ............................................................................................................... 28

Table 5: PIP Validation Review Summary .............................................................................................. 29

Table 6: Access to Care Components ...................................................................................................... 33

Table 7: Timeliness of Services Components ......................................................................................... 35

Table 8: Quality of Care Components...................................................................................................... 37

Table 9: Distribution of Services, by Type of Provider ......................................................................... 49

Table 10: Summary of Technology Staff Changes ................................................................................... 50

Table 11: Summary of Data Analytical Staff Changes ............................................................................. 50

Table 12: Primary EHR Systems/Applications ...................................................................................... 51

Table 13: EHR Functionality .................................................................................................................... 53

Table 14: MHP Summary of Short Doyle/Medi-Cal Claims ...................... Error! Bookmark not defined.

Table 15: Summary of Top Three Reasons for Claim Denial ................... Error! Bookmark not defined.

Figure 1A: Overall Average Approved Claims per Beneficiary, CY14-16 Error! Bookmark not defined.

Figure 1B: Overall Penetration Rates, CY14-16 ........................................ Error! Bookmark not defined.

Figure 2A: Foster Care Average Approved Claims per Beneficiary ......... Error! Bookmark not defined.

Figure 2B: Foster Care Penetration Rates, CY14-16 ................................. Error! Bookmark not defined.

Figure 3A: Latino/Hispanic Average Approved Claims per Beneficiary, CY14-16 Error! Bookmark not

defined.

Figure 3B: Latino/Hispanic Penetration Rates, CY14-16 ......................... Error! Bookmark not defined.

Figure 4A: 7-day Outpatient Follow-up and Rehospitalization Rates ..... Error! Bookmark not defined.

Figure 4B: 30-day Outpatient Follow-up and Rehospitalization Rates ... Error! Bookmark not defined.

Figure 5A: Beneficiaries Served, by Diagnostic Categories, CY16 ........... Error! Bookmark not defined.

Figure 5B: Total Approved Claims by Diagnostic Categories, CY16 ........ Error! Bookmark not defined.

- 5 -


SANTA CLARA MHP SUMMARY OF FINDINGS

Beneficiaries Served in Calendar Year 2016 22,901

MHP Threshold Language(s) Spanish, Vietnamese, Mandarin, Tagalog, Cantonese, Farsi

MHP Size— Large

MHP Region — Bay Area

MHP Location — Santa Clara

MHP County Seat — San Jose

Introduction

This is a large Bay Area county and provides services and programs for a multilingual, culturally

diverse community. With key stakeholders, the Mental Health Plan (MHP) has come closer to their

vision to fully integrate the values, infrastructure, processes, services, and supports to better meet

the needs of consumers and their families. It achieved combining its departments into the

Integrated Behavioral Health Organization and continues to merge models of integrated care and

apply best practices throughout its system.

During the fiscal year 2017-2018 (FY17-18) review, California External Quality Review

Organization (CalEQRO) reviewers found the following overall significant changes, efforts, and

opportunities related to access, timeliness, quality, and outcomes of the MHP and its contract

provider services. Further details and findings from EQRO-mandated activities are provided in this

report.

Access

The MHP revised its processes for access to services with closer integration of the mental health

Access Call Center and the substance use treatment services (SUTS) Gateway Center. It comes

closer to integration by moving the Access Call Center to the same Cisco call center software that

the Gateway Center uses. With this process, a new streamlined screening tool has been

implemented that reduces the time needed to refer a client.

This simplifies transfer between the two call centers and gives staff access to information gathered

by each call center, yet they remain physically and operationally separate organizational units.

Improvements are underway to fully integrate these units.

- 6 -


The MHP contracted with both MTM Consulting and Harbage Consulting to review both the quality

and fiscal impacts of its system. This work continues with the intention of creating efficiencies and

seamless service delivery.

Timeliness

Various services focused on delivering expedited care that includes the expansion of the jail

diversion continuum of care for timely treatment post incarceration, launching four new mobile

crisis teams, and a crisis residential unit (CSU) that expanded timely response to urgent conditions.

The focus on the Welcome to Treatment group as a component of the clinical PIP was successful in

strategies to improve timeliness and subsequently improve timely triage and identification of

consumer needs.

The rate for the 7-day follow-up appointment post hospitalization has improved, and the 30-day

follow-up appointment shows greater improvement. This may be a result of an inpatient liaison to

assure consumers receive a follow-up session.

Quality

The MHP reflects dedication to quality from leadership to line staff. The system continues to adapt

its procedures to further advance a quality management organization. The challenges remain in the

ability to compete for, and hire, qualified personnel to keep up with its intended progress. The

system itself is growing and continues to experience uncertainty in these times of economic,

political and fiscal challenges.

Multiple initiatives continue to provide the foundation for on-going services to targeted groups.

Examples include the waiver for the Organized Delivery Services (ODS) for substance use disorders,

the Whole Person Care (WPC) grant, the Coordination Care Reform (CCR) policies and procedures

modeled after the Katie A. implementation, and its collaboration with jail diversion provide access

for these vulnerable populations.

The Epic HealthLink and Netsmart myAvatar conversion plan for simultaneous implementations

are ambitious undertakings given the nominal level of available technical and project management

staffing in the MHP for its information systems.

The MHP makes extensive use of data to inform decision-making and in the last year has begun

producing information dashboards that are informative and widely available.

Outcomes

To further support its operational integration across all facets, the MHP successfully developed the

infrastructure at the executive level to manage the system changes. It recruited and hired four new

executive positions for Administrative Services, Adult/Older Adult Services, Children, Youth and

Family Services and Quality Management.

- 7 -


The MHP has created a Consumer and Family Affairs Division Director position reporting to the

deputy director. The position requirements include lived experience and will provide leadership,

oversight, and direction for peer staff and peer programs.

An 18-month to 24-month project plan and implementation timeline for the adult/older adult

redesign is under development.

The MHP expanded project plans and developed implementation timelines for the new Mental

Health Services Act (MHSA) innovation projects which included individual placement and support

employment, faith-based supports, psychiatric emergency response teams (PERTS) and adolescent

treatment initiatives.

Outcome tools are used and intended to be included within the electronic health record (EHR) as

well which will advance access to data regarding consumer progress.

- 8 -


INTRODUCTION The United States Department of Health and Human Services (HHS), Centers for Medicare and

Medicaid Services (CMS) requires an annual, independent external evaluation of State Medicaid

Managed Care programs by an External Quality Review Organization (EQRO). External Quality

Review (EQR) is the analysis and evaluation by an approved EQRO of aggregate information on

quality, timeliness, and access to health care services furnished by Prepaid Inpatient Health Plans

(PIHPs) and their contractors to recipients of State Medicaid managed care services. The CMS (42

CFR §438; Medicaid Program, External Quality Review of Medicaid Managed Care Organizations)

rules specify the requirements for evaluation of Medicaid managed care programs. These rules

require an on-site review or a desk review of each Medi-Cal Mental Health Plan.

The State of California Department of Health Care Services (DHCS) contracts with 56 county Medi-

Cal MHPs to provide Medi-Cal covered specialty mental health services (SMHS) to Medi-Cal

beneficiaries under the provisions of Title XIX of the federal Social Security Act.

This report presents the FY17-18 findings of an EQR of the Santa Clara MHP by the California

External Quality Review Organization, Behavioral Health Concepts, Inc. (BHC).

The EQR technical report analyzes and aggregates data from the EQR activities as described below:

Validation of Performance Measures1

Both a statewide annual report and this MHP-specific report present the results of CalEQRO’s

validation of eight mandatory performance measures (PMs) as defined by DHCS. The eight PMs

include:

• Total beneficiaries served by each county MHP;

• Total costs per beneficiary served by each county MHP;

• Penetration rates in each county MHP;

• Count of Therapeutic Behavioral Services (TBS) beneficiaries served compared to the

4percent Emily Q. Benchmark2;

• Total psychiatric inpatient hospital episodes, costs, and average length of stay (LOS);

1 Department of Health and Human Services. Centers for Medicare and Medicaid Services (2012). Validation of Performance

Measures Reported by the MCO: A Mandatory Protocol for External Quality Review (EQR), Protocol 2, Version 2.0,

September, 2012. Washington, DC: Author. 2 The Emily Q. lawsuit settlement in 2008 mandated that the MHPs provide TBS to foster care children meeting certain at-risk

criteria. These counts are included in the annual statewide report submitted to DHCS, but not in the individual county-level

MHP reports.

- 9 -


• Psychiatric inpatient hospital 7-day and 30-day rehospitalization rates;

• Post-psychiatric inpatient hospital 7-day and 30-day Specialty Mental Health Services

(SMHS) follow-up service rates; and

• High-Cost Beneficiaries (HCBs), incurring approved claims of $30,000 or higher during a

calendar year.

Performance Improvement Projects3

Each MHP is required to conduct two Performance Improvement Projects (PIPs)—one Clinical and

one Non-clinical—during the 12 months preceding the review. The PIPs are discussed in detail later

in this report.

MHP Health Information System Capabilities4

Using the Information Systems Capabilities Assessment (ISCA) protocol, CalEQRO reviewed and

analyzed the extent to which the MHP meets federal data integrity requirement for Health

Information Systems (HIS), as identified in 42 CFR §438.242. This evaluation included a review of

the MHP’s reporting systems and methodologies for calculating PMs.

Validation of State and County Consumer Satisfaction Surveys

CalEQRO examined available consumer satisfaction surveys conducted by DHCS, the MHP, or its

subcontractors.

CalEQRO also conducted 90-minute focus groups with beneficiaries and family members to obtain

direct qualitative evidence from beneficiaries.

Review of Recommendations and Assessment of MHP Strengths

and Opportunities

The CalEQRO review draws upon prior years’ findings, including sustained strengths, opportunities

for improvement, and actions in response to recommendations. Other findings in this report

include:

3 Department of Health and Human Services. Centers for Medicare and Medicaid Services (2012). Validating Performance

Improvement Projects: Mandatory Protocol for External Quality Review (EQR), Protocol 3, Version 2.0, September 2012.

Washington, DC: Author. 4 Department of Health and Human Services. Centers for Medicare and Medicaid Services (2012). EQR Protocol 1: Assessment

of Compliance with Medicaid Managed Care Regulations: A Mandatory Protocol for External Quality Review (EQR), Protocol

1, Version 2.0, September 1, 2012. Washington, DC: Author.

- 10 -


• Changes, progress, or milestones in the MHP’s approach to performance management —

emphasizing utilization of data, specific reports, and activities designed to manage and

improve quality.

• Ratings for key components associated with the following three domains: access, timeliness,

and quality. Submitted documentation as well as interviews with a variety of key staff,

contracted providers, advisory groups, beneficiaries, and other stakeholders inform the

evaluation of the MHP’s performance within these domains. Detailed definitions for each of

the review criteria can be found on the CalEQRO website, www.caleqro.com.

http://www.caleqro.com/

- 11 -


PRIOR YEAR REVIEW FINDINGS, FY16-17 In this section, the status of last year’s (FY16-17) recommendations are presented, as well as

changes within the MHP’s environment since its last review.

Status of FY16–17 Review of Recommendations

In the FY16-17 site review report, the CalEQRO made a number of recommendations for

improvements in the MHP’s programmatic and/or operational areas. During the FY17-18 site visit,

CalEQRO and MHP staff discussed the status of those FY16-17 recommendations, which are

summarized below.

Assignment of Ratings

Met is assigned when the identified issue has been resolved.

Partially Met is assigned when the MHP has either:

• Made clear plans and is in the early stages of initiating activities to address the

recommendation; or

• Addressed some but not all aspects of the recommendation or related issues.

Not Met is assigned when the MHP performed no meaningful activities to address the

recommendation or associated issues.

Key Recommendations from FY16-17

Recommendation #1: Develop a project plan to integrate the two Access Call Centers as the

current model is not an effective solution to support community-based capacity challenges.

Implement the plan as soon as practical since the MHP identified the lack of community-based

capacity as the number one priority.

Status: Met

• The MHP brought the mental health Access Call Center and the SUTS Gateway Center

closer to integration by moving the Access Call Center to the same Cisco call center

software that the Gateway Center uses. This simplifies transfer of callers between the

two call centers and gives them access to information gathered by each call center, but

they remain physically and operationally separate organizational units. The MHP

recognized more work is required to integrate these units.

• In addition, a new streamlined screening tool has been implemented which reduces the

amount of time needed by Access Call Center staff to refer a client.

- 12 -


• The MHP has contracted with MTM Consulting, a nationally recognized behavioral

health consulting group, with expertise in call center operations and same day access.

Executive leaders, division directors and center managers from both call centers have

been working collaboratively with MTM Consulting to evaluate operational and

performance data.

• While still in the evaluation phase, several potential areas for improvement have been

identified and preliminary planning for implementation has begun. The preliminary

improvement plans include:

o Developing policy and procedures for same day access into clinics

o Changes to scheduling management of clinical staff

o Updating no-show and cancellation policies

o Reducing documentation processes

Recommendation #2: Implementation of Practice Management System Solution (PMSS) electronic

health record addressing two critical elements. Those elements are:

• Knowledge of interoperability standards: The project will require that both Information

Technology (IT) and behavioral health program staff have advance knowledge of

interoperability (two-way exchange of data between systems) standards.

• Communication plan: As contract providers serve over 85 percent of specialty mental

services, a communication plan that shares technical data exchange requirements with

contractors or their IS vendors is a must.

Status: Not Met

• The MHP has selected its PMSS and focused its efforts on the two-way exchange of data

between contract providers and the MHP. Not yet resolved is the development of a

realistic timeline for a communications plan that shares technical data exchange

information with contract providers.

• With a target date of July 1, 2018, for initiating electronic data interchange (EDI)

between MHP and its numerous contract providers, it was not clear that

interoperability standards had been effectively communicated to their management,

their technical staff, and their system vendors.

• Since EDI involves a two-way information exchange, there needs to be a formal process

to ensure not just the MHP, but also the contract providers and their vendors are ready

to participate in an EDI go-live.

• In addition, issues of confidentiality and privacy for mental health, substance use clients,

and its contract providers will need to be addressed.

- 13 -


• Operational meetings continue with the CBOs to coordinate change management

activities and address communication strategies for staff.

Recommendation #3: Continue to work with its county Human Resources division to expedite and

expand recruitment strategies for increasing line staff hires for the vacant positions, and to

consider an equitable step increase for clinical supervision, critical to its expansion efforts.

Status: Partially Met

• The MHP has been successful in filling line staff positions.

• The MHP has requested a third step for clinical staff, recognizing the importance of

clinical supervision and a career pathway for clinicians. The MHP has drafted a job

specification and the executive team met with the deputy county executive for the

employee services agency and the deputy director for human resources, yet this issue

remains under review. Confirmation of this class has not been established.

• Clinical supervisors remain without a formal classification for their positions and are

assigned these as other duties. As a result, without the clinical supervisor class, the

supervisory experience cannot be used as qualifying experience when applying for

management positions.

Recommendation #4: Investigate the feasibility of using telepsychiatry services to augment

medication support or other tele-health services as an improvement option.

Status: Met

• A provider survey was developed and distributed by staff to determine contract

provider interest in telehealth.

• A telepsychiatry policy has been drafted and a pilot telehealth project with one of the

larger contract providers continues. This remains in the planning and development

stage to support expansion efforts.

• Telepsychiatry appears to still be in the investigative stage as a pilot implementation

functioning within the MHP was not developed.

Recommendation #5: Resume discussions with the county Human Resources division addressing

the need for a peer employee career ladder and consider full-time benefitted positions with job

mobility.

Status: Met

• The MHP has created a Consumer and Family Affairs Division Director position

reporting to the deputy director. The position requirements include lived experience.

The position will provide leadership, oversight, and direction for peer staff and peer

programs.

- 14 -


• The MHP director has met with the county consumer employees, who have researched

job classifications from other counties that support a peer career ladder.

• Discussions about a peer employee career ladder are reported between the MHP and

the county human resources, but no concrete steps towards a peer employee career

ladder have, so far, come from those discussions.

• Consumer staff are playing pivotal roles in the development of the Peer Respite Center

through

• WPC, and the development of a Peer Navigation program is planned for 2018.

- 15 -


Changes in the MHP Environment and Within the MHP—Impact

and Implications

Discussed below are any changes since the last CalEQRO review that were identified as having a

significant effect on service provision or management of those services. This section emphasizes

systemic changes that affect access, timeliness, and quality, including any changes that provide

context to areas discussed later in this report.

Access to Care

• The MHP implemented two adult community-based crisis residential facilities. It is

preparing to launch county-operated mobile crisis teams and open a CSU in January

2018.

• The MHP implemented the Cisco phone system in the mental health Access Call Center

to link with SUTS Gateway Center to improve call response time and capture call data

for service analysis. However, the two call centers, as their names imply, remain

operationally and physically separate organizational units.

• The leadership participated in the Santa Clara Valley Health and Hospital Systems WPC

implementation of system-wide care coordination planning.

• Expanded initiatives include the Jail Diversion continuum of care to transition

consumers from custody into the community at the new Muriel Wright service complex.

• Children’s services integrated the community care reform program principles in

partnership with social services and juvenile probation.

• The executive team convened a professional shortage workgroup to identify innovative

approaches to recruit and retain licensed staff.

Timeliness of Services

• The expansion of the jail diversion continuum of care expedited timely treatment post

incarceration.

• The MHP launched four new mobile crisis teams and a CSU which expanded timely

response to urgent conditions.

Quality of Care

• The department developed its Integrated Organizational Structure combining multiple

departments.

- 16 -


• Management recruited and filled four new executive positions for Administrative

Services, Adult/Older Adult Services, Children, Youth and Family Services and Quality

Management.

• Leadership created a Consumer and Family Affairs Division Director position.

• The MHP selected Epic HealthLink and Netsmart myAvatar as the county electronic

health record and launched the discovery process.

• Data services developed system and operations dashboards to ensure compliance with

the CMS much-anticipated Mega-Rule update and to track and analyze data for

continuous quality improvement.

• The management took lead roles in the Santa Clara Valley Health and Hospital System’s

WPC implementation of system-wide care coordination planning.

Consumer Outcomes

• The MHP developed and launched an 18 to 24-month project plan and implementation

timeline for the adult/older adult redesign.

• Access activities integrated the mental health Access Call Center and substance use

SUTS Gateway Center on the same call center system and piloted one same-day access

clinic.

• Service expansion included plans and implementation timelines for MHSA innovation

projects. These were individual placement and supportive employment, faith-based

supports, PERTS and headspace which is based on the successful Australian headspace

model to treat emerging mental health problems in adolescents.

• The MHP created a Consumer and Family Affairs Division Director position.

- 17 -


PERFORMANCE MEASUREMENT As noted above, CalEQRO is required to validate the following PMs as defined by DHCS:

• Total beneficiaries served by each county MHP;

• Total costs per beneficiary served by each county MHP;

• Penetration rates in each county MHP;

• Count of TBS Beneficiaries Served Compared to the 4percent Emily Q. Benchmark (not

included in MHP reports; this information is included in the Annual Statewide Report

submitted to DHCS);

• Total psychiatric inpatient hospital episodes, costs, and average LOS;

• Psychiatric inpatient hospital 7-day and 30-day rehospitalization rates;

• Post-psychiatric inpatient hospital 7-day and 30-day SMHS follow-up service rates; and

• HCBs incurring $30,000 or higher in approved claims during a calendar year.

HIPAA Suppression Disclosure:

Values are suppressed to protect confidentiality of the individuals summarized in the data sets

where beneficiary count is less than or equal to eleven (*). Additionally, suppression may be

required to prevent calculation of initially suppressed data, corresponding penetration rate

percentages (n/a); and cells containing zero, missing data or dollar amounts (-).

- 18 -


Total Beneficiaries Served

Table 1 provides detail on beneficiaries served by race/ethnicity.

Starting with CY16 performance measures, CalEQRO has incorporated the ACA Expansion data in

the total Medi-Cal enrollees and beneficiaries served. See Attachment C, Table C1 for the

penetration rate and approved claims per beneficiary for just the CY16 ACA Penetration Rate and

Approved Claims per Beneficiary.

Penetration Rates and Approved Claim Dollars per Beneficiary

The penetration rate is calculated by dividing the number of unduplicated beneficiaries served by

the monthly average enrollee count. The average approved claims per beneficiary served per year

is calculated by dividing the total annual dollar amount of Medi-Cal approved claims by the

unduplicated number of Medi-Cal beneficiaries served per year.

Regarding calculation of penetration rates, the Santa Clara MHP uses the same method used by

CalEQRO.

Race/Ethnicity

Average Monthly

Unduplicated

Medi-Cal Enrollees

% Enrollees

Unduplicated

Annual Count

of Beneficiaries

Served

% Served

White 58,907 13.0% 5,011 21.9%

Latino/Hispanic 168,992 37.2% 8,751 38.2%

African-American 15,085 3.3% 1,236 5.4%

Asian/Pacific Islander 159,007 35.0% 4,346 19.0%

Native American 1,602 0.4% 166 0.7%

Other 50,363 11.1% 3,391 14.8%

Total 453,954 100% 22,901 100%

Table 1: Santa Clara MHP Medi-Cal Enrollees and Beneficiaries Served in CY16,

by Race/Ethnicity

The total for Average Monthly Undupl icated Medi-Cal Enrol lees i s not a di rect sum of the averages above i t.

The averages are ca lculated independently.

- 19 -


Figures 1A and 1B show 3-year (CY14-16) trends of the MHP’s overall approved claims per

beneficiary and penetration rates, compared to both the statewide average and the average for

large MHPs.

- 20 -


Figures 2A and 2B show 3-year (CY14-16) trends of the MHP’s foster care (FC) approved claims per beneficiary and penetration rates, compared to both the statewide average and the average for large MHPs.

- 21 -


Figures 3A and 3B show 3-year (CY14-16) trends of the MHP’s Latino/Hispanic approved claims

per beneficiary and penetration rates, compared to both the statewide average and the average for

large MHPs.

- 22 -


High-Cost Beneficiaries

Table 2 compares the statewide data for High-Cost Beneficiaries for CY16 with the MHP’s data for

CY16, as well as the prior two years. HCBs in this table are identified as those with approved claims

of more than $30,000 in a year.

See Attachment C, Table C2 for the distribution of the MHP beneficiaries served by approved claims

per beneficiary (ACB) range for three cost categories: under $20,000; $20,000 to $30,000; and

those above $30,000.

MHP YearHCB

Count

Total

Beneficiary

Count

HCB %

by

Count

Average

Approved

Claims

per HCB

HCB

Total Claims

HCB % by

Approved

Claims

Statewide CY16 19,019 609,608 3.12% $53,215 $1,012,099,960 28.90%

CY16 2,362 22,901 10.31% $56,592 $133,669,749 45.85%

CY15 1,315 22,707 5.79% $54,684 $71,909,163 35.71%

CY14 1,149 17,407 6.60% $56,985 $65,475,939 39.47%

Table 2: Santa Clara MHP High-Cost Beneficiaries

Santa Clara

- 23 -


Timely Follow-up After Psychiatric Inpatient Discharge

Figures 4A and 4B show the statewide and MHP 7-day and 30-day outpatient follow-up and

rehospitalization rates for CY15 and CY16.

- 24 -


Diagnostic Categories

Figures 5A and 5B compare the breakdown by diagnostic category of the statewide and MHP

number of beneficiaries served and total approved claims amount, respectively, for CY16.

MHP self-reported percent of consumers served with co-occurring (substance abuse and mental

health) diagnoses: 6.0percent.

- 25 -


Performance Measures Findings—Impact and Implications

Access to Care

• Overall penetration rates are slightly above the rates for other large counties and the

state average.

• By race, the White Medi-Cal enrollees are 13 percent of the total enrolled population,

and 21.9 percent of beneficiaries.

• Asian Pacific Islanders are 35 percent of the enrolled population, and only 19 percent of

the beneficiaries.

• Penetration rates for children ages 0-5 and youth ages 6-17 are significantly above large

county and state averages.

• Statewide, high-cost beneficiaries (HCB) comprise 3.12 percent of the beneficiaries

served. For the MHP these comprise 10.31 percent of the beneficiaries served.

• For CY16, HCBs account for 45.85 percent of approved Medi-Cal claims in Santa Clara

County while statewide HCBs account for 28.90 percent of approved claims.

• By race or ethnicity, for all categories of beneficiaries, the percentage of HCBs served is

at least 2.2 times the statewide figure. For African-Americans, it is 3.1 times the

statewide percentage. For Hispanics, the percentage of HCBs in Santa Clara is 5.1 times

the statewide percentage for Hispanics.

• Hispanics are 38 percent of the beneficiaries served, but the account for 46 percent of

the MHP’s HCB approved claims.

• By age group, the percentage of HCB served is 6.3 times the statewide percentage for

the 0-5 age group and decreases steadily as age increases until it reaches 1.5 times the

statewide percentage for the 60 years and older group.


• While the 7-day outpatient follow-up rate post-hospitalization has improved from about

30 percent in CY15 to more than 40 percent in CY16, and is above the state average rate,

the rehospitalization rate remains above the State average rehospitalization rate.

• The 30-day outpatient follow-up rate post-hospitalization follows the same pattern as

the 7-day outpatient follow-up rate post-hospitalization and the rehospitalization rate

again remains higher than the state average.

- 26 -


Quality of Care

• Looking at the entire beneficiary population in Santa Clara County, average claims per

beneficiary (ACB) is more than twice what it is in other large counties and the statewide

average; and ACB has increased significantly in CY16 from both CY14 and CY15.

• The disparity between large county and the statewide ACB is greatest for children ages

0-5 and youth ages 6-17.

• The ACB for Hispanic and African-American beneficiaries is significantly higher than

other racial and ethnic groups.

• Based on eligibility category, approved claims per beneficiary range for three cost

categories: under $20,000; $20,000 to $30,000, and those above $30,000; it is the three

categories specific to children that have the highest ACB, with foster care having the

highest ACB by a wide margin.

• By service category, there are only two categories with ACB above $10,000 for large

counties and statewide: TBS and Day Treatment. For the MHP, four service categories

have ACB above $11,000: Inpatient ($15,580), Residential Services ($11,280), Mental

Health Services ($10,566), and TBS ($11,465).

• Beneficiaries receiving more than 15 services make up 39.57 percent of the statewide

beneficiary population; in Santa Clara, 51.29 percent of beneficiaries receive more than

15 services.

Consumer Outcomes

• Both 7-day and 30-day rehospitalization rates increased in CY16 from their comparative

CY15 rates and slightly exceed statewide averages.

- 27 -


PERFORMANCE IMPROVEMENT PROJECT

VALIDATION A Performance Improvement Project (PIP) is defined by CMS as “a project designed to assess and

improve processes and outcomes of care that is designed, conducted, and reported in a

methodologically sound manner.” The Validating Performance Improvement Projects Protocol

specifies that the EQRO validate two PIPs at each MHP that have been initiated, are underway, were

completed during the reporting year, or some combination of these three stages. DHCS elected to

examine projects that were underway during the preceding calendar year.

Santa Clara MHP PIPs Identified for Validation

Each MHP is required to conduct two PIPs during the 12 months preceding the review. CalEQRO

reviewed and validated two MHP-submitted PIPs, as shown below.

Table 3 lists the findings for each section of the evaluation of the PIPs, as required by the PIP

Protocols: Validation of Performance Improvement Projects.5

Table 3: PIPs Submitted by Santa Clara MHP

PIPs for Validation

# of PIPs PIP Titles

Clinical PIP 1 Welcome to Treatment Group

Non-clinical PIP 1 Contract Renewal Process

Table 4, on the following page, provides the overall rating for each PIP, based on the ratings given to

the validation items: Met (M), Partially Met (PM), Not Met (NM), Not Applicable (NA), Unable to

Determine (UTD), or Not Rated (NR).

5 2012 Department of Health and Human Services, Centers for Medicare and Medicaid Service Protocol 3 Version 2.0,

September 2012. EQR Protocol 3: Validating Performance Improvement Projects.

- 28 -


Table 4: PIP Validation Review

Item Rating

Step PIP Section Validation Item Clinical Non-

clinical

1 Selected Study

Topics 1.1 Stakeholder input/multi-functional team M NR

1.2 Analysis of comprehensive aspects of enrollee needs, care, and services

M NR

1.3 Broad spectrum of key aspects of enrollee care and services M NR

1.4 All enrolled populations M NR

2 Study Question 2.1 Clearly stated M NR

3 Study 3.1 Clear definition of study population M NR

Population 3.2 Inclusion of the entire study population M NR

4 Study

Indicators 4.1 Objective, clearly defined, measurable indicators M NR

4.2 Changes in health status, functional status, enrollee satisfaction, or processes of care

M NR

5 Sampling Methods

5.1 Sampling technique specified true frequency, confidence interval and margin of error

NA NR

5.2 Valid sampling techniques that protected against bias were employed

NA NR

5.3 Sample contained sufficient number of enrollees NA NR

6 Data Collection 6.1 Clear specification of data M NR

Procedures 6.2 Clear specification of sources of data M NR

6.3 Systematic collection of reliable and valid data for the study population

M NR

6.4 Plan for consistent and accurate data collection M NR

6.5 Prospective data analysis plan including contingencies M NR

6.6 Qualified data collection personnel M NR

7 Assess

Improvement Strategies

7.1 Reasonable interventions were undertaken to address causes/barriers

M NR

8 Review Data Analysis and

8.1 Analysis of findings performed according to data analysis plan M NR

Interpretation

of Study Results 8.2 PIP results and findings presented clearly and accurately M NR

8.3 Threats to comparability, internal and external validity PM NR

8.4 Interpretation of results indicating the success of the PIP and follow-up

M NR

9 Validity of

Improvement 9.1 Consistent methodology throughout the study NA NR

9.2 Documented, quantitative improvement in processes or outcomes of care

NA NR

9.3 Improvement in performance linked to the PIP NA NR

9.4 Statistical evidence of true improvement NA NR

9.5 Sustained improvement demonstrated through repeated measures

NA NR

- 29 -


Table 5 provides a summary of the PIP validation review.

Table 5: PIP Validation Review Summary

Summary Totals for PIP Validation Clinical PIP Non-clinical

PIP

Number Met 19 0

Number Partially Met 1 0

Number Not Met 0 0

Number Applicable (AP)

(Maximum = 28 with Sampling; 25 without Sampling) 20 0

Overall PIP Rating ((#Met*2) + (#Partially Met))/(AP*2) 97.5 % 0 %

Clinical PIP—Welcome to Treatment Group

The MHP presented its study question for the clinical PIP as follows:

“Will the development and implementation of Welcome to Treatment at Downtown Behavioral

Health (DTBH) increase timely access to care from 18.1 days to 14 days by December 2017,

engagement, and overall consumer perception within specialty behavioral health outpatient

services?”

Date PIP began: December 2015

Status of PIP: Completed

The goal of the PIP was to engage and improve timely delivery of services to individuals waiting for

specialty behavioral health treatment at Downtown Behavioral Health.

The problem facing the Santa Clara System of Care was increased demand in relation to service

provision capabilities, thereby creating delays to services. Research provided showed that delays in

providing timely access to services for consumers seeking specialty mental health reduces quality

of care and poor consumer outcomes.

To address improvements to timeliness, the MHP focused primarily on its welcoming strategies to

immediately attend to consumer requests and to enhance engagement. It initiated several

strategies in its Welcome to Treatment Group.

The MHP identified activities such as these to accomplish its goals:

- 30 -


• To improve outcomes of care and support engagement in the treatment services, a

consumer attends a group session in which information is provided in a welcome packet

that consumers can utilize when the clinic is closed.

• To familiarize the consumer with the treatment environment, to encourage consumer

willingness to engage, and to reduce the no-show rates, a clinician and a peer support

worker review the welcome packet and provide other information regarding the available

wellness and recovery services.

• The clinician determines/inquires whether a need for medication refill prior to intake is

warranted and whether to schedule a medication evaluation date.

• Consumers are guided on the process of obtaining medication from Urgent Care if necessary

prior to psychiatrist’s evaluation.

• Face-to-face interactions between consumers and their treatment team are coordinated to

encourage engagement and treatment success. The clerical staff will schedule an intake

appointment during the group and the consumer meets with assigned clinician the same

day if possible.

• Consumers are given a tour of Zephyr, a self-help center which is located on the grounds of

DTBH.

• The client perception data developed by the DTBH is collected from consumers using the

Welcoming Group Satisfaction survey with a 5-point Likert scale (strongly disagree,

disagree, neutral, agree, and strongly agree).

Results were reported which reflect the following:

• Improved timeliness to access for consumers showed a 4 percent improvement for

average calendar days to initial services at DTBH.

• A 7 percent improvement for consumers receiving initial services within 14 days.

• No statistical evidence on the data analysis table on engagement was collected,

however, anecdotal information noted engagement between staff and consumers has

improved as evidenced by decreased no-show rates and increase in utilization of

services at the clinic.

• The survey showed 81.8 percent of consumers found the Welcoming Group to be useful

and 89.1 percent reported they were able to receive adequate information.

• Future activities will consider providing a short therapy group to enhance consumer

coping skills.

The results from this PIP may be generalizable to other locations and the MHP is expanding the

program to another county clinic and potentially two contract providers.

- 31 -


Relevant details of these issues and recommendations are included within the comments found in

the PIP validation tool.

The technical assistance provided to the MHP by CalEQRO consisted of discussing the value of

replicating this PIP in other settings. The value was emphasized and the unexpected results in the

ability to triage consumer needs was highlighted. The MHP determined this would conclude the

submission of this PIP and it would consider a new PIP for the next review cycle. Topics were not

discussed at this session. The MHP was encouraged to consult with EQRO early and often during

PIP formulations.

Non-clinical PIP—2018 Contract Renewal Process

The MHP presented its study question for the non-clinical PIP as follows:

“Will the implementation of the FY 2018 Contract Renewal Process Pilot Project, which includes the

following enhancements as listed below, improve timely execution of the county Short

Doyle/Mental Health Services Act (SD/MHSA) service contracts?

• The redesign of the Contracting Services Request (CSR) form to include only vital and

relevant information will help facilitate the approval process of the CSR in a timely manner.

• The implementation of the redesigned CSR form for SD/MHSA service contracts in digitized

format for use in DocuSign, a digital transaction management system that provides

electronic signature technology for facilitating electronic exchanges of signed documents,

will enable efficient tracking and review/approval of CSRs.

• The unbundling of service provider contracts into separate contracts and package by

program division: Family and Child, Adult and Older Adult, Criminal Justice Services,

Integrated Behavioral Health, and Supportive Housing Services specific program contracts

and expedite execution of contracts.

• Conducting various meetings related to the pilot project with all parties involved: pilot

project team, executive team, and CBO contract providers will ensure everyone knows the

goal of the pilot project and roles and responsibilities of each area.

• The creation a status tracking sheet to show current renewal status of all FY18 SD/MHSA

service contracts and making it available online/SharePoint site to all staff involved in the

pilot project so that everyone is aware and current on the status of contracts throughout the

renewal process.”

Date PIP began: February 2017

Status of PIP: Submission determined not to be a PIP (not rated)

- 32 -


The project’s aim, specific to SD/MHSA mental health service contracts only, was to pilot a new

approach to increase timely delivery of executed contracts, ensuring they are in place at the start of

the fiscal year for consumers.

It was determined that the non-clinical PIP did not meet the criteria for a PIP secondary to the lack

of consumer benefit. Although commendable efforts were done in the MHP’s process and business

procedures, there were no stated outcomes for the consumer.

This PIP does not reflect a data collection and analysis of comprehensive aspects of enrollee needs,

care and services. The improvements to the contract renewal procedures did prove successful,

however, it was deemed to be limited to the business practice change. This did not result in

producing outcomes that were consumer related. Services to consumers were not impacted if

contracts were not renewed timely. To qualify, the PIP has to be connected to consumer outcomes

and, as stated, this is a process change.

However, this PIP did reconstruct the method for timely completion of the MHP contract

agreements. The MHP managed to complete over 95 percent of its contracts with organizational

providers and introduced new efficiencies to do this. Discussion occurred on-site to emphasize

using EQRO for technical assistance early in its PIP discussions and throughout the development of

its PIPs.

In conclusion, this activity does not meet the requirements for a PIP; rather, it serves as more of a

practice improvement which focuses on contract renewal dates to validate effectiveness.

Relevant details of these issues and recommendations are included within the comments found in

the PIP validation tool for instructional purposes only.

The technical assistance provided to the MHP by CalEQRO consisted of discussion of this PIP and

the elements that would have helped in the PIP process. Topics were not discussed at the review

and TA was offered although it was not scheduled. The MHP was encouraged to consult with EQRO

early and often during PIP formulations and the MHP recognized this option.

PIP Findings—Impact and Implications

Access to Care

• The average calendar days for access to services was improved by 4 percent.

• Consumers were scheduled without delay for a Welcome to Treatment group upon

initial request.


• Timeliness to the initial service within the 14-day standard was improved by 7 percent.

- 33 -


• Earlier initial appointments can lead to effective engagement for consumer benefit.

Quality of Care

• A secondary benefit emerging from this PIP allowed staff to immediately triage

consumer needs, potentially lending to earlier symptom interventions.

• Improved outcomes for timely access to care coupled with consumer satisfaction led the

MHP to replicate this approach in other clinic settings.

Consumer Outcomes

• Consumer satisfaction was reported at an improved rate and led to consideration of a

treatment group.

• Replicating this approach provides expanded quality for consumers.

PERFORMANCE AND QUALITY MANAGEMENT

KEY COMPONENTS CalEQRO emphasizes the MHP’s use of data to promote quality and improve performance.

Components widely recognized as critical to successful performance management include an

organizational culture with focused leadership and strong stakeholder involvement, effective use of

data to drive quality management, a comprehensive service delivery system, and workforce

development strategies that support system needs. These are discussed below, along with their

quality rating of Met (M), Partially Met (PM), or Not Met (NM).

Access to Care

Table 6 lists the components that CalEQRO considers representative of a broad service delivery

system that provides access to consumers and family members. An examination of capacity,

penetration rates, cultural competency, integration, and collaboration of services with other

providers forms the foundation of access to and delivery of quality services.

Table 6: Access to Care Components

Component Quality Rating

1A Service accessibility and availability are reflective of cultural competence principles and practices

M

- 34 -


The MHP strived to increase the penetration rate for services delivered with a stated goal of at

least one percentage point for each ethnic group. The MHP collects, reports, and reviews data

from regions, program level, age groups, and racial/ethnic groups to reflect culturally informed

activities.

The MHP has assumed the lead role in orchestrating WPC initiative with behavioral health

integrated activities. A group of 40-50 peer support workers convened with a kick-off for the

development of peer respite site that will launch later this year.

The Santa Clara Early Child Development Training Academy demonstrated the MHP’s

commitment to early years’ treatment for the 0-5 age group. Clinical staff attend monthly

seminars and use reflective supervision principles to apply this knowledge in their practice.

1B Manages and adapts its capacity to meet consumer service needs PM

The MHP screens all foster youth for mental health symptoms of which 1700 screenings were

completed that included the Katie A. subclass. Since February 2016 the MHP opened Katie A.

services ICC and IHBS to any youth that met medical necessity with full scope Medi-Cal. County

agencies use the same screening form and referrals can be made from child welfare, probation,

and community services.

The MHP designed an electronic capacity tracking tool to improve appointment availability

management at the Access Call Center, worked with rapid cycle teams and contracted with the

MTM Consulting firm to implement improvement ideas.

Recent contracts with Harbage Consulting will provide a new foundation for service delivery.

The MHP is moving from its “no wrong door” approach to a more centralized access system with

same day treatment as the goal.

The MHP continued work from its prior PIP focused on consumer engagement by defining the

current performance indicator for engagement to include at least one face-to-face visit within 30

days following hospitalization.

The MHP leadership was well-aware of network adequacy as its biggest challenge. The local area

poses high cost housing and a shortage of qualified professionals in community mental health

programs that must compete with lucrative private sector health care systems. Employee

retention at the contract providers is a continuing challenge in this well-funded system.

1C Integration and/or collaboration with community-based services to improve access

M

A partnership with Stanford University led to psychiatry residents working with the jail inmates.

The Valley Health system intends to use peer support workers at its call center and integrate the

systems for a coordinated and seamless entry to services.

With over 80 percent of its services delivered under contract, the MHP continues to engage with

its broad-based contract providers. Having implemented performance indicators at its

- 35 -


contracted providers, the MHP’s Pay for Success program completed the first year with positive

data and extended this incentive program for five years.

The MHP initiated monthly Clinical Record meetings to review data dashboards with

stakeholders for compliance with standards of care and review the State all-county letters and

discuss the impact.


As shown in Table 7, CalEQRO identifies the following components as necessary to support a full-

service delivery system that provides timely access to mental health services. This ensures

successful engagement with consumers and family members and can improve overall outcomes,

while moving beneficiaries throughout the system of care to full recovery.

Table 7: Timeliness of Services Components


2A Tracks and trends access data from initial contact to first appointment PM

The MHP tracks and reports the time from initial contact to first kept clinical appointment. The

MHP’s standard is 14 days, with most recent data indicating a mean of 14.7 days for adult

services, and 15.9 days for children and youth. Attainment of standard is 62.5 percent for adults

and 56.7 percent for children and youth.

The MHP has initiated multiple improvements focused on centralizing its access points with the

intention of increasing positive results for this metric yet it continues to struggle with

scheduling of initial appointments.

2B Tracks and trends access data from initial contact to first psychiatric appointment

M

The MHP utilizes a 30-day initial psychiatry contact standard. The adult services mean is 28.2

days, with 33.1 days the children and youth result. Attainment of standard is 54 percent for

adults and 44.7 percent for children and youth.

The MHP may wish to target initial psychiatry services with an improvement activity if the

interval between initial contact to first appointment continues to remain low in the achievement

of its standard.

2C Tracks and trends access data for timely appointments for urgent conditions

M

- 36 -


The MHP maintains a one-day standard for response to urgent conditions. It reports 100 percent

achievement of this for both the adult and children’s services. Given this is a large size county,

this achievement warrants recognition.

2D Tracks and trends timely access to follow-up appointments after hospitalization

M

The MHP tracks and reports a standard to follow-up post hospitalization of seven days. Of the

total 1,850 hospitalizations, 1,359 were adults and 491 were children. The adult services mean

is 7.3 days and for children it reports 5.3 days. Achievement of this is reported at 41.6 percent

for adults and 68.8 percent for children.

2E Tracks and trends data on rehospitalizations M

There were 216 rehospitalizations within 30 days of discharge from inpatient care. The MHP

reported a rehospitalization rate of 12.2 percent for adult services and 10.2 percent for children.

The data is inclusive of all hospital psychiatric admissions.

2F Tracks and trends no-shows M

The MHP tracks its no-show data for clinical staff and psychiatrists with a goal not to exceed 25

percent. The adult services indicated an 18.8 percent no-show rate for psychiatrists and a 4.5

percent no-show rate for clinicians. Children’s services indicated a 12.9 percent no-show rate for

psychiatrists and a 5.4 percent rate no-show rate for clinicians.

As discussed with the MHP on-site, it may benefit the MHP to revisit the 25 percent standard to

evaluate whether it aligns with its goals and is comparable to the no-show standard in other

similar-sized counties. Given the demand for psychiatric service, the no-show rates potentially

reflect expensive unused capacity.

Quality of Care

In Table 8, CalEQRO identifies the components of an organization that is dedicated to the overall

quality of care. Effective quality improvement activities and data-driven decision making require

strong collaboration among staff (including consumer/family member staff), working in

information systems, data analysis, clinical care, executive management, and program leadership.

Technology infrastructure, effective business processes, and staff skills in extracting and utilizing

data for analysis must be present in order to demonstrate that analytic findings are used to ensure

overall quality of the service delivery system and organizational operations.

- 37 -


Table 8: Quality of Care Components


3A Quality management and performance improvement are organizational priorities

M

The planning from its recent contracts with MTM Consulting and Harbage Consulting will guide

system revisions and improved efficiency. This will be the foundation for improvements in care

and delivery system quality. The goal is to achieve National Council of Quality Assurance (NCQA)

accreditation, a nationally recognized leader with a commitment to comprehensive research and

dedication to quality in healthcare.

The MHP created and filled its new Quality Management Director position. The MHP provided an

evaluation of the prior year and a current Quality Improvement Work Plan (QIWP). The MHP

had the foresight to include an infrastructure improvement plan to support its efforts. The

Quality Management redesign is underway to reflect the Quality Circle structure, which supports

system integration and an increased focus on providing value to its beneficiaries.

The QIWP had measurable, quantifiable goals focused on consumer outcomes. For example, the

focus of QI Grand Rounds in June and November 2017 was to increase client engagement with

goals focused on improved timely access.

The MHP created the Practice Standards Manual, intended to build a bridge between

documentation regulations and recovery-oriented best clinical practices. Monthly QI meetings

with all providers occur to discuss State notices and practice guidelines. This is a model of

developing the “why” regarding documentation with best practice and recovery orientation

thus, integrating Quality Improvement and Quality Assurance activities.

Among the MHP’s current initiatives, the implementation of its EHR system and conversion to

electronic data interchange with its contract providers, there are serious concerns about the

viability of its very aggressive schedule in the face of potentially significant constraints in the

technologically-skilled and experienced resources to support and sustain the implementation. .

Although this falls short for its quality care, the MHP has stated it has taken measures since the

review, these were not in place at the on-site review. CalEQRO anticipates evaluating these

activities at the next review cycle. This is a vital component of the continued quality care system.

3B Data are used to inform management and guide decisions M

Data reporting is extending to various components of the work force within the system of care.

The Data Decision team initiated a dashboard for adult and older adults. It continues to track

discharge success via the Milestones of Recovery Scale (MORS) and produces reporting for this

for all providers.

- 38 -




Data are published as a web link to provider staff. Training is planned for February 2018. This

process lines up with the quality management goals, streamlines the access to data, and allows

for stakeholders to manipulate the data to suit their needs.

With the new EHR system, all providers will enter data into Epic HealthLink. The data entered

into Epic HealthLink from county mental health and SUD providers will be transmitted

electronically to Netsmart myAvatar system for claims processing. Contract providers will

transmit their claiming data electronically from their local EHR to Netsmart myAvatar.

3C Evidence of effective communication from MHP administration, and stakeholder input and involvement on system planning and implementation

\M

Leadership established a comprehensive QI staff training program and opportunities to share

best practices to infuse quality care throughout the system.

The MHP combined the QI division and the Decision Support division and established its Quality

Circles which encouraged a unified cross-functional work style.

The MHP has made great efforts to communicate with staff in the past two years. This includes:

monthly staff bulletins, monthly all manager meetings, bi-annual all staff meetings, a staff-led

communication work group and continues to elicit stakeholder feedback for MHSA planning.

These are two-way exchanges of information that provide staff with an opportunity to suggest

ideas such as developing guidelines for e-mail reduction, track the number of meeting staff are

required to attend and develop a calendar of events important to the MHP. Each of these ideas is

being followed up and implemented at the time of the review.

The addition of the four new executive mangers has increased the visibility and staff access to

executive level decision making. Overall, these efforts improved transparency significantly with

staff being informed of changes, promotional opportunities, fiscal and political issues. In

addition, staff have been invited to participate in a number of improvement activities designed

to enhance their working conditions and increase job satisfaction.

The MHP contracted with Resource Development Associates (RDA) to conduct the MHSA System

Evaluation and conducting surveys/focus groups which included participation from CBO

partners/contract providers.

Contrasted against the cohesion developed among the executive team, the staff, family members,

and contract providers indicated information flow continues to be primarily one directional.

This may be an unintended result of multiple projects simultaneously absorbing the executive

team’s time. Stakeholder perception may be slow to feel the effects of the value leadership

- 39 -




places on its input. To its credit, the MHP leadership values its stakeholder relationships on

multiple levels and its efforts will continue to permeate through the system although this takes

sustained effort and time which is the MHP’s commitment. CalEQRO anticipates witnessing this

infusion at the next review.

3D Evidence of a systematic clinical continuum of care PM

The MHP continues to expand its service components with the SUTS, WPC initiative and its

Integrated Behavioral Health consolidation.

Enhanced efforts to foster integration of mental health and substance use treatment services are

evolving with its Practice Standards manual and QI workgroup addressing both units. A

documentation manual incorporated both requirements and continually emphasized the Golden

Thread linking assessment, diagnosis, treatment and documentation.

The family members and providers felt transitions in service were increasingly difficult citing

system barriers for providers to coordinate with one another. The parent focus group noted

transitions repeatedly resulted in large service gaps leading to significant challenges for both the

youth and family.

Most surprising is the continued challenge of establishing a bona fide clinical supervisory

position. Current practice accounts for clinical supervision yet it is an assigned responsibility

for licensed clinical staff who assume a lead role for their colleagues training. The current

practice also means that a supervisory assignment is not qualifying experience when applying

for management positions. This effectively negates opportunities within the career ladder for

some of the MHP’s most experienced clinicians and unintentionally contributes to feeling

undervalued.

Leadership values its assigned clinical supervisor employees and continues to advocate for

dedicated clinical supervisor positions, yet it appears the final decision lies beyond their direct

control. This is a situation atypical to other comparable MHPs and potentially undermines the

MHPs effectiveness. Several meetings have been held with the county Employee Services Agency

(ESA) deputy’s executive staff. These meetings have been received positively and the MHP has

been assured that the creation of a new supervisory position will be given top priority. This

specific situation is broader than the department, as similar codes supervisory codes are used by

other county departments. In addition, ESA is conducting a county-wide classification study of

the Senior Healthcare Program Manager classification, to review roles and functions and

consider impaction with other supervisory and management positions. CalEQRO anticipates a

status update at the next review cycle.

- 40 -




3E Evidence of consumer and family member employment in key roles throughout the system

PM

The MHP created a new position entitled Consumer and Family Affairs Division Director with an

open recruitment in process. The current mental health Peer Support Workers play a vital role

in many services especially within the contract providers. These staff have gained experience in

coordinating care, working with clients and providing home-linked services.

Consumers expressed regret that a clear career ladder with promotional opportunities were

limited. It may benefit the MHP for mental health Peer Support Workers to have an opportunity

to compete promotionally for entry level mental health worker positions. With the initiation of

the new Consumer and Family Affairs Division Director, the MHP could be taking the next steps

to a clear career path for the consumer employee track.

3F Consumer run and/or consumer driven programs exist to enhance wellness and recovery

PM

The wellness centers Zephyr and Narvaez Self-Help Centers are consumer-run recovery-

oriented programs. The hours are limited to Tuesday to Friday from 9:00 a.m. to 3:30 p.m. These

sites averaged 40 persons daily. The new welcoming process includes material about the Self-

Help Centers and often a visit to a Self-Help Center is conducted with a Peer Support Worker.

Given there exist two Self-Help Centers with limited hours, consumers and staff shared that an

additional center could benefit consumers in areas that do not yet have a wellness center.

Consideration for the addition a third center might bring increased attendance in other regions.

The MHP improved access to treatment by providing an alternative step-down level of care for

adults with its crisis residential treatment facility.

The MHP developed and promoted a Wellness Level of Care across it community-based

organizational providers.

3G Measures clinical and/or functional outcomes of consumers served M

Consumer focus groups reported that all persons were engaged in their own treatment planning.

Improvement goals are noted in the QIWP for successful outcomes. For example, the MHP noted

its goal to increase the number of adult clients who successfully discharge, as measured by

outcome scores at discharge of at least 35 percent and it supports Family and Child Services to

maintain its level at a minimum of 58 percent.

Several focus groups among staff, providers, and family members indicated that the system

appears impacted with quality affected due to the efforts to meet consumer demand within the

- 41 -




limits of available resources. This was evidenced by increased time to initial screenings and

large clinician caseloads potentially impacting the care provided to consumers.

Continued use of the MORS and Child and Adolescent Needs and Strengths (CANS) report scores

assist in the level of care decisions for consumers. Other instruments across the behavioral

health system are used such as the Patient Health Questionnaire 9 (PHQ-9) to monitor and

assess the severity of depressive symptoms, and the screening for anxiety with the Generalized

Anxiety Disorders (GAD) tool.

3H Utilizes information from Consumer Satisfaction Surveys M

The MHP solicits feedback on the statewide Consumer Perception Survey and conducts this

twice a year for a two-week period.

Various specialized surveys are distributed based on the program needs. The MHP developed a

Peer Support Satisfaction Survey, which collected information from clients on their satisfaction

with peer support services.

The data is published by the Decision Support team and shared among stakeholders.

Key Components Findings—Impact and Implications

Access to Care

• Access to appointments has been improved by the implementation of the CISCO call

center phone system at the Access Call Center. It is the same system used by the SUTS

Gateway Center. The two systems are linked for information sharing purposes which

facilitates transfers between the two organizational units when needed. Call response

times and accurate call data capture for service analysis have both improved.

• The MHP extended its continuum of care with the addition of two adult community-

based crisis residential facilities, deployment of mobile crisis teams, and the opening of

a community-based crisis stabilization unit.

• The MHP has contracted with MTM Consulting, a nationally recognized behavioral

health consulting group, with expertise in call center operations and same day

appointment access.

• The MHP has also contracted with Harbage Consulting. It plans to use this service to

assess organizational performance and upgrade communication, work efficiency, and

- 42 -


decision-making processes and align these with its mission and philosophy to meet

capacity.

• There are reports and some evidence of the MHP lacking sufficient capacity to meet

service demand. Anecdotal reports of an adult waiting list for initial appointments

reaching up to hundreds of beneficiaries are an area of concern.


• The MHP developed an alert tracking and data management system to flag when a client

has been admitted to hospitals for a smoother transition of care from inpatient to

outpatient care.

• Timely service delivery was expanded with additional services for urgent response time

with the CSU and the mobile crisis teams.

• Screening all foster youth for mental health symptoms in a timely manner using a

collaborative approach with all stakeholders reflects the commitment to this target

group.

Quality of Care

• The MHP developed its Integrated Organizational Structure combining multiple

departments for efficiencies in service delivery.

• The MHP successfully recruited and filled four new executive positions for

Administrative Services, Adult/Older Adult Services, Children, Youth and Family

Services, and Quality Management.

• After selecting Epic HealthLink and Netsmart myAvatar as the County electronic health

record, the MHP launched its initiative to establish electronic data exchange among

contract providers.

• The MHP continued to develop system and operational dashboards to ensure

compliance with the CMS regulatory standards and to track and analyze data for

continuous quality improvement.

• Through its Quality Circles, the MHP intends to restructure components of the Quality

Management Committee to help build the capacity of staff and providers to initiate

quality improvement actions.

• The MHP Golden Thread treatment process starts with engaging the client so that an

assessment can be conducted that leads to formulating a treatment plan for services and

documenting progress towards treatment goals.

• A train-the-trainer workshop in collaboration with Public Health was offered to build

capacity of staff in the use of quality management tools and methods.

- 43 -


• The turnover rate for clinicians in the contract provider network raises concerns about

timeliness and quality of care. A high staff vacancy rate directly impacts accessibility

and timeliness of services.

• Substantial improvements have been made to provide consistent messages and

communication efforts regarding documentation standards. The Quality Assurance

Work Group established a forum for raising and discussing documentation issues with

staff and contract providers. The agenda, decision-making, and minutes are shared from

these activities. Stakeholders are encouraged to be present.

• Dispersing information to all stakeholders simultaneously may be indicated with the

intent to strengthen the link between leadership, contract providers and other

stakeholder groups.

Consumer Outcomes

• Dedication of resources to consumers was reflected with the creation of the Consumer

and Family Affairs Division Director position.

• Review of the transitions between providers to determine if successful discharge

procedures may warrant improvements.

• The MHP hired a full-time hospital liaison for adult services that is designated to track

client admissions and assist with a warm hand-off to outpatient clinics.

• Continued advocacy for Peer Support Workers to have an opportunity to promotionally

compete for entry level mental health worker positions may be indicated. Continued

advocacy for Peer Support Workers to have positions available within the county and to

have an opportunity to compete for entry level county mental health worker positions

would provide increased options.

• Consumer run self-help wellness centers have limited hours in comparison to the MHP

business hours. Zephyr is open Tuesday thru Friday from 9 a.m. to 3 p.m. The MHP has

indicated it has added additional resources to ensure current operational hours are met.

• The Self-Help Centers are important community wellness resources that can only

contribute to the extent they are properly staffed and resourced. The MHP has

indicated staffing has been added to support the operations.

• Consumers receive a quantity of services as demonstrated by beneficiary retention rates

showing more than 50 percent of beneficiaries receiving more than 15 services and

average claims per beneficiary (ACB) in some categories more than twice the large

county or State averages.

• It is not clear if these higher retention rates and ACBs are leading to better client

outcomes. The slightly higher rehospitalization rates in Santa Clara compared to the

- 44 -


statewide rates leave open the question of whether the higher costs are delivering the

desired results.

• It is unclear at this point whether this is evidence of an appropriate level of care for the

population served or an indicator that consumers are being retained in treatment

because of limited resources to refer to at the next lower level of care. Staffing capacity

potentially affects new consumers’ access to on-going treatment. This is a critical

question that requires attention.

- 45 -


CONSUMER AND FAMILY MEMBER

FOCUS GROUPS

CalEQRO conducted two 90-minute focus groups with consumers and family members during the

site review of the MHP. As part of the pre-site planning process, CalEQRO requested focus groups

with 8 to 10 participants each, the details of which can be found in each section below.

The consumer/family member focus group is an important component of the CalEQRO site review

process. Obtaining feedback from those who are receiving services provides significant information

regarding quality, access, timeliness, and outcomes. The focus group questions are specific to the

MHP being reviewed and emphasize the availability of timely access to care, recovery, peer support,

cultural competence, improved outcomes, and consumer and family member involvement. CalEQRO

provides gift certificates to thank the consumers and family members for their participation.

Consumer/Family Member Focus Group 1

• A culturally diverse group of adult beneficiaries with a mix of new consumers who have

initiated/utilized services within the past 12 months.

• This group was held at the MHP Administrative offices located at 1075 E Santa Clara Street,

San Jose.

Number of participants: 6

There were no participants who entered services within the past year.

Participants described their experience as the following:

• This group of consumers was very enthusiastic about the services they were receiving.

Consumers stated accessing services was straightforward and supportive. All described

being involved in their treatment planning.

• Participants received multiple services including medications support, counseling, and

case management.

• The majority of the participants had sessions with a psychiatrist either monthly or

every other month and a therapist on a regular basis with either twice monthly or every

three weeks. Most were also involved in group therapy and appreciated the group

process. They reported feeling support from the therapist as well as from their peers in

the group.

• In addition, all consumers had a case manager and were most enthused about the

support they gave to consumers. They described receiving assistance with housing,

- 46 -


transportation, paperwork, appointment reminders, food, and life-skills. As no one was

aware of services to assist them with either education or employment, they felt that this

resource might be helpful for them.

• Regular communication and home visits occurred with the case managers. Several

attended peer support groups and found them helpful. Group members mentioned that

having an ethnically matched case manager increased trust.

• No one had used the crisis service in the last year; however, all would recommend using

this service when necessary.

General comments regarding service delivery that were mentioned included the following:

• The consensus was services were consistently available and scheduled on a regular

basis.

• Staff provided support and indicated recovery was attainable.

• All felt that they had improved as a result of the services received.

Recommendations for improving care included the following:

• Increase availability of the group therapy sessions.

• Navigators would be useful at the onset of treatment to assist consumers in building

confidence in using the system services.

• Improve timeliness for the initial psychiatry appointment.

Interpreter used for focus group 1: No

Consumer/Family Member Focus Group 2

• A culturally diverse group of parents/caregivers of child/youth beneficiaries with a mix of

new consumers who have initiated/utilized services within the past 12 months.

• This group was held at the Kidscope Center located at 828 Bascom Avenue, San Jose.

Number of participants: 15

The participants who entered services within the past year described their experiences as the

following:

• Psychiatry services were provided seamlessly for the majority of participants.

• Initial entry into services was delayed and often confusing as to which service would be

provided.

- 47 -


General comments regarding service delivery that were mentioned included the following:

• Participants experienced different requirements and individual methods of paperwork

for each agency at initial entry.

• Group participants voiced concern about transfers from one agency to another

mentioning this resulting in undue delays in treatment.

• Treatment was thought to be provided as a package and consumers suggested this

potentially ended treatment prematurely due to limited symptom reduction.

Recommendations for improving care included the following:

• Provide standardized paperwork and procedures among the various agencies providing

care.

• Transition to a lower level of care gradually.

• Provide bridging to community referrals for support once treatment concludes.

• Review procedures for discharging consumers as youth may be inadvertently impacted.

Interpreter used for focus group 2: Yes Language(s): Spanish

Consumer/Family Member Focus Group Findings—

Implications

Access to Care

• The majority of participants indicated access to services was readily obtained.

• Transferring services often resulted in delays according to family members.

• Once service was obtained, case management was available.


• Most of the participants received services in a timely manner and were served

consistently.

• Some experienced delays in change of providers.

Quality of Care

• Participants indicated staff were supportive and recognized recovery was possible.

- 48 -


• The majority indicated involvement in their treatment plans.

• Treatment was perceived to end prematurely for some participants.

• Entry paperwork varied among contract providers.

Consumer Outcomes

• Some participants were aware of a consumer survey and have submitted these in the

past.

• Participants were either involved in their own treatment plan or family members were

included.

• Group therapy for adults, often co-led by peers appeared to be beneficial.

- 49 -


INFORMATION SYSTEMS REVIEW Understanding an MHP’s information system’s capabilities is essential to evaluating its capacity to

manage the health care of its beneficiaries. CalEQRO used the written response to standard

questions posed in the California-specific ISCA, additional documents submitted by the MHP, and

information gathered in interviews to complete the information systems evaluation.

Key Information Systems Capabilities Assessment (ISCA)

Information Provided by the MHP

The following information is self-reported by the MHP through the ISCA and/or the site review.

Table 9 shows the percentage of services provided by type of service provider.

Table 9: Distribution of Services, by Type of Provider

Type of Provider Distribution

County-operated/staffed clinics 11 %

Contract providers 88 %

Network providers 1 %

Total 100 %

Percentage of total annual MHP budget dedicated to supporting information technology operations

(includes hardware, network, software license, IT staff): 3.73 percent

The budget determination process for information system operations is:

MHP currently provides services to consumers using a telepsychiatry application:

☐ Yes ☒ No ☐ In pilot phase

☒ Under MHP control

☐ Allocated to or managed by another County department

☐ Combination of MHP control and another County department or Agency

- 50 -


Summary of Technology and Data Analytical Staffing

MHP self-reported technology staff changes (Full-time Equivalent [FTE]) since the previous

CalEQRO review are shown in Table 10.

MHP self-reported data analytical staff changes (in FTEs) that occurred since the previous CalEQRO

review are shown in Table 11.

The following should be noted with regard to the above information:

• Table 10 shows only FTEs directly employed by the MHP. The MHP also receives

information technology resources from the Health and Hospital System and Santa Clara

County IT, as well at vendors and consultants.

• The Epic HealthLink and Netsmart myAvatar implementation plan for simultaneous

implementations are ambitious undertakings given the level of available technical and

project management staffing in the MHP.

Current Operations

• The MHP makes extensive use of data to inform decision-making and in the last year has

begun producing information dashboards that are informative and widely available.

• The budget devoted to information systems is reported as 3.73 percent. This number

includes resources in the Health and Hospital System not directly under the control of

the MHP but is supporting its operations.

Table 10: Technology Staff

IS FTEs (Include Employees

and Contractors)

# of New FTEs

# Employees / Contractors Retired,

Transferred, Terminated

Current # Unfilled Positions

7 0 1 1

Table 11: Data Analytical Staff

IS FTEs (Include Employees

and Contractors)

# of New FTEs

# Employees / Contractors Retired,

Transferred, Terminated

Current # Unfilled Positions

2 2 0 0

- 51 -


Table 12 lists the primary systems and applications the MHP uses to conduct business and manage

operations. These systems support data collection and storage, provide electronic health record

(EHR) functionality, produce Short-Doyle/Medi-Cal (SD/MC) and other third-party claims, track

revenue, perform managed care activities, and provide information for analyses and reporting.

Table 12: Primary EHR Systems/Applications

System/Application Function Vendor/Supplier Years Used

Operated By

Pro-Filer

Service recording, State reporting, billing, capacity management, assessment, referral, managed care

Co-Centrix 13 BHS IS

Diamond

Managed care service recording and billing

Dell 20 BHS IS

Epic HealthLink

EHR for BAP, EPS and BHS programs in FQHC clinics

Epic <1 HHS IS

Priorities for the Coming Year

• Implementation of Epic HealthLink clinical functions for MHP county-operated sites

(screening, scheduling, registration, clinical documentation, charge capture).

• Implementation of the Netsmart Practice Management for County and contract

providers (billing, claiming, reporting, QI and related functions).

• Preparing to initiate electronic exchange of service and billing data between contract

providers EHR systems and Netsmart myAvatar, effective July 2018.

Major Changes Since Prior Year

• Began the build for the implementation of the Epic HealthLink system.

• Began the build for the implementation of the Netsmart myAvatar system.

- 52 -


• Upgraded Cocentrix Pro-Filer from release 14 to release 15.

• Supported two Pay-For-Success projects, one called Project Welcome Home, an

innovative partnership approach to housing the homeless, and the other was to reduce

use of emergency psychiatric services and acute care services.

Other Significant Issues

• No currently active telepsychiatry implementation; still in the investigation stage.

• MHP self-reported co-occurring disorders rate is 6 percent, which is exceptionally low.

• High Cost Beneficiary (HCB) spending in some beneficiary groups reaches five or six

times statewide HCB percentages for those groups. This raises the question whether the

level of spending in these categories is driving outcomes that justify that level of

spending and whether that focus of spending is constraining spending in other areas of

need.

• MHP requested 4 additional FTEs to support Netsmart myAvatar and the contract

providers’ use of electronic data exchange with Netsmart myAvatar on a continuing

basis after the implementation is complete. This may prove to be a conservative request

although the MHP asserts it considers this adequate and will monitor staffing patterns

and adjust accordingly.”

• MHP requested six additional FTEs for continuing support of Epic HealthLink after the

implementation is complete.

• Netsmart is doing much of the work on the myAvatar build and data migration. Once the

implementation is complete, the vendor will be much less involved, leaving support and

maintenance to very limited local resources currently.

• The use of the integrated scheduling functionality in Epic HealthLink will be a huge

culture change for clinicians who are accustomed to having direct control of their own

appointment schedule in the current systems environment. This is a culture change that

is important to the success of Epic HealthLink implementation and, therefore, it needs to

be carefully managed. Clinical leadership needs to be especially clear and consistent in

their communication on this subject.

• The MHP stated when they move contract providers to the electronic data exchange

with Netsmart myAvatar, a steering committee previously used will be revived. Given

the stated target date of July 2018, to begin EDI with the contract providers, the revival

of that steering committee is overdue, and it should be engaged with all possible haste.

• At present, the contract provider’s awareness of this project seems limited to the broad

concepts and the target date. There does not seem to be an awareness of what

transactions will be exchanged, what exactly the contract providers and their vendors

- 53 -


need to do, in what order, and when they need to do it to make a coordinated transition

to electronic data exchange with Netsmart myAvatar.

• MHP staff described a detailed communications plan for this project yet it remains

unclear that the right information at the right level of detail is reaching the stakeholders

most responsible for making this a successful project.

Plans for Information Systems Change

• Currently implementing Epic HealthLink for clinical functionality.

• Currently implementing Netsmart myAvatar for billing and State reporting mandates.

Current Electronic Health Record Status

Table 13 summarizes the ratings given to the MHP for EHR functionality.

Table 13: EHR Functionality

Rating

Function System/Application Present Partially Present

Not Present

Not Rated

Alerts Epic (FQHC only) x

Assessments Pro-filer x

Care Coordination x

Document imaging/storage Pro-filer x

Electronic signature—consumer

x

Laboratory results (eLab) Epic (FQHC only) x

Level of Care/Level of Service

x

Outcomes x

Prescriptions (eRx) Epic (FQHC only) x

Progress notes Pro-filer x

Referral Management Pro-filer x

Treatment plans Pro-filer x

Summary Totals for EHR Functionality: 5 3 0 4

Progress and issues associated with implementing an electronic health record over the past year

are discussed below:

• No significant functional improvements were implemented to the legacy EHR (Pro-filer)

system during the past year.

- 54 -


Consumer’s Chart of Record for county-operated programs (self-reported by MHP):

☐ Paper ☐ Electronic ☒ Combination

Personal Health Record

Do consumers have online access to their health records either through a Personal Health Record

(PHR) feature provided within the EHR, consumer portal, or third-party PHR?

☐ Yes ☒ No

If no, provide the expected implementation timeline.

☒ Within 6 months ☐ Within the next year

☐ Within the next two years ☐ Longer than 2 years

Medi-Cal Claims Processing

MHP performs end-to-end (837/835) claim transaction reconciliations:

If yes, product or application:

Pro-Filer Version 15.3.2

Method used to submit Medicare Part B claims:

☐ Paper ☐ Electronic ☒ Clearinghouse

Table 14 summarizes the MHP’s SDMC claims.

Table 15 summarizes the most frequently cited reasons for claim denial.

Number

Submitted

Gross Dollars

Billed

Number

Denied

Dollars

Denied

Percent

Denied

Gross Dollars

Adjudicated

Claim

Adjustments

Gross Dollars

Approved

706,691 $508,351,045 26,105 $20,659,099 4.06% $487,691,946 $221,389,285 $266,302,661

Table 14: Santa Clara MHP Summary of CY16 Short Doyle/Medi-Cal Claims

Includes services provided during CY16 with the most recent DHCS process ing date of May 19, 2017.

The statewide average denia l rate for CY2016 was 4.48 percent.

Change to the FFP reimbursement percentage for ACA a id codes delayed a l l cla im payments between the months of January-May 2017.

☒ Yes ☐ No

- 55 -


• Denied claim transactions listed with the denial reason description as “Missing,

incomplete, invalid ICD-10 diagnosis or condition” are generally re-billable within the

State claims resubmission guidelines.

Information Systems Review Findings—Implications

Access to Care

• The MHP’s current systems do not offer the mental health Access Call Center or SUTS

Gateway Center on-line real-time information about available appointment slots at

MHP, including contract provider sites.

• Access Call Center employees continue an outdated method to schedule appointments.

This involves walking to a group of hand-maintained “white boards” to identify

available treatment slots while placing the caller on hold. Note, the MHP plans to retire

the “white boards” once Netsmart myAvatar is fully operational.

• Reviewers heard repeated references to a concern regarding waiting lists at the Access

Call Center for adults and children. The number of adults and children awaiting

treatment slots through the Access Call Center should be numbers known to MHP

leadership in real-time every day. This can be a constant indicator for how the MHP is

doing with treatment capacity and access to services.


• MHP staff stated with the Epic HealthLink implementation end-user testing was just

beginning and intended to go into production use on February 28, 2018. Configuring

behavioral health functionality in what is a hospital information system (Epic) is a

challenge. This is new territory for the MHP and possibly Epic.

• The concern here is that with testing just starting, the MHP may not yet have a final,

locked-down, training environment and fully-populated final training materials with

five weeks left to go-live.

Denial Reason DescriptionNumber

Denied

Dollars

Denied

Percent

of Total

DeniedBeneficiary not eligible or aid code invalid or restricted service indicator must be "Y" 14,603 $10,013,338 48%Missing, incomplete, invalid ICD-10 diagnosis or condition 3,123 $3,804,765 18%Other coverage must be billed prior to submission of this claim 4,180 $3,418,426 17%Total Denied Claims 26,105 $20,659,099 100%

Table 15: Santa Clara MHP Summary of CY16 Top Three Reasons for Claim Denial

- 56 -


• With what appears to be minimal staffing on the project team and a small amount of

training delivered, this could be either a very challenging go-live or a delayed go-live. It

can be expected, in either case that during the adoption phase clinicians will be slower

in Epic HealthLink than they were in Pro-filer and that may impact workflow in the

clinics.

Quality of Care

• Epic HealthLink is, at its core, an inpatient and outpatient hospital information system.

The MHP should be prepared for the need to make adjustments after the system goes

into production use. This will require resources that are very customer-support

oriented and experts in Epic HealthLink and either conversant with or at least have

access to staff who are conversant with the MHP’s clinical service delivery policies and

procedures.

• The MHP will be dependent for its success on the reliable seamless coordinated

operation of two highly complex information systems working together in a way that

has little precedent in California.

• For some time, likely measured in years, this new information systems infrastructure

will require considerable expert support. The number and level of resources currently

available to support mission critical operations and data exchange processes will need

to continue to be monitored for the impact on the system.

• The MHP can expect a considerable number of requests for either modifying reporting

out of Netsmart myAvatar and Epic HealthLink or creating new reports to help manage

services in this new and very complex information environment. It is unclear that the

MHP has the depth of staffing resources to provide timely turnaround on these requests.

Consumer Outcomes

• With a potential delay in start-up secondary to the monumental undertaking in the

conversion process, consumer appointments may be adversely affected as staff attend

to necessary adjustments in learning the system.

- 57 -


SITE REVIEW PROCESS BARRIERS The following conditions significantly affected CalEQRO’s ability to prepare for and/or conduct a

comprehensive review:

No barriers were encountered during this review.

- 58 -


CONCLUSIONS During the FY17-18 annual review, CalEQRO found strengths in the MHP’s programs, practices, or

information systems that have a significant impact on the overall delivery system and its

supporting structure. In those same areas, CalEQRO also noted opportunities for quality

improvement. The findings presented below relate to the operation of an effective managed care

organization, reflecting the MHP’s processes for ensuring access to and timeliness of services and

improving the quality of care.

Strengths and Opportunities

Access to Care

Strengths:

• Implementation of the CISCO call system strengthens the ability to streamline and

coordinate initial consumer contact.

• The new Integrated Organizational Structure potentially created a clean structure with

room for growth; all age groups share administrative, operational and fiscal integration.

• The MHP is currently working to introduce same day Access service.

• Hiring consultants, as the MHP has done, may contribute to quicker implementation of

efficiencies across the MHP system.

Opportunities:

• Continue to monitor its new call center processes for efficiencies and timeliness to meet

its standards.

• Network adequacy across the system remains a challenge in regard to serving

consumers. Primary challenges are hiring and retention, especially with the contract

providers.

• External factors are the shortage of qualified staff across the Bay Area, coupled with

high housing costs and low unemployment rates that continue to impact services,

recruitment and retention.

• With the continued integration of the mental health Access Call Center and SUTS

Gateway Center to streamline operations, and coordinate services to beneficiaries with

co-occurring disorders, consumers benefit.

• With the implementation of Netsmart myAvatar, retiring the “white boards” that track

appointment slots and provide Access Call Center employees on-line real-time access to

available appointment slots lead to updated access processes.

- 59 -



Strengths:

• The clinical PIP was successful in its focus on strategies to improve timeliness and

subsequently improve timely triage and identification of consumer needs.

• The 7-day outpatient follow-up rate post-hospitalization improved, and the 30-day

outpatient follow-up rate post-hospitalization shows greater improvement. This may be

a result of an inpatient liaison to assure consumers receive a follow-up session.

Opportunities:

• Review the timeline and business plan for the MHP’s contract providers’ conversion to

the EHR to ensure efficiencies in the workflow and prevent delays in claims submissions

as well.

• Investigate the anecdotal references to a call-back list at the Access Call Center for

numerous consumers waiting for appointments. Issues regarding unused capacity may

be indicated. This is further supported by an average clinical staff productivity of 49

percent.

• Potentially assigning a peer support worker at the inpatient follow-up, rather than using

the clinical staff, as resources are scarce, could improve follow-up. The WPC initiative

includes plans to establish peer navigators to assist with this follow-up.

Quality of Care

Strengths:

• Renewed vision has emerged with leadership and its efforts to integrate care among

mental health, quality management, physical health, and substance use services.

• The MHP has strengthened its infrastructure with additional management positions to

provide oversight to operationalize its integration efforts.

• The expanded efforts and an integrated vision among leadership appears to have

resulted in a cohesive management team.

• Consistently analyzing the impact of the effectiveness of the MHP initiatives to improve

service timeliness and quality are underway with the Data Dashboards.

Opportunities:

- 60 -


• Reviewing the MHP EHR business plan and implementation timeline can ensure that its

decisions fully support the integrity and realistic capability of a roll-out among contract

providers to achieve functionality and to be successful is duly warranted.

• Potentially unintended results were reported from focus groups that expressed the

reorganization resulted in stakeholders feeling distanced from leadership and some

indicated they felt their input had been devalued.

• Opportunities to include various levels of stakeholders, and to continue transparent

communication are encouraged.

• Differences in communications and decisions among contract monitors were expressed

in focus groups. Contract providers suggest a standardized policy so that contract

monitors have more clear expectations. The new contract process may expedite this

effort.

• Provider to provider transfer is awkward and reportedly is frayed. This suggests a

burden to both consumer and providers in pursuing seamless service delivery.

• Publishing announcements and changes in processes through a portal or web page for

providers rather than relying on emails lists could assist with consistent messaging.

• While effective communication is apparent at the executive level, evaluation of methods

to spread to various levels is encouraged.

• The MHP is overdue in establishing a Clinical Supervisor classification. Although this

topic has been a focus, results were not yet achieved. The class might prove to be an

important channel of communication to employees in addition to other benefits.

Consumer Outcomes

Strengths:

• The clinical PIP indicated improvements to both access and timeliness for which the

MHP intends to replicate in additional clinics.

• Creation and current recruiting for the Consumer and Family Affairs Division Director

position indicates commitment to strengthen the consumer voice.

• The MHP has relatively admirable penetration rates and based on ACB figures appears

to deliver a more than usual level of service to its beneficiaries.

Opportunities:

• With only two wellness self-help centers in the county, Zephyr in San Jose and

Esperanza in Gilroy, evaluating the option of increasing the number is suggested. Both

centers appear underfunded for staff and other resources.

- 61 -


• While the high penetration rates and average claim per beneficiary figures are

significant, analyze whether this is translating into equally admirable outcomes.

Rehospitalization rates, for instance, remain somewhat above the state average.

Recommendations

• Immediately apply additional resources to the implementation of electronic data

exchange between the contract providers and the MHP using Netsmart myAvatar.

o Release thorough Companion Guides that detail the transactions or web services to

be implemented with full detail of data content and formatting specifications as

soon as possible.

o Institute a formal process for determining the readiness of trading partners, that is,

contract providers and their vendors and take a low-risk approach to on-boarding

contract providers. Start with one or two contract providers that are technically

capable, with vendors that have experience with electronic data integration, and

who are eager to participate in this transition.

o Limit the number of contract providers on-boarding in any month to what your

existing staff can realistically support through a challenging transition.

• Strongly advocate for the four additional full-time employees for the Netsmart

myAvatar support team and the six additional staff for the Epic HealthLink support

team. Either of these systems require considerable daily support and oversight. With

them working together, the support need will be even greater. Do not underestimate the

continuing support need involved in managing systems of this scale and complexity.

• Continue to advocate and strengthen the potential of creating a clinical supervisory

position or equivalent. Consider a comparison with other comparable MHPs. Consider

the effects this classification would also provide as a career path for talented clinicians

that could lead to management and possibly reduce turnover in key positions. At a

minimum, determine whether other compensatory benefits could be applicable for the

additional responsibilities.

• Review the recommendation to increase wellness Self-Help Center funding which

contribute to daily activities at the current sites. Study the need for a third center given

the large geographic region of the county.

• Investigate the exceptionally high level of high cost beneficiary (HCB)spending for

Hispanics and children ages 6-17 to determine if it reflects management priorities and

treatment needs of this population versus lack of step-down services. Evaluate the

clinical outcomes for this group. Analyze the transition process or other factors that

may increase costs without improving outcomes.

- 62 -


ATTACHMENTS

Attachment A: CalEQRO On-site Review Agenda

Attachment B: On-site Review Participants

Attachment C: Approved Claims Source Data

Attachment D: CalEQRO Performance Improvement Plan (PIP) Validation Tools

- 63 -


Attachment A—On-site Review Agenda

The following sessions were held during the MHP on-site review, either individually or in

combination with other sessions.

Table A1—EQRO Review Sessions - Santa Clara MHP

Opening Session – Changes in the past year; current initiatives; and status of previous year’s recommendations

Use of Data to Support Program Operations

Disparities and Performance Measures/ Timeliness Performance Measures

Quality Improvement and Outcomes

Performance Improvement Projects

Health Plan and Mental Health Plan Collaboration Initiatives

Clinical Line Staff Group Interview

Clinical Supervisors Group Interview

Consumer Employee Group Interview

Consumer Family Member Focus Group(s)

Contract Provider Group Interview – Administration and Operations

Contract Provider Group Interview –Quality Management

Validation of Findings for Pathways to Mental Health Services (Katie A./CCR)

ISCA/Billing/Fiscal

EHR Deployment

Access Call Center Site Visit

Wellness Center Site Visit

Site Visit to Innovative Clinical Programs: Innovative program/clinic that serve special populations or offer special/new outpatient services.

- 64 -


Attachment B—Review Participants

CalEQRO Reviewers

Jovonne Price, Quality Reviewer Maureen Bauman, Quality Reviewer Bill Ullom, Chief, Information Systems Reviewer Robert Greenless, Information Systems Reviewer Walter Shwe, Consumer/Family Member Consultant

Additional CalEQRO staff members were involved in the review process, assessments, and

recommendations. They provided significant contributions to the overall review by participating in

both the pre-site and the post-site meetings and in preparing the recommendations within this

report.

Sites of MHP Review

MHP Sites

2325 Enborg Lane San Jose, CA 95128

1075 E. Santa Clara Street San Jose, CA 95116 Kidscope Center 828 Bascom Avenue San Jose, CA 95116 Contract Provider Sites

None

- 65 -


Table B1 - Participants Representing the MHP

Last Name First Name Position Agency

Tullys Toni Director

Santa Clara

County

Behavioral Health

Services

Department

(BHSD)

Wiley Deane Deputy Director BHSD

Copley Bruce Director, Alcohol &

Drug Services, Access BHSD

Webber Whitney Quality Management

Director BHSD

Landreneau Todd Adult/Older Adult

Director BHSD

Terao Sherri Family & Children’s

Director BHSD

Ho Tiffany Medical Director BHSD

Luna Laura

Senior Health Care

Program Manager-

Health Plan

BHSD

Olivares Gabby Division Director-

Criminal Justice

BHSD

Hernandez Sandra Division Director-

Integrated Health BHSD

Obilor Margaret Division Director-

Adult-Older Adult BHSD

Juarez Maretta Division Director-

Family & Children BHSD

Garcia Pat Director of

Administration BHSD

Le Mikelle

Senior Manager-

MHUC, Call Center,

CWBC

BHSD

Anderson Alicia

Sr. Health Care

Program Manager-

WPC

BHSD

- 66 -




Chu Dinh

Sr. Health Care

Program Manager-

WPC

BHSD

Faria-Costa Zelia Senior Manager-F&C

Division BHSD

Jung Soo Senior Manager-

Housing BHSD

Harnish Mary Compliance & Privacy

Mgr., MHS

BHSD

Berman Cheryl

SUTS Division

Director-Adult-Older

Adult

BHSD

Garrison Victoria Senior Manager-CJS BHSD

Yuter Sheila Senior Manager-

HealthLink BHSD

Ledesma Margaret Senior Manager-Katie

A.

BHSD

Acevedo Domingo QA Manager BHSD

Hogan Sheryl Senior Health Care

Analyst BHSD

Nguyen Hung Senior Manager-

Decision Support BHSD

Ferris Jeannette Senior Manager-

Learning Partnership BHSD

Pangilinan MariaEva Cultural Competency BHSD

Nguyen Thuhien ECCAC BHSD

Rocco Bob Quality Imp. Coord II BHSD

Morale Jeanne Senior Health Care

Program Manager BHSD

Chiu Suzanne LCSW, Clinical Lead

Downtown MHC BHSD

Iwu Loretta

Contract Monitor,

Adult-Older Adult

Programs BHSD

- 67 -




Ruiz Enrique MHPSW BHSD

Lai Evonne Contracts Manager BHSD

Robben Vince Finance BHSD

Janini Yasmina Decision Support BHSD

Marquez Veronica Decision Support BHSD

Rodriguez Arisve Clinician Kidscope-BHSD

Nghe Diem Clinician Call Center-BHSD

Picazo Connie Clinician Call Center-BHSD

Ortiz Yesenia Clinician CWBC-BHSD

Rodriguez Luis Clinician CWBC-BHSD

Ponce Mario Clinician CalWorks-BHSD

Urrutia Manuel Clinician CalWorks-BHSD

Trugman Paul Clinician CalWorks-BHSD

Cowan Lance Clinician Downtown-BHSD

Lopez-Molina Mauricio Clinician Las Plumas-BHSD

Bravo Wendy Clinician Las Plumas-BHSD

Koomson James Community Worker Las Plumas-BHSD

Vierra Amanda Clinical Standards-MH BHSD

Powell Larry Clinical Standards-MH BHSD

- 68 -


Nelson Tianna

Clinical Standards-

SUTS BHSD

Jones Jennifer Program Manager-

Consumer Affairs BHSD

Kibel Sharla Clinician Kidscope-BHSD

Cornejo Vanessa Clinician Call Center-BHSD

Estrada Jose Clinician Juvenile Hall-

BHSD

Ascue Yukie Clinician Sunnyvale-BHSD

Salada Brian Program Manager-

Sunnyvale Sunnyvale-BHSD

Antons Peter

Program Manager-Las

Plumas/Receiving

Center

Las Plumas-BHSD

Brewer Martha Program Manager-

CalWorks CalWorks-BHSD

Talamantez Rachel Program Manager-

Kidscope Kidscope-BHSD

Bueno Lidia Program Manager-

Kidscope Kidscope-BHSD

DeLeon Judy Program Manager-

CWBC & MHUC BHSD

Lownsbery Steve Clinical Standards-

SUTS BHSD

Hardy John MHPSW BHSD

Blade Michelle MHPSW BHSD

Huang Sophia MHPSW BHSD

Guido Diana MHPSW BHSD

Rahim Hussein MHPSW BHSD

Torres Sharon ECCAC BHSD

De Castro Eramelisse ECCAC BHSD

Wardle Billy Clinician Call Center-BHSD

- 69 -


Attachment C—Approved Claims Source Data

Approved Claims Summaries are provided separately to the MHP in a HIPAA-compliant manner.

Values are suppressed to protect confidentiality of the individuals summarized in the data sets

where beneficiary count is less than or equal to eleven (*). Additionally, suppression may be

required to prevent calculation of initially suppressed data, corresponding penetration rate

percentages (n/a); and cells containing zero, missing data or dollar amounts (-).

Table C1 shows the penetration rate and approved claims per beneficiary for just the CY16 ACA

Penetration Rate and Approved Claims per Beneficiary. Starting with CY16 performance measures,

CalEQRO has incorporated the ACA Expansion data in the total Medi-Cal enrollees and beneficiaries

served.

Table C2 shows the distribution of the MHP beneficiaries served by approved claims per beneficiary

range for three cost categories: under $20,000; $20,000 to $30,000, and those above $30,000.

Entity

Average

Monthly ACA

Enrollees

Number of

Beneficiaries

Served

Penetration

Rate

Total Approved

Claims

Approved

Claims per

Beneficiary

Statewide 3,674,069 141,926 3.86% $611,752,899 $4,310

Large 1,778,582 67,721 3.81% $318,050,214 $4,696

Santa Clara 136,481 4,939 3.62% $41,406,080 $8,383

Table C1: Santa Clara MHP CY16 Medi-Cal Expansion (ACA) Penetration Rate

and Approved Claims per Beneficiary

Range of

ACB

MHP Count of

Beneficiaries

Served

MHP

Percentage of

Beneficiaries

Statewide

Percentage of

Beneficiaries

MHP Total

Approved

Claims

MHP

Approved

Claims per

Beneficiary

Statewide

Approved

Claims per

Beneficiary

MHP

Percentage

of Total

Approved

Claims

Statewide

Percentage

of Total

Approved

Claims

< $20K 18,592 81.18% 94.05% $110,403,052 $5,938 $3,612 37.87% 59.13%

>$20K -

$30K1,947 8.50% 2.83% $47,443,332 $24,367 $24,282 16.27% 11.98%

>$30K 2,362 10.31% 3.12% $133,669,749 $56,592 $53,215 45.85% 28.90%

Table C2: Santa Clara MHP CY16 Distribution of Beneficiaries by ACB Range

- 70 -


Attachment D—PIP Validation Tool

PERFORMANCE IMPROVEMENT PROJECT (PIP) VALIDATION WORKSHEET FY17-18 CLINICAL PIP

GENERAL INFORMATION

MHP: Santa Clara

PIP Title: Welcome to Treatment Group

Start Date (MM/DD/YY): 12/15/16

Completion Date (MM/DD/YY): 12/31/17

Projected Study Period (#of Months): 12

Completed: Yes ☒ No ☐

Date(s) of On-Site Review (MM/DD/YY):

01/23-25/17

Name of Reviewer: Jovonne Price

Status of PIP (Only Active and ongoing, and completed PIPs are rated):

Rated

☐ Active and ongoing (baseline established and interventions started)

☒ Completed since the prior External Quality Review (EQR)

Not rated. Comments provided in the PIP Validation Tool for technical assistance purposes only.

☐ Concept only, not yet active (interventions not started)

☐ Inactive, developed in a prior year

☐ Submission determined not to be a PIP

☐ No Non-Clinical PIP was submitted

Brief Description of PIP (including goal and what PIP is attempting to accomplish):

- 71 -


The goal of the PIP was to engage and improve timely delivery of services to individuals waiting for specialty behavioral health treatment at Downtown Behavioral Health.

ACTIVITY 1: ASSESS THE STUDY METHODOLOGY

STEP 1: Review the Selected Study Topic(s)

Component/Standard Score Comments

1.1 Was the PIP topic selected using stakeholder input? Did the MHP develop a multi-functional team compiled of stakeholders invested in this issue?

☒ Met

☐ Partially Met

☐ Not Met

☐ Unable to Determine

The multi-functional team was made up of representatives from outpatient Contracted and County Clinics (manager and staff), Central Wellness (manager), and Narvaez Behavioral Health (manager and staff). Suzanne Chiu (Lead DTBH) and Enrique Ruiz (Peer Support Worker DTBH) were the group facilitators. There were 2 consumers involved in development of the Welcome to Treatment Group, Peer Support Workers.

- 72 -


1.2 Was the topic selected through data collection and analysis of comprehensive aspects of enrollee needs, care, and services?

☒ Met

☐ Partially Met

☐ Not Met


The Performance Improvement Project is focused on timeliness to outpatient services. The Santa Clara County Behavioral Health Department (BHSD) was having difficulty meeting its timeliness metrics. Some factors that might have contributed to the problem are as follows:

• Limited outpatient capacity

• Lack of a step-down process for clients to transition to a lower level of care

• Limited coordination of referral process among the County, contracted agency clinics and Call Center

• Lack of follow up and outreach to support a warm hand-off and engagement with outpatient services

• Lack of client knowledge of behavioral health services and community resources.

Behavioral Health Call Center specialty behavioral health report validates the number of consumers needing services.

An average of 54 days had elapsed from initial request for these consumers who had not yet been assigned to specialty mental health services.

Select the category for each PIP:

Clinical:

☐ Prevention of an acute or chronic condition ☐ High volume services

☒ Care for an acute or chronic condition ☐ High risk conditions

Non-Clinical:

☐ Process of accessing or delivering care

- 73 -


1.3 Did the Plan’s PIP, over time, address a broad spectrum of key aspects of enrollee care and services?

Project must be clearly focused on identifying and correcting deficiencies in care or services, rather than on utilization or cost alone.

☒ Met

☐ Partially Met

☐ Not Met


The problem facing the Santa Clara System of care was increased demand in relation to service provision capabilities, thereby creating delays to services. Delays in providing timely access to services for consumers seeking specialty mental health reduces quality of care and poor consumer outcomes (Evans N, 2014).

• To improve outcomes of care and support engagement in the treatment services, information that consumers can utilize when the clinic is closed is provided to the attendees in a welcome packet.

• To familiarize the consumer with the treatment environment, encourage the consumers’ willingness to engage and stay in treatment, help the consumer feel safe and secure, and reduce the no-show rates, a clinician and a peer support worker reviews the welcome packet and provides other information regarding the available wellness and recovery services.

• The clinician checks in with consumers for brief intervention to see if there will be need for medication refill prior to intake and estimated medication evaluation date by the psychiatrist. This process makes the consumer feel heard and supported with their individual needs are taken into consideration as evident by some of the surveys completed by the consumers at the end of the group. This improves outcome by ensuring medication stability for consumers and decreases the likelihood for consumer decompensation due to being out of medication prior to their appointment.

• The clinician triages the needs of the clients with a brief assessment to determine if the consumers need to be connected to urgent care for medication support or an earlier appointment for stabilization. This process facilitates treatment timeliness, which helps to prevent further decompensation and possible hospitalization. Whereby, further stabilization is necessary, a 5150 protocol is initiated,

- 74 -


and the consumer is transported to emergency psychiatry services.

• Consumers are guided on the process of obtaining medication from BH Urgent Care if necessary prior to psychiatrist’s evaluation. This process encourages staff/consumer interactions for better support and good quality care.

• Face to face interactions between consumers and their treatment team are coordinated to encourage engagement and treatment success. The clerical staff will schedule an intake appointment during the group and the consumer meets with assigned clinician the same day if possible.

• In addition, individuals are informed about and given a tour of Zephyr, a self-help center which is located on the grounds of the Downtown Behavioral Health Clinic. This center provides peer run groups, computer classes, consumer rights workshops, and other wellness and recovery activities, which is available to consumers as a treatment option. The center provides additional peer support to help improve interactions and socialization with other consumers, which increases attendance and utilization of the Self-Help Center by the attendees of the Welcome to Treatment Group. This improves outcomes by increasing peer support interactions and resources.

1.4 Did the Plan’s PIPs, over time, include all enrolled populations (i.e., did not exclude certain enrollees such as those with special health care needs)?

Demographics:

☐ Age Range ☐ Race/Ethnicity ☐ Gender ☐ Language ☐ Other

☒ Met

☐ Partially Met

☐ Not Met


The population for this project are those consumers who are referred through the Call Center and includes the following categories:

• Those who call the BHS Call Center requesting services

• Those referred by their Primary Care Physician for mental health services

• Those who are attempting to reenter the system (either because they were previously closed or some other reasons).

Totals 4 Met 0 Partially Met 0 Not Met 0 UTD

- 75 -


STEP 2: Review the Study Question(s)

2.1 Was the study question(s) stated clearly in writing?

Does the question have a measurable impact for the defined study population?

Include study question as stated in narrative:

☒ Met

☐ Partially Met

☐ Not Met


Will the development and implementation of Welcome to Treatment at Downtown Behavioral Health increase timely access to care from 18.1 days to 14 days by December 2017, engagement, and overall consumer perception within specialty behavioral health outpatient services?


STEP 3: Review the Identified Study Population

3.1 Did the Plan clearly define all Medi-Cal enrollees to whom the study question and indicators are relevant?

Demographics:


☒ Met

☐ Partially Met

☐ Not Met


The population at DTMH is predominantly Hispanic and White. The gender distribution is somewhat evenly distributed, with slightly more males. The population is mainly adults ages 26-59.

See PIP submission for graphs indicating the demographics.

3.2 If the study included the entire population, did its data collection approach capture all enrollees to whom the study question applied?

Methods of identifying participants:

☒ Utilization data ☒ Referral ☐ Self-identification

☐ Other:

☒ Met

☐ Partially Met

☐ Not Met



- 76 -


STEP 4: Review Selected Study Indicators

4.1 Did the study use objective, clearly defined, measurable indicators?

List indicators: 1. Average calendar days to initial service at DTBH 2. Percent of consumers at DTBH who have their initial service within

14 days 3. Percent of consumers opened to DTBH after referral from the Call

Center 4. Percent of consumers who were not opened to DTBH after

referral from the Call Center because they ended up receiving crisis services

5. Average number of face to face services within 60 days of admit for consumers who are opened to Downtown Mental Health

6. Percent of new consumer admits with 4 visits within 30 calendar days at DTBH

7. Percent of consumers who strongly agreed or disagreed Welcome to Treatment Group was useful at DTBH

8. Percent of consumers who strongly agreed or disagreed they were able to receive adequate information about the services they would be receiving at DTBH

9. Percent of consumers who strongly agreed or disagreed they felt more knowledgeable in how to better navigate their treatment at DTBH

☒ Met

☐ Partially Met

☐ Not Met


Goals for each indicator:

1. 14 days

2. 2 percentage points improvement over baseline

3. 20percent increase from baseline with appropriate openings

4. n/a

5. 75 percent (MHP standards: A minimum of 2 hours per visit and 4 contacts per 30 days)

6. 75 percent

7. Client perception data

8. 75percent

9. 75percent

- 77 -


4.2 Did the indicators measure changes in: health status, functional status, or enrollee satisfaction, or processes of care with strong associations with improved outcomes? All outcomes should be consumer focused.

☐ Health Status ☒ Functional Status

☒ Member Satisfaction ☐ Provider Satisfaction

Are long-term outcomes clearly stated? ☒ Yes ☐ No

Are long-term outcomes implied? ☒ Yes ☐ No

☒ Met

☐ Partially Met

☐ Not Met


• The client perception data was collected using a 5-question Welcoming Group Satisfaction survey with a 5-point Likert scale (strongly disagree, disagree, neutral, agree, strongly agree) developed by DTBH.

• Timely access to service within 14 days to understand how to improve the effectiveness of consumer’s early engagement and support, which is a proven method of keeping consumer’s in treatment.

• To show an increase in the number of consumers attending DTBH.

• To demonstrate we can keep people out of crisis settings through the interventions.

• To demonstrate consumer engagement through face to face contacts within 60 days after being opened at DTBH

• Percentage of new consumer admits with 4 visits within 30 days is a standard set forth for ongoing rapport and engagement during the first 30 days

• To obtain consumers’ perception on usefulness of the Group to determine if providing a Welcome to treatment group is beneficial to the clients

• To obtain consumers’ perception on adequacy of information received about services from the Group. To determine areas of improvement in the information provided to clients during the group.

• To obtain consumers’ perception on their ability to navigate treatment at DTBH.


- 78 -


STEP 5: Review Sampling Methods

5.1 Did the sampling technique consider and specify the:

a) True (or estimated) frequency of occurrence of the event?

b) Confidence interval to be used?

c) Margin of error that will be acceptable?

☐ Met

☐ Partially Met

☐ Not Met

☒ Not Applicable


5.2 Were valid sampling techniques that protected against bias employed?

Specify the type of sampling or census used:

<Text>

☐ Met

☐ Partially Met

☐ Not Met

☒ Not Applicable


5.3 Did the sample contain a sufficient number of enrollees?

______N of enrollees in sampling frame

______N of sample

______N of participants (i.e. – return rate)

☐ Met

☐ Partially Met

☐ Not Met

☒ Not Applicable


Totals 0 Met 0 Partially Met 0 Not Met 3 NA 0 UTD

STEP 6: Review Data Collection Procedures

6.1 Did the study design clearly specify the data to be collected?

☒ Met

☐ Partially Met

☐ Not Met


See Indicators in Item 4.1 above.

6.2 Did the study design clearly specify the sources of data?

Sources of data:

☒ Member ☐ Claims ☐ Provider

☒ Other: Unicare

☒ Met

☐ Partially Met

☐ Not Met


All of the access, engagement, and outcomes data for this PIP is collected in Unicare, the MHP’s electronic health record. The access data is captured by BHS Call Center staff for all consumers.

The second set of data, service documentation related to

engagement and outcomes, is also collected in Unicare. This

- 79 -


information is double-checked for accuracy because wrong

information may result in payment denial by the insurer or rejection

by the state reporting (CSI) for errors.

By combining these two sets of data in Unicare, the Decision Support

team in the Quality Management Departments has the ability to

calculate the performance indicators for access, engagement, and

outcomes for this PIP.

DTBH clinic staff asked all consumers at the conclusion of the

Welcoming Group to complete a survey on their experience.

- 80 -


6.3 Did the study design specify a systematic method of collecting valid and reliable data that represents the entire population to which the study’s indicators apply?

☒ Met

☐ Partially Met

☐ Not Met


Once the form is filled out and saved, it is available for other Call Center staff and the receiving treatment providers to review. Call Center staff collects the data on an ongoing basis, the data is analyzed monthly by the Decision Support team and reported quarterly.

When the consumers or family members call the toll-free number, BHS Call Center staff begin the screening process to capture the individual’s demographic and contact information. In addition, the staff make an initial screening to determine the consumer level of care. Based on all the collected information and availability of capacity, the BHS Call Center staff make appropriate treatment referrals. The access data is captured in a User Defined Data (UDD) Mental Health Referral Form. Once the form is filled out and saved, it is available for other Call Center staff and the receiving treatment providers to review.

Once the consumer shows up to their appointment at the treatment location, clerical staff verify and document the consumer demographics, contact and insurance information in Unicare. After this process, the consumer then meets with their assigned clinician. The clinician documents all services provided to the consumer along with progress notes in Unicare.

The Unicare data provided is routinely reviewed monthly and quarter by the management team, quality performance committee and A/OA system of care committee. The client perception data was collected at the end session.

- 81 -


6.4 Did the instruments used for data collection provide for consistent, accurate data collection over the time periods studied?

Instruments used:

☒ Survey ☒ Medical record abstraction tool

☐ Outcomes tool ☐ Level of Care tools

☐ Other:

☒ Met

☐ Partially Met

☐ Not Met


Unicare and consumer survey were used consistently.

6.5 Did the study design prospectively specify a data analysis plan?

Did the plan include contingencies for untoward results?

☒ Met

☐ Partially Met

☐ Not Met


Daily log-in with real-time notes of consumer calls; monthly and quarterly reports to management for review.

Untoward results would be addressed on-going by the appropriate unit, i.e., staff, QM, or Data as applicable.

6.6 Were qualified staff and personnel used to collect the data?

Project leader:

Name: Loretta Iwu and Suzanne Chiu

Title: MHPS PSW

Role: Contract Manager/Lead Clinician

Other team members:

Names: See comments section.

☒ Met

☐ Partially Met

☐ Not Met


Call Center staff including clerical and licensed clinicians collected data from consumers. Unicare data was collected by staff in the Decision Support team and the Quality Management staff. All staff are qualified per job specifications.


- 82 -


STEP 7: Assess Improvement Strategies

7.1 Were reasonable interventions undertaken to address causes/barriers identified through data analysis and QI processes undertaken?

Describe Interventions:

• The clinician triages the needs of the clients with a brief assessment to determine if the consumers need to be connected to urgent care for medication support or an earlier appointment for stabilization

• The clinician checks in with consumers for brief intervention to see if there will be need for medication refill.

• Provide a welcoming pamphlet and packet to consumers who are seeking services for additional education and behavioral health resources, which enables consumers to make informed decisions about treatment options to improve engagement

• Create therapeutic, supportive environment that facilitates client-staff therapeutic alliance, as consumers, families, and other support systems connects with the case manager and clinician the same day of group for better support and engagement.

• Also streamline workload and workflow for staff from documentation to caseload.

☒ Met

☐ Partially Met

☐ Not Met



- 83 -


STEP 8: Review Data Analysis and Interpretation of Study Results

8.1 Was an analysis of the findings performed according to the data analysis plan?

This element is “Not Met” if there is no indication of a data analysis plan (see Step 6.5)

☒ Met

☐ Partially Met

☐ Not Met

☐ Not Applicable


See Item 6.3 and Item 6.5 above for details.

- 84 -


8.2 Were the PIP results and findings presented accurately and clearly?

Are tables and figures labeled? ☒ Yes ☐ No

Are they labeled clearly and accurately? ☒ Yes ☐ No

☒ Met

☐ Partially Met

☐ Not Met

☐ Not Applicable


Results are clear and labeled.

• The Unicare data provided is routinely reviewed monthly and quarter by the management team, quality performance committee and A/OA system of care committee. The client perception data was collected using a 5-question Welcoming Group Satisfaction survey with a 5-point Likert scale (strongly disagree, disagree, neutral, agree, strongly agree) developed by DTBH.

• The average calendar days to initial service at DTBH improved by 4percent with 19percent improvement of level 2 consumers participating in Welcoming Group being opened within 14 days of initial service. This result may trigger the need to include the call center, all county clinics, and contract providers to further improve timely access to care.

• The project obtained real-time referrals during the 12-month period which lead to improved results as evidenced by a reduction of an average wait time from 54 days to 32 days. There may be a need to continue real-time referrals which will result in zero wait time for consumers seeking specialty mental health.

• Engagement between staff/consumers as shown increased face-to-face interactions, an improvement of 664percent for DTBH and 820percent for the Welcoming Group. This may trigger additional clinicians to participate in the facilitation of the group to improve engagement and to provide clinical support to consumers.

- 85 -


• Based on consumer perception data, 81.8percent of consumers found the group useful. There may be a need to implement a short therapy group as part of the welcome to treatment to help enhance consumers’ coping skills to add value to the group.

8.3 Did the analysis identify: initial and repeat measurements, statistical significance, factors that influence comparability of initial and repeat measurements, and factors that threaten internal and external validity?

Indicate the time periods of measurements: ___________________

Indicate the statistical analysis used: _________________________

Indicate the statistical significance level or confidence level if available/known: _______percent ______Unable to determine

☐ Met

☒ Partially Met

☐ Not Met

☐ Not Applicable


The MHP did not analyze with respect to the demographic parameters of our study population before and after the intervention. The measures for the study populations were taken at different times of the year.

Data was collected regularly throughout the year.

See Item 8.2 above.

8.4 Did the analysis of the study data include an interpretation of the extent to which this PIP was successful and recommend any follow-up activities?

Limitations described:

Conclusions regarding the success of the interpretation:

Recommendations for follow-up:

☒ Met

☐ Partially Met

☐ Not Met

☐ Not Applicable


• Based on the data analysis results, the PIP is successful as evidenced by improved timeliness to access for consumers which is reflected in the 4percent improvement for average calendar days to initial services at DTBH.

• There has been an improvement for percent of consumers at DTBH who have their initial services within 14 days with 7percent increase after the project.

• There is no statistical evidence on the data analysis table on engagement, however, engagement between staff and consumers has improved since the implementation of the group as evidence by decreased no show rates and increase in utilization of services at the clinic.

• There was 81.8percent of consumers who reported the Welcoming Group to be useful and 89.1percent who reported they were able to receive adequate information.

See Item 8.2 above.


- 86 -


STEP 9: Assess Whether Improvement is “Real” Improvement

9.1 Was the same methodology as the baseline measurement used when measurement was repeated?

Ask: At what interval(s) was the data measurement repeated?

Were the same sources of data used?

Did they use the same method of data collection?

Were the same participants examined?

Did they utilize the same measurement tools?

☐ Met

☐ Partially Met

☐ Not Met

☒ Not Applicable


Some baseline data was not available as some interventions were newly created. The MHP now has baseline data for these indicators.

Since it is early into the phase of determining real improvement this section is not applicable at this time.

The MHP intends to replicate the group in other centers as it has improved consumers information regarding navigating the system and allowed staff to intervene in a triage manner.

- 87 -


9.2 Was there any documented, quantitative improvement in processes or outcomes of care?

Was there: ☒ Improvement ☐ Deterioration

Statistical significance: ☒ Yes ☐ No

Clinical significance: ☒ Yes ☐ No

☐ Met

☐ Partially Met

☐ Not Met

☒ Not Applicable


See Item 8.4 above.

9.3 Does the reported improvement in performance have internal validity; i.e., does the improvement in performance appear to be the result of the planned quality improvement intervention?

Degree to which the intervention was the reason for change:

☐ No relevance ☐ Small ☐ Fair ☒ High

☐ Met

☐ Partially Met

☐ Not Met

☒ Not Applicable


These results from this PIP may be generalizable to other locations and the MHP is expanding the program to another County clinic and potentially two contract providers.

9.4 Is there any statistical evidence that any observed performance improvement is true improvement?

☐ Weak ☐ Moderate ☐ Strong

☐ Met

☐ Partially Met

☐ Not Met

☒ Not Applicable


No statistical testing was performed; data suggests improved outcomes related to the goals.

9.5 Was sustained improvement demonstrated through repeated measurements over comparable time periods?

☐ Met

☒ Partially Met

☐ Not Met

☒ Not Applicable


This was completed with a first year of data which can be used for repeat measures. The results are promising and show success. This PIP is intended to continue and the MHP will calculate future results for comparisons.


- 88 -


ACTIVITY 2: VERIFYING STUDY FINDINGS (OPTIONAL)


Were the initial study findings verified (recalculated by CalEQRO) upon repeat measurement?

☐ Yes

☒ No

ACTIVITY 3: OVERALL VALIDITY AND RELIABILITY OF STUDY RESULTS: SUMMARY OF AGGREGATE VALIDATION FINDINGS

Conclusions:

The MHP was successful with this PIP in improving timeliness to services within the 14-day standard. The Welcome to Treatment group posed success based on consumer feedback and the intent is to expand the PIP to other clinics and contract provider services.

Recommendations:

Based on the initial success, the MHP is encouraged to continue this PIP and to expand it to improve timeliness to services.

Check one: ☒ High confidence in reported Plan PIP results ☐ Low confidence in reported Plan PIP results

☐ Confidence in reported Plan PIP results ☐ Reported Plan PIP results not credible

☐ Confidence in PIP results cannot be determined at this time

- 89 -


PERFORMANCE IMPROVEMENT PROJECT (PIP) VALIDATION WORKSHEET FY17-18 NON-CLINICAL PIP

GENERAL INFORMATION

MHP: Santa Clara

PIP Title: FY2018 Contract Renewal Process Pilot Project

Start Date (MM/DD/YY): 02/01/2017

Completion Date (MM/DD/YY): 09/30/2017

Projected Study Period (#of Months): 12

Completed: Yes ☒ No ☐

Date(s) of On-Site Review (MM/DD/YY):

01/23-25/2018

Name of Reviewer: Jovonne Price

Status of PIP (Only Active and ongoing, and completed PIPs are rated):

Rated

☐ Active and ongoing (baseline established and interventions started)

☐ Completed since the prior External Quality Review (EQR)

Not rated. Comments provided in the PIP Validation Tool for technical assistance purposes only.

☐ Concept only, not yet active (interventions not started)

☐ Inactive, developed in a prior year

☒ Submission determined not to be a PIP

☐ No Clinical PIP was submitted

Brief Description of PIP (including goal and what PIP is attempting to accomplish):

- 90 -


The project’s aim, specific to Short-Doyle (SD) /Mental Health Services Act (MHSA) mental health service contracts only, was to pilot a new approach to increase timely delivery of executed contracts, ensuring they are in place at the start of the fiscal year for consumers.

It was determined that the non-clinical PIP did not meet the criteria for a PIP secondary to the lack of consumer benefit. Although commendable efforts were done in the MHP’s process and business procedures, there were no stated outcomes for the consumer. However, this PIP did reconstruct the method for timely completion of the MHP contract agreements. The MHP managed to complete over 95percent of its contracts with organizational providers and introduced new efficiencies to do this. Discussion occurred on-site to emphasize using EQRO for technical assistance in developing its PIPs.

- 91 -


ACTIVITY 1: ASSESS THE STUDY METHODOLOGY

STEP 1: Review the Selected Study Topic(s)


1.1 Was the PIP topic selected using stakeholder input? Did the MHP develop a multi-functional team compiled of stakeholders invested in this issue?

☐ Met

☐ Partially Met

☐ Not Met


Although the multi-functional team does not include direct representation from consumer/family members the pilot project was conducted with the County Mental Health Plan (MHP) beneficiaries/enrollees in mind. The County MHP in the beneficiary/recipient’s assigned county is responsible for providing MHP-covered services for eligible recipients in that county. Timely execution of SD/MHSA Community Based Organization service provider contracts which ensures services are in place for Santa Clara County MHP clients throughout the year. Given the PIP is focused on improving operational management of contracts administration, the pilot project does include participants, specifically BHSD staff from specific areas of the Department, who directly contribute to the development and execution of SD/MHSA contracts: Administration, Programs-CYF and A/OA, Finance and Contracts.

- 92 -


1.2 Was the topic selected through data collection and analysis of comprehensive aspects of enrollee needs, care, and services?

☐ Met

☐ Partially Met

☐ Not Met


This does not reflect a data collection and analysis of comprehensive

aspects of enrollee needs, care and services. Services were not

stopped because contracts were not renewed timely. To qualify, the

PIP has to be connected to consumer outcomes as stated this is a

process change.

Prior to the implementation of the FY2018 Contract Renewal Process Pilot Project, BHSD has not been able to execute SD/MHSA renewal contracts by the start of the fiscal year in recent years and has had to execute contracts on a retroactive basis-after the start date of the contract effective date:

In FY2016, BHSD conducted two contract amendment processes: a) Term 1-renew existing SD/MHSA contracts and b) Term 2-amend contracts to reflect contractual changes resulting from Request for Proposal (RFP) activities/awardee results.

For Term 1, 48 of the 51 SD/MHSA contracts renewed on time, but for Term 2, the second contract process rolled out in FY2016, only 10 of the 39 SD/MHSA contracts, which represents about 26percent of the SD/MHSA contracts, renewed on time.

In FY2017, only one of the 36 SD/MHSA CBO service contracts renewed on time.

Select the category for each PIP:

Clinical:

☐ Prevention of an acute or chronic condition ☐ High volume services

☐ Care for an acute or chronic condition ☐ High risk conditions

Non-Clinical:

☐ Process of accessing or delivering care

- 93 -


1.3 Did the Plan’s PIP, over time, address a broad spectrum of key aspects of enrollee care and services?

Project must be clearly focused on identifying and correcting deficiencies in care or services, rather than on utilization or cost alone.

☐ Met

☐ Partially Met

☐ Not Met


The PIP does not discuss in depth or measure consumer outcomes as required to meet criteria.

The goal of the project was to ensure timely execution of service contracts utilized by the County’s Medi-Cal beneficiaries/enrollees. A service contract is an agreement between the Behavioral Health Services Department (County) and a service provider (contractor) in which a contractor agrees to provide mental health services to eligible Medi-Cal beneficiaries of Santa Clara County based on the terms reflected in the contract.

BHSD serves an estimated 30,000 consumers annually with about 30percent of the services being provided by the County and the majority of the services at 70percent are provided by contracted, community-based agencies, who are working in the communities where clients/consumers live.

1.4 Did the Plan’s PIPs, over time, include all enrolled populations (i.e., did not exclude certain enrollees such as those with special health care needs)?

Demographics:


☐ Met

☐ Partially Met

☐ Not Met


A service contract is an agreement between the Behavioral Health Services Department (County) and a service provider (contractor) in which a contractor agrees to provide mental health services to eligible Medi-Cal beneficiaries of Santa Clara County based on the terms reflected in the contract. The FY2018 Contract Renewal Process Pilot Project’s goal was to improve the timely execution of SD/MHSA service contracts provided by community- based organizations, contractors, who serve the County’s Medi-Cal beneficiaries across the age continuum from 0-5 to older adults.


- 94 -


STEP 2: Review the Study Question(s)

2.1 Was the study question(s) stated clearly in writing? Does the question have a measurable impact for the defined study population?

Include study question as stated in narrative: See comments. There is no indication of consumer outcomes included in the study question. It appears to be strictly a business process change which could inadvertently benefit consumers. To date, there is no evidence that is the case.

☐ Met

☐ Partially Met

☐ Not Met


See item 2.1. The MHP presented this study question. This is focused on a business process change and is not consumer focused. “Will the implementation of the FY2018 Contract Renewal Process Pilot Project, which includes the following enhancements as listed below, improve timely execution of the County’s SD/MHSA service contracts?

• The redesign of the Contracting Services Request (CSR) Form to include only vital/relevant information will help facilitate the approval process of the CSR in a timely manner.

• The implementation of the redesigned CSR Form for SD/MHSA service contracts in digitized format for use in DocuSign, a digital transaction management system that provides electronic signature technology for facilitating electronic exchanges of signed documents, will enable efficient tracking and review/approval of CSRs.

• The unbundling of service provider contracts into separate contracts and package by program division: F&C, A/OA, CJS, IBH, and SHS, will provide each program division better control over their specific program contracts and expedite execution of contracts.

• Conducting various meetings related to the pilot project with all parties involved: pilot project team, executive team, and CBO contract providers will ensure everyone knows the goal of the pilot project and roles and responsibilities of each area.

• The creation a status tracking sheet to show current renewal status of all FY18 SD/MHSA service contracts and making it available online/SharePoint site to all staff involved in the pilot project so that everyone is aware and current on the status of contracts throughout the renewal process. “

- 95 -



STEP 3: Review the Identified Study Population

3.1 Did the Plan clearly define all Medi-Cal enrollees to whom the study question and indicators are relevant?

Demographics:


☐ Met

☐ Partially Met

☐ Not Met


The MHP has not included a target group of consumers. This appears to be a business procedural shift which will lead to timely contract completions.

A service contract is an agreement between the Behavioral Health Services Department (County) and a service provider (contractor) in which a contractor agrees to provide mental health services to eligible Medi-Cal beneficiaries of Santa Clara County based on the terms reflected in the contract.

3.2 If the study included the entire population, did its data collection approach capture all enrollees to whom the study question applied?

Methods of identifying participants:

☐ Utilization data ☐ Referral ☐ Self-identification

☒ Other: Contract renewal process/completion dates

☐ Met

☐ Partially Met

☐ Not Met



STEP 4: Review Selected Study Indicators

4.1 Did the study use objective, clearly defined, measurable indicators?

List indicators:

• Turnaround Time (TAT) of SD/MHSA CSR Form Approval

• percent of SD/MHSA contracts executed and effective at the start of FY2018

☒ Met

☐ Partially Met

☐ Not Met


- 96 -


4.2 Did the indicators measure changes in: health status, functional status, or enrollee satisfaction, or processes of care with strong associations with improved outcomes? All outcomes should be consumer focused.

☐ Health Status ☐ Functional Status

☐ Member Satisfaction ☐ Provider Satisfaction

Are long-term outcomes clearly stated? ☐ Yes ☐ No

Are long-term outcomes implied? ☐ Yes ☐ No

☒ Met

☐ Partially Met

☐ Not Met


The MHP does not provide evidence that consumer care was impacted as when contracts were amended due to timeliness, services continued. This appears to be focused on the process changes.

Process measures, while acceptable, must offer strong clinical evidence that the process being measured is meaningfully associated with outcomes. This determination should be based on published guidelines, including citations from randomized clinical trials, case control studies, or cohort studies. At a minimum, the PIP should be able to demonstrate a consensus among relevant practitioners with expertise in the defined area who attest to the importance of a given process.

While enrollee satisfaction is an important outcome of care in clinical areas, improvement in satisfaction should not be the only measured outcome of a clinical project. Some improvement in health or functional status should be addressed

The goal of the project was to ensure timely execution of service contracts utilized by the County’s Medi-Cal beneficiaries/enrollees.

This enables consistent and efficient continued care to consumers served who are served by 70percent contract providers. However, there is no evidence that consumers were not served.


STEP 5: Review Sampling Methods

5.1 Did the sampling technique consider and specify the:

a) True (or estimated) frequency of occurrence of the event?

b) Confidence interval to be used?

c) Margin of error that will be acceptable?

☐ Met

☐ Partially Met

☐ Not Met

☒ Not Applicable


No sampling used.

- 97 -


5.2 Were valid sampling techniques that protected against bias employed?

Specify the type of sampling or census used:

<Text>

☐ Met

☐ Partially Met

☐ Not Met

☒ Not Applicable


5.3 Did the sample contain a sufficient number of enrollees?

______N of enrollees in sampling frame

______N of sample

______N of participants (i.e. – return rate)

☐ Met

☐ Partially Met

☐ Not Met

☒ Not Applicable


Totals 0 Met 0 Partially Met 0 Not Met 0 Not Applicable 0 UTD

- 98 -


STEP 6: Review Data Collection Procedures

6.1 Did the study design clearly specify the data to be collected?

The MHP would need to define: Who will collect data, how often it will be collected, who will analyze the data, how often will it be analyzed, what do they expect the results to be, what will they do if the results are different than what they expect.

☐ Met

☐ Partially Met

☐ Not Met


Data is related to the business processes and does not reflect consumer outcomes.

• Turnaround Time (TAT) of SD/MHSA CSR Form Approval

• percent of SD/MHSA contracts executed and effective at the start of FY2018

6.2 Did the study design clearly specify the sources of data?

Sources of data:

☐ Member ☐ Claims ☐ Provider

☒ Other: contract renewal process-see comments

☐ Met

☐ Partially Met

☐ Not Met


The FY2018 Contract Renewal Process Pilot Project Data Set includes:

• All completed and approved SD/MHSA CSR Forms. As part of the CSR set-up in DocuSign, a copy of a submitted CSR is automatically emailed to the Contracts Manager to give notice that a request has come through and pending approval. A final approved copy of the CSR is also emailed to the Contracts Manager at the end of the process. The CSR Form includes submission date and approval date to enable the team to calculate the CSR approval turnaround time (TAT) from start to finish with the approval of the form.

• The number of contracts approved/executed through the Master Contracts List (MCL) process or by the Board of Supervisors through a Legislative File which are public documents.

• A contract renewal status tracking sheet of all FY18 SD/MHSA service contracts was developed based on input on the needs of all the areas: Programs, Finance, and Contracts and ensure up-to-date contract renewal status information. The file was posted online/SharePoint site to all pilot project participants, and the Contracts team was requested to maintain and update the data file.

- 99 -


6.3 Did the study design specify a systematic method of collecting valid and reliable data that represents the entire population to which the study’s indicators apply?

☐ Met

☐ Partially Met

☐ Not Met


In the past, the original CSR Form did not have a date field for the CSR submission date, and therefore BHSD was unable to track the submission date to determine the average approval turnaround time (TAT) of CSRs and assess performance from year to year.

With the new SD/MHSA CSR Form, the CSR submission date field is included along with the approval date of the CSR which enabled BHSD to determine and calculate the approval TAT per form, calculate the average TAT during the pilot project, as well as TAT by program division level.

6.4 Did the instruments used for data collection provide for consistent, accurate data collection over the time periods studied?

Instruments used:

☐ Survey ☐ Medical record abstraction tool

☐ Outcomes tool ☐ Level of Care tools

☒ Other: ongoing collection during the contract renewal period

☐ Met

☐ Partially Met

☐ Not Met


- 100 -


6.5 Did the study design prospectively specify a data analysis plan?

Did the plan include contingencies for untoward results?

☐ Met

☐ Partially Met

☐ Not Met


The FY2018 Contract Renewal Process Pilot Project Data Set includes:

• All completed and approved SD/MHSA CSR Forms. As part of the CSR set-up in DocuSign, a copy of a submitted CSR is automatically emailed to the Contracts Manager to give notice that a request has come through and pending approval. A final approved copy of the CSR is also emailed to the Contracts Manager at the end of the process. The CSR Form includes submission date and approval date to enable the team to calculate the CSR approval turnaround time (TAT) from start to finish with the approval of the form.

• The number of contracts approved/executed through the Master Contracts List (MCL) process or by the Board of Supervisors through a Legislative File which are public documents.

• A contract renewal status tracking sheet of all FY18 SD/MHSA service contracts was developed based on input on the needs of all the areas: Programs, Finance, and Contracts and ensure up-to-date contract renewal status information. The file was posted online/SharePoint site to all pilot project participants, and the Contracts team was requested to maintain and update the data file.

Limited information was included in the event of untoward results. A defined process to ensure timely completion was not presented. Oversight regarding meeting the timelines was not clear, rather it was hoped staff would adhere to the renewal time.

Contingency plans were limited to “encouraged team members to be proactive to ensure that contracts are moving through the process in an effective, timely manner”.

- 101 -


6.6 Were qualified staff and personnel used to collect the data?

Project leader

Name: Jeanne Moral

Title: Sr. Heath Care Program Manager, System Initiatives

Role: Lead facilitator

Other team members:

Names: various members of the Contracts Team, IT staff

☐ Met

☐ Partially Met

☐ Not Met



- 102 -


STEP 7: Assess Improvement Strategies

- 103 -


7.1 Were reasonable interventions undertaken to address causes/barriers identified through data analysis and QI processes undertaken?

Describe Interventions:

• Unbundle service provider contracts into separate contracts and package by program division: F&C, A/OA, CJS, IBH, and SHS.

• Redesign the Contracting Service Request (CSR) Form, digitize it and make it available in DocuSign.

• Create a tracking sheet to show current renewal status of all FY18 SD/MHSA service contracts and make it available online/SharePoint site to all pilot project participants. Include the following status categories for each contract:

1. CSR Received

2. Exhibit A’s Received

3. Exhibit B’s Received

4. Drafting Contract

5. Contract's Unit Review of Draft Contract

6. Finance's Unit Review of Draft Contract

7. Program's Unit Review of Draft Contract

8. With Contractor/Agency for Signature

9. With County Counsel/OBA for Signature

10. With BSHD Department Head for Signature

11. With General Fund Financial Services Director and HHS CFO for signature

12. With HHS CEO for signature (if applicable)

13. Procurement (if applicable)

14. Finalized for BOS Meeting

15. With HHS Accounts Payable Team

☐ Met

☐ Partially Met

☐ Not Met


Interventions were focused on the business processes; no interventions were focused on the consumer benefit.

- 104 -


• Request the County Office of Budget Analysis (OBA) to consider BHSD’s FY18 SD/MHSA contracts for the County’s Master Contract List (MCL).


- 105 -


STEP 8: Review Data Analysis and Interpretation of Study Results

- 106 -


8.1 Was an analysis of the findings performed according to the data analysis plan?

This element is “Not Met” if there is no indication of a data analysis plan (see Step 6.5)

☐ Met

☐ Partially Met

☐ Not Met

☐ Not Applicable


There are no consumer outcomes evaluated.

Graphs and charts were included in the submission document as well as a narrative of the data analysis.

Average Approval Turnaround Time of FY2018 SD/MHSA CSRs:

In the past, the original CSR Form did not have a date field for the CSR submission date, and therefore BHSD was unable to track the submission date to determine the average approval turnaround time (TAT) of CSRs and assess performance from year to year. With the new SD/MHSA CSR Form, the CSR submission date field is included along with the approval date of the CSR which enabled BHSD to determine and calculate the approval TAT per form, calculate the average TAT during the pilot project, as well as TAT by program division level.

For the FY2018 SD/MHSA contract renewal process, the average approval TAT for the 61 SD/MHSA contracts was three days, see attached file for details. In the past, based on staff’s experience CSR approval took weeks and sometimes months but as illustrated the pilot project surpassed the two-week CSR approval TAT goal. In addition, the majority of the CSRs were approved within two days as pictured here. At the top of the range, only one CSR reflected a 10-day approval TAT.

Number of Completed Contracts by Execution Type:

The County of Santa Clara has two contract execution types: MCL = Master Contract List or Board = Board of Supervisors. Agreements approved for the MCL get Delegation of Authority (DOA) and does not require a Legislative File (LF) for full execution of the contract while contracts that exceed $5 million require execution of the contract through the regular Board of Supervisors (BOS) process which requires an LF which is a lengthy process. With the unbundling of the service contracts as described in prior sections into five distinct program divisions, the Contracts Unit analyzed the updated maximum

- 107 -


financial obligation (MFO) for each contract and found the majority of the FY18 SD/MHSA contracts were below $5 Million MFO cap each which made them eligible for the MCL.

The MCL request submitted by BHSD was initially submitted to the Office of Budget Analysis (OBA) to ensure the requested contracts met MCL requirements. 48 of the 61 FY2018 SD/MHSA contracts were eventually approved for the MCL by the BOS on May 9, 2017: the BOS adopted a resolution delegating authority to Agency and/or Department Heads to implement the MCL for FY2018. The recommendation of the Contracts Unit to request MCL approval of the eligible SD/MHSA contracts in April 2017 helped facilitate the completion of the FY2018 SD/MHSA contracts by the target date since it largely minimized the number of contracts that had to go through the LF portion of the process. The other contracts-13 that required BOS approval were approved in June 2017.

Only 3 of the 61 SD/MHSA contracts were not executed at the start of the contract term date while the vast majority, 95percent of the contracts, were renewed on time. As illustrated, the FY2018 completion rate is again closer to the 94percent completion rate achieved for the FY2016 Term 1 SD/MHSA renewal process and within the 90-100percent target goal of the pilot project. Refer to the excel file for details.

Percent of SD/MHSA contracts executed and effective at the start of FY2018:

Of the three contracts executed after the start of the fiscal year: one contract was with the City of San Jose and required BHSD to follow the contract approval process required by the City, which involved the approval of the contract by their City Council; the contract item could not be included in the agenda until after the start of the fiscal year. As for the other two contracts, the signature process took longer than expected: one with the service provider-vendor and other with County Counsel.

- 108 -


8.2 Were the PIP results and findings presented accurately and clearly?

Are tables and figures labeled? ☐ Yes ☐ No

Are they labeled clearly and accurately? ☐ Yes ☐ No

☐ Met

☐ Partially Met

☐ Not Met

☐ Not Applicable


8.3 Did the analysis identify: initial and repeat measurements, statistical significance, factors that influence comparability of initial and repeat measurements, and factors that threaten internal and external validity?

Indicate the time periods of measurements: contract renewal process

Indicate the statistical analysis used: _________________________

Indicate the statistical significance level or confidence level if available/known: _______percent ______Unable to determine

☐ Met

☐ Partially Met

☐ Not Met

☐ Not Applicable


The MHP provided limited information regarding the factors that influence comparability and factors that threaten internal and external validity.

Although these may be implied, such as staff adhering to timelines and contract provider completion of contracts, it would benefit the MHP to include these for future elements to consider.

- 109 -


8.4 Did the analysis of the study data include an interpretation of the extent to which this PIP was successful and recommend any follow-up activities?

Limitations described:

Conclusions regarding the success of the interpretation:

Recommendations for follow-up:

☐ Met

☐ Partially Met

☐ Not Met

☐ Not Applicable


The focus was on the contracting process, this does not meet criteria without any consumer focus.

Overall, the FY2018 Contract Renewal Process Pilot Project improved the Department’s administration and execution of the SD/MHSA contracting process.

BHSD has significantly improved the completion rate of the execution of SD/MHSA CBO service provider contracts for FY2018 with the new approach and enhancements described in the prior sections. In addition, the SD/MHSA CSR approval TAT was quicker than expected. Although the Department did not have a TAT baseline, since this information was not previously recorded, the average TAT approval of the SD/MHSA CSR Forms during the pilot project was three days. In addition, 95percent of the FY2018 SD/MHSA service contracts were executed at the start of the contract term date. The FY2018 renewal completion rate is closer to the 94percent completion rate achieved during the FY2016 Term 1 SD/MHSA contract renewal process and within the 90-100percent target goal of the pilot project. Please refer to Item 8 section for additional details.

Follow-up activities have included these:

• Due to the successful implementation and ease of use of the digitized SD/MHSA CSR Form and updated CSR Form review/approval process, the Contracts Team and Project Lead developed a redesigned CSR Form for non-SD/MHSA service agreements for use in DocuSign which was implemented in Fall 2017.

• For the FY2019 SD/MHSA contract renewal process, BHSD plans to continue the approach utilized in this pilot project, encourage new recommendations from team members involved in the contracting work and promote continuous operational enhancements that would benefit the County’s

- 110 -


Medi-Cal beneficiaries-clients/consumers, ensuring service contracts are in place at the start of every fiscal year.

• In addition, the Contracts Team worked with SCVHHS Information Services Department to include minor adjustments to the SD/MHSA CSR Form based on feedback/input provided by staff/users of the new SD/MHSA CSR Form. The updated SD/MHSA CSR Form will be rolled out as part of the FY2019 SD/MHSA contract renewal process. The Contracts Team plans to review and update the SD/MHSA CSR Form periodically to ensure the functionality/purpose of the form is relevant, up-to-date, in collaboration with Programs, Finance, and Administration.

• The Department will continue to track CSR Form approval TAT and completion rates of SD/MHSA contract renewals by fiscal year and perform any necessary adjustments to the contracting process to ensure the Department is on track for timely execution of SD/MHSA contracts.


STEP 9: Assess Whether Improvement is “Real” Improvement

9.1 Was the same methodology as the baseline measurement used when measurement was repeated?

Ask: At what interval(s) was the data measurement repeated?

Were the same sources of data used?

Did they use the same method of data collection?

Were the same participants examined?

Did they utilize the same measurement tools?

☐ Met

☐ Partially Met

☐ Not Met

☐ Not Applicable


The Department was unable to set a baseline CSR turnaround time (TAT) at the start of the project. Prior to the pilot project’s rollout, the initial submission date of the CSR Form by the CSR requester was not previously recorded.

Through the pilot project, BHSD is now able to document/record CSR submission dates and going forward the team can use the CSR approval TAT results from the FY18 pilot project as a baseline for future renewal contracting processes and can assist the Department to set standards and expectations around turnaround times for future contract renewal activities and update/make improvements as needed.

- 111 -


9.2 Was there any documented, quantitative improvement in processes or outcomes of care?

Was there: ☒ Improvement ☐ Deterioration

Statistical significance: ☒ Yes ☐ No

Clinical significance: ☒ Yes ☐ No

☐ Met

☐ Partially Met

☐ Not Met

☐ Not Applicable


Overall, the FY2018 Contract Renewal Process Pilot Project improved the Department’s administration and execution of the SD/MHSA contracting process. BHSD has significantly improved the completion rate of the execution of SD/MHSA CBO service provider contracts for FY2018 with the new approach and enhancements described in the prior sections.

In addition, the SD/MHSA CSR approval TAT was quicker than expected. Although the Department did not have a TAT baseline, since this information was not previously recorded, the average TAT approval of the SD/MHSA CSR Forms during the pilot project was three days.

In addition, 95percent of the FY2018 SD/MHSA service contracts were executed at the start of the contract term date. The FY2018 renewal completion rate is closer to the 94percent completion rate achieved during the FY2016 Term 1 SD/MHSA contract renewal process and within the 90-100percent target goal of the pilot project. Please refer to Item 8 section for additional details.

- 112 -


9.3 Does the reported improvement in performance have internal validity; i.e., does the improvement in performance appear to be the result of the planned quality improvement intervention?

Degree to which the intervention was the reason for change:

☐ No relevance ☐ Small ☐ Fair ☒ High

☐ Met

☐ Partially Met

☐ Not Met

☐ Not Applicable


The formation of a core team with representation from each of the contributing areas: Programs-CYF and A/OA System of Care, Finance, and Contracts helped facilitate the renewal process, troubleshoot with fellow team members, and represent their areas needs/concerns throughout the process.

The redesign of the Contracting Services Request (CSR) Form to include only vital/relevant information helped facilitate the approval process of the CSR.

The implementation of the redesigned CSR Form in digitized format for use in DocuSign, a digital transaction management system that provides electronic signature technology for facilitating electronic exchanges of signed documents, enabled efficient tracking and review/approval of CSR Forms.

The unbundling of service provider contracts into separate contracts and package by program division: F&C, A/OA, CJS, IBH, and SHS, provided each program area better control over their specific program contracts and expedite execution of contracts.

Conducting various meetings related to the pilot project with all parties involved: pilot project team, executive team, and CBO contract providers ensured everyone - staff/CBOS were all aware of the goal of the pilot project and roles and responsibilities of each area.

- 113 -


9.4 Is there any statistical evidence that any observed performance improvement is true improvement?

☐ Weak ☐ Moderate ☒ Strong

☐ Met

☐ Partially Met

☐ Not Met

☐ Not Applicable


Through the pilot project, the Department was able to improve the operational efficiency in the administration and execution of SD/MHSA service contracts for FY2018: the approval turnaround time (TAT) of SD/MHSA CSR Forms during the pilot project was three days and 95percent of SD/MHSA contracts were in place at the start of FY2018.

This is the first time BHSD has been able to measure approval turnaround time (TAT) for each SD/MHSA CSR submission and calculate the TAT.

As illustrated, the FY2018 completion rate is again closer to the 94percent completion rate achieved for the FY2016 Term 1 SD/MHSA renewal process and within the 90-100percent target goal of the pilot project

9.5 Was sustained improvement demonstrated through repeated measurements over comparable time periods?

☐ Met

☐ Partially Met

☐ Not Met

☐ Not Applicable


Due to the productive timeline with successful results, this was the first data collection. BHSD plans to continue to track and analyze timely execution of SD/MHSA contracts annually and also perform any necessary adjustments to the process in collaboration with programs, finance and contract teams to ensure contracts are executed timely every fiscal year and in place for the County’s MHP beneficiaries and enrollees.


ACTIVITY 2: VERIFYING STUDY FINDINGS (OPTIONAL)


Were the initial study findings verified (recalculated by CalEQRO) upon repeat measurement?

☐ Yes

☒ No

- 114 -


ACTIVITY 3: OVERALL VALIDITY AND RELIABILITY OF STUDY RESULTS: SUMMARY OF AGGREGATE VALIDATION FINDINGS

Conclusion:

This PIP was determined not to meet the criteria for a PIP secondary to the lack of consumer outcomes and the limited benefit to the consumer. The details were discussed on-site at the time of the review. The MHP was encouraged to continue to consult at the earliest stage with EQRO to determine the validity of its new PIP for the next review cycle.

The MHP was able to demonstrate a successful PIP in a brief time with greatly improved results as far as the contract process.

Recommendations:

Due to the success of the contract renewal process and its efficiencies the MHP intends to continue the efforts of the PIP.

The MHP demonstrated successful results and is encouraged that the MHP intends to continue these efforts at the improved processes for contract renewal.

Check one: ☐ High confidence in reported Plan PIP results ☐ Low confidence in reported Plan PIP results

☐ Confidence in reported Plan PIP results ☐ Reported Plan PIP results not credible

☐ Confidence in PIP results cannot be determined at this time

FY17 18 MEDI-CAL SPECIALTY MENTAL HEALTH EXTERNAL … and... · MHP Location — Santa Clara MHP...

Documents

Transcript of FY17 18 MEDI-CAL SPECIALTY MENTAL HEALTH EXTERNAL … and... · MHP Location — Santa Clara MHP...