Post on 18-Aug-2020
FY17–18 MEDI-CAL SPECIALTY MENTAL HEALTH
EXTERNAL QUALITY REVIEW
SANTA CLARA MHP FINAL REPORT
Behavioral Health Concepts, Inc.
5901 Christie Avenue, Suite 502
Emeryville, CA 94608
info@bhceqro.com
www.caleqro.com
855-385-3776
Prepared for:
California Department of
Health Care Services (DHCS)
Review Dates:
January 23-25, 2018
- 2 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
TABLE OF CONTENTS SANTA CLARA MHP SUMMARY OF FINDINGS .............................................................................. 5
Introduction ................................................................................................................................................................................ 5
Access ............................................................................................................................................................................................ 5
Timeliness.................................................................................................................................................................................... 6
Quality ........................................................................................................................................................................................... 6
Outcomes ..................................................................................................................................................................................... 6
INTRODUCTION...................................................................................................................................... 8
Validation of Performance Measures ............................................................................................................................... 8
Performance Improvement Projects ................................................................................................................................ 9
MHP Health Information System Capabilities .............................................................................................................. 9
Validation of State and County Consumer Satisfaction Surveys ........................................................................... 9
Review of Recommendations and Assessment of MHP Strengths and Opportunities ................................ 9
PRIOR YEAR REVIEW FINDINGS, FY16-17 .................................................................................. 11
Status of FY16–17 Review of Recommendations ..................................................................................................... 11
Changes in the MHP Environment and Within the MHP—Impact and Implications ................................ 15
PERFORMANCE MEASUREMENT .................................................................................................... 17
Total Beneficiaries Served ................................................................................................................................................. 18
Penetration Rates and Approved Claim Dollars per Beneficiary ....................................................................... 18
High-Cost Beneficiaries ....................................................................................................................................................... 22
Timely Follow-up After Psychiatric Inpatient Discharge ...................................................................................... 23
Diagnostic Categories .......................................................................................................................................................... 24
Performance Measures Findings—Impact and Implications .............................................................................. 25
PERFORMANCE IMPROVEMENT PROJECT VALIDATION ....................................................... 27
Santa Clara MHP PIPs Identified for Validation ........................................................................................................ 27
Clinical PIP—Welcome to Treatment Group.............................................................................................................. 29
Non-clinical PIP—2018 Contract Renewal Process ................................................................................................ 31
PIP Findings—Impact and Implications ...................................................................................................................... 32
PERFORMANCE AND QUALITY MANAGEMENT KEY COMPONENTS ................................... 33
Access to Care ......................................................................................................................................................................... 33
Timeliness of Services ......................................................................................................................................................... 35
Quality of Care ........................................................................................................................................................................ 36
Key Components Findings—Impact and Implications .......................................................................................... 41
CONSUMER AND FAMILY MEMBER ............................................................................................... 45
FOCUS GROUPS ..................................................................................................................................... 45
Consumer/Family Member Focus Group 1 ................................................................................................................ 45
Consumer/Family Member Focus Group 2 ................................................................................................................ 46
Consumer/Family Member Focus Group Findings—Implications ................................................................... 47
- 3 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
INFORMATION SYSTEMS REVIEW ................................................................................................. 49
Key Information Systems Capabilities Assessment (ISCA) Information Provided by the MHP ........... 49
Summary of Technology and Data Analytical Staffing ........................................................................................... 50
Current Operations ............................................................................................................................................................... 50
Priorities for the Coming Year ......................................................................................................................................... 51
Major Changes Since Prior Year ...................................................................................................................................... 51
Other Significant Issues ...................................................................................................................................................... 52
Plans for Information Systems Change ........................................................................................................................ 53
Current Electronic Health Record Status ..................................................................................................................... 53
Personal Health Record ...................................................................................................................................................... 54
Medi-Cal Claims Processing .............................................................................................................................................. 54
Information Systems Review Findings—Implications .......................................................................................... 55
SITE REVIEW PROCESS BARRIERS ................................................................................................. 57
CONCLUSIONS ....................................................................................................................................... 58
Strengths and Opportunities ............................................................................................................................................ 58
Recommendations................................................................................................................................................................. 61
ATTACHMENTS .................................................................................................................................... 62
Attachment A—On-site Review Agenda ...................................................................................................................... 63
Attachment B—Review Participants ............................................................................................................................. 64
Attachment C—Approved Claims Source Data ......................................................................................................... 69
Attachment D—PIP Validation Tool .............................................................................................................................. 70
- 4 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
LIST OF TABLES AND FIGURES Table 1: MHP Medi-Cal Enrollees and Beneficiaries Served in CY16, by Race/Ethnicity ................... 18
Table 2: High-Cost Beneficiaries ................................................................ Error! Bookmark not defined.
Table 3: PIPs Submitted by MHP ............................................................................................................. 27
Table 4: PIP Validation Review ............................................................................................................... 28
Table 5: PIP Validation Review Summary .............................................................................................. 29
Table 6: Access to Care Components ...................................................................................................... 33
Table 7: Timeliness of Services Components ......................................................................................... 35
Table 8: Quality of Care Components...................................................................................................... 37
Table 9: Distribution of Services, by Type of Provider ......................................................................... 49
Table 10: Summary of Technology Staff Changes ................................................................................... 50
Table 11: Summary of Data Analytical Staff Changes ............................................................................. 50
Table 12: Primary EHR Systems/Applications ...................................................................................... 51
Table 13: EHR Functionality .................................................................................................................... 53
Table 14: MHP Summary of Short Doyle/Medi-Cal Claims ...................... Error! Bookmark not defined.
Table 15: Summary of Top Three Reasons for Claim Denial ................... Error! Bookmark not defined.
Figure 1A: Overall Average Approved Claims per Beneficiary, CY14-16 Error! Bookmark not defined.
Figure 1B: Overall Penetration Rates, CY14-16 ........................................ Error! Bookmark not defined.
Figure 2A: Foster Care Average Approved Claims per Beneficiary ......... Error! Bookmark not defined.
Figure 2B: Foster Care Penetration Rates, CY14-16 ................................. Error! Bookmark not defined.
Figure 3A: Latino/Hispanic Average Approved Claims per Beneficiary, CY14-16 Error! Bookmark not
defined.
Figure 3B: Latino/Hispanic Penetration Rates, CY14-16 ......................... Error! Bookmark not defined.
Figure 4A: 7-day Outpatient Follow-up and Rehospitalization Rates ..... Error! Bookmark not defined.
Figure 4B: 30-day Outpatient Follow-up and Rehospitalization Rates ... Error! Bookmark not defined.
Figure 5A: Beneficiaries Served, by Diagnostic Categories, CY16 ........... Error! Bookmark not defined.
Figure 5B: Total Approved Claims by Diagnostic Categories, CY16 ........ Error! Bookmark not defined.
- 5 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
SANTA CLARA MHP SUMMARY OF FINDINGS
Beneficiaries Served in Calendar Year 2016 22,901
MHP Threshold Language(s) Spanish, Vietnamese, Mandarin, Tagalog, Cantonese, Farsi
MHP Size— Large
MHP Region — Bay Area
MHP Location — Santa Clara
MHP County Seat — San Jose
Introduction
This is a large Bay Area county and provides services and programs for a multilingual, culturally
diverse community. With key stakeholders, the Mental Health Plan (MHP) has come closer to their
vision to fully integrate the values, infrastructure, processes, services, and supports to better meet
the needs of consumers and their families. It achieved combining its departments into the
Integrated Behavioral Health Organization and continues to merge models of integrated care and
apply best practices throughout its system.
During the fiscal year 2017-2018 (FY17-18) review, California External Quality Review
Organization (CalEQRO) reviewers found the following overall significant changes, efforts, and
opportunities related to access, timeliness, quality, and outcomes of the MHP and its contract
provider services. Further details and findings from EQRO-mandated activities are provided in this
report.
Access
The MHP revised its processes for access to services with closer integration of the mental health
Access Call Center and the substance use treatment services (SUTS) Gateway Center. It comes
closer to integration by moving the Access Call Center to the same Cisco call center software that
the Gateway Center uses. With this process, a new streamlined screening tool has been
implemented that reduces the time needed to refer a client.
This simplifies transfer between the two call centers and gives staff access to information gathered
by each call center, yet they remain physically and operationally separate organizational units.
Improvements are underway to fully integrate these units.
- 6 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
The MHP contracted with both MTM Consulting and Harbage Consulting to review both the quality
and fiscal impacts of its system. This work continues with the intention of creating efficiencies and
seamless service delivery.
Timeliness
Various services focused on delivering expedited care that includes the expansion of the jail
diversion continuum of care for timely treatment post incarceration, launching four new mobile
crisis teams, and a crisis residential unit (CSU) that expanded timely response to urgent conditions.
The focus on the Welcome to Treatment group as a component of the clinical PIP was successful in
strategies to improve timeliness and subsequently improve timely triage and identification of
consumer needs.
The rate for the 7-day follow-up appointment post hospitalization has improved, and the 30-day
follow-up appointment shows greater improvement. This may be a result of an inpatient liaison to
assure consumers receive a follow-up session.
Quality
The MHP reflects dedication to quality from leadership to line staff. The system continues to adapt
its procedures to further advance a quality management organization. The challenges remain in the
ability to compete for, and hire, qualified personnel to keep up with its intended progress. The
system itself is growing and continues to experience uncertainty in these times of economic,
political and fiscal challenges.
Multiple initiatives continue to provide the foundation for on-going services to targeted groups.
Examples include the waiver for the Organized Delivery Services (ODS) for substance use disorders,
the Whole Person Care (WPC) grant, the Coordination Care Reform (CCR) policies and procedures
modeled after the Katie A. implementation, and its collaboration with jail diversion provide access
for these vulnerable populations.
The Epic HealthLink and Netsmart myAvatar conversion plan for simultaneous implementations
are ambitious undertakings given the nominal level of available technical and project management
staffing in the MHP for its information systems.
The MHP makes extensive use of data to inform decision-making and in the last year has begun
producing information dashboards that are informative and widely available.
Outcomes
To further support its operational integration across all facets, the MHP successfully developed the
infrastructure at the executive level to manage the system changes. It recruited and hired four new
executive positions for Administrative Services, Adult/Older Adult Services, Children, Youth and
Family Services and Quality Management.
- 7 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
The MHP has created a Consumer and Family Affairs Division Director position reporting to the
deputy director. The position requirements include lived experience and will provide leadership,
oversight, and direction for peer staff and peer programs.
An 18-month to 24-month project plan and implementation timeline for the adult/older adult
redesign is under development.
The MHP expanded project plans and developed implementation timelines for the new Mental
Health Services Act (MHSA) innovation projects which included individual placement and support
employment, faith-based supports, psychiatric emergency response teams (PERTS) and adolescent
treatment initiatives.
Outcome tools are used and intended to be included within the electronic health record (EHR) as
well which will advance access to data regarding consumer progress.
- 8 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
INTRODUCTION The United States Department of Health and Human Services (HHS), Centers for Medicare and
Medicaid Services (CMS) requires an annual, independent external evaluation of State Medicaid
Managed Care programs by an External Quality Review Organization (EQRO). External Quality
Review (EQR) is the analysis and evaluation by an approved EQRO of aggregate information on
quality, timeliness, and access to health care services furnished by Prepaid Inpatient Health Plans
(PIHPs) and their contractors to recipients of State Medicaid managed care services. The CMS (42
CFR §438; Medicaid Program, External Quality Review of Medicaid Managed Care Organizations)
rules specify the requirements for evaluation of Medicaid managed care programs. These rules
require an on-site review or a desk review of each Medi-Cal Mental Health Plan.
The State of California Department of Health Care Services (DHCS) contracts with 56 county Medi-
Cal MHPs to provide Medi-Cal covered specialty mental health services (SMHS) to Medi-Cal
beneficiaries under the provisions of Title XIX of the federal Social Security Act.
This report presents the FY17-18 findings of an EQR of the Santa Clara MHP by the California
External Quality Review Organization, Behavioral Health Concepts, Inc. (BHC).
The EQR technical report analyzes and aggregates data from the EQR activities as described below:
Validation of Performance Measures1
Both a statewide annual report and this MHP-specific report present the results of CalEQRO’s
validation of eight mandatory performance measures (PMs) as defined by DHCS. The eight PMs
include:
• Total beneficiaries served by each county MHP;
• Total costs per beneficiary served by each county MHP;
• Penetration rates in each county MHP;
• Count of Therapeutic Behavioral Services (TBS) beneficiaries served compared to the
4percent Emily Q. Benchmark2;
• Total psychiatric inpatient hospital episodes, costs, and average length of stay (LOS);
1 Department of Health and Human Services. Centers for Medicare and Medicaid Services (2012). Validation of Performance
Measures Reported by the MCO: A Mandatory Protocol for External Quality Review (EQR), Protocol 2, Version 2.0,
September, 2012. Washington, DC: Author. 2 The Emily Q. lawsuit settlement in 2008 mandated that the MHPs provide TBS to foster care children meeting certain at-risk
criteria. These counts are included in the annual statewide report submitted to DHCS, but not in the individual county-level
MHP reports.
- 9 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
• Psychiatric inpatient hospital 7-day and 30-day rehospitalization rates;
• Post-psychiatric inpatient hospital 7-day and 30-day Specialty Mental Health Services
(SMHS) follow-up service rates; and
• High-Cost Beneficiaries (HCBs), incurring approved claims of $30,000 or higher during a
calendar year.
Performance Improvement Projects3
Each MHP is required to conduct two Performance Improvement Projects (PIPs)—one Clinical and
one Non-clinical—during the 12 months preceding the review. The PIPs are discussed in detail later
in this report.
MHP Health Information System Capabilities4
Using the Information Systems Capabilities Assessment (ISCA) protocol, CalEQRO reviewed and
analyzed the extent to which the MHP meets federal data integrity requirement for Health
Information Systems (HIS), as identified in 42 CFR §438.242. This evaluation included a review of
the MHP’s reporting systems and methodologies for calculating PMs.
Validation of State and County Consumer Satisfaction Surveys
CalEQRO examined available consumer satisfaction surveys conducted by DHCS, the MHP, or its
subcontractors.
CalEQRO also conducted 90-minute focus groups with beneficiaries and family members to obtain
direct qualitative evidence from beneficiaries.
Review of Recommendations and Assessment of MHP Strengths
and Opportunities
The CalEQRO review draws upon prior years’ findings, including sustained strengths, opportunities
for improvement, and actions in response to recommendations. Other findings in this report
include:
3 Department of Health and Human Services. Centers for Medicare and Medicaid Services (2012). Validating Performance
Improvement Projects: Mandatory Protocol for External Quality Review (EQR), Protocol 3, Version 2.0, September 2012.
Washington, DC: Author. 4 Department of Health and Human Services. Centers for Medicare and Medicaid Services (2012). EQR Protocol 1: Assessment
of Compliance with Medicaid Managed Care Regulations: A Mandatory Protocol for External Quality Review (EQR), Protocol
1, Version 2.0, September 1, 2012. Washington, DC: Author.
- 10 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
• Changes, progress, or milestones in the MHP’s approach to performance management —
emphasizing utilization of data, specific reports, and activities designed to manage and
improve quality.
• Ratings for key components associated with the following three domains: access, timeliness,
and quality. Submitted documentation as well as interviews with a variety of key staff,
contracted providers, advisory groups, beneficiaries, and other stakeholders inform the
evaluation of the MHP’s performance within these domains. Detailed definitions for each of
the review criteria can be found on the CalEQRO website, www.caleqro.com.
- 11 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
PRIOR YEAR REVIEW FINDINGS, FY16-17 In this section, the status of last year’s (FY16-17) recommendations are presented, as well as
changes within the MHP’s environment since its last review.
Status of FY16–17 Review of Recommendations
In the FY16-17 site review report, the CalEQRO made a number of recommendations for
improvements in the MHP’s programmatic and/or operational areas. During the FY17-18 site visit,
CalEQRO and MHP staff discussed the status of those FY16-17 recommendations, which are
summarized below.
Assignment of Ratings
Met is assigned when the identified issue has been resolved.
Partially Met is assigned when the MHP has either:
• Made clear plans and is in the early stages of initiating activities to address the
recommendation; or
• Addressed some but not all aspects of the recommendation or related issues.
Not Met is assigned when the MHP performed no meaningful activities to address the
recommendation or associated issues.
Key Recommendations from FY16-17
Recommendation #1: Develop a project plan to integrate the two Access Call Centers as the
current model is not an effective solution to support community-based capacity challenges.
Implement the plan as soon as practical since the MHP identified the lack of community-based
capacity as the number one priority.
Status: Met
• The MHP brought the mental health Access Call Center and the SUTS Gateway Center
closer to integration by moving the Access Call Center to the same Cisco call center
software that the Gateway Center uses. This simplifies transfer of callers between the
two call centers and gives them access to information gathered by each call center, but
they remain physically and operationally separate organizational units. The MHP
recognized more work is required to integrate these units.
• In addition, a new streamlined screening tool has been implemented which reduces the
amount of time needed by Access Call Center staff to refer a client.
- 12 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
• The MHP has contracted with MTM Consulting, a nationally recognized behavioral
health consulting group, with expertise in call center operations and same day access.
Executive leaders, division directors and center managers from both call centers have
been working collaboratively with MTM Consulting to evaluate operational and
performance data.
• While still in the evaluation phase, several potential areas for improvement have been
identified and preliminary planning for implementation has begun. The preliminary
improvement plans include:
o Developing policy and procedures for same day access into clinics
o Changes to scheduling management of clinical staff
o Updating no-show and cancellation policies
o Reducing documentation processes
Recommendation #2: Implementation of Practice Management System Solution (PMSS) electronic
health record addressing two critical elements. Those elements are:
• Knowledge of interoperability standards: The project will require that both Information
Technology (IT) and behavioral health program staff have advance knowledge of
interoperability (two-way exchange of data between systems) standards.
• Communication plan: As contract providers serve over 85 percent of specialty mental
services, a communication plan that shares technical data exchange requirements with
contractors or their IS vendors is a must.
Status: Not Met
• The MHP has selected its PMSS and focused its efforts on the two-way exchange of data
between contract providers and the MHP. Not yet resolved is the development of a
realistic timeline for a communications plan that shares technical data exchange
information with contract providers.
• With a target date of July 1, 2018, for initiating electronic data interchange (EDI)
between MHP and its numerous contract providers, it was not clear that
interoperability standards had been effectively communicated to their management,
their technical staff, and their system vendors.
• Since EDI involves a two-way information exchange, there needs to be a formal process
to ensure not just the MHP, but also the contract providers and their vendors are ready
to participate in an EDI go-live.
• In addition, issues of confidentiality and privacy for mental health, substance use clients,
and its contract providers will need to be addressed.
- 13 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
• Operational meetings continue with the CBOs to coordinate change management
activities and address communication strategies for staff.
Recommendation #3: Continue to work with its county Human Resources division to expedite and
expand recruitment strategies for increasing line staff hires for the vacant positions, and to
consider an equitable step increase for clinical supervision, critical to its expansion efforts.
Status: Partially Met
• The MHP has been successful in filling line staff positions.
• The MHP has requested a third step for clinical staff, recognizing the importance of
clinical supervision and a career pathway for clinicians. The MHP has drafted a job
specification and the executive team met with the deputy county executive for the
employee services agency and the deputy director for human resources, yet this issue
remains under review. Confirmation of this class has not been established.
• Clinical supervisors remain without a formal classification for their positions and are
assigned these as other duties. As a result, without the clinical supervisor class, the
supervisory experience cannot be used as qualifying experience when applying for
management positions.
Recommendation #4: Investigate the feasibility of using telepsychiatry services to augment
medication support or other tele-health services as an improvement option.
Status: Met
• A provider survey was developed and distributed by staff to determine contract
provider interest in telehealth.
• A telepsychiatry policy has been drafted and a pilot telehealth project with one of the
larger contract providers continues. This remains in the planning and development
stage to support expansion efforts.
• Telepsychiatry appears to still be in the investigative stage as a pilot implementation
functioning within the MHP was not developed.
Recommendation #5: Resume discussions with the county Human Resources division addressing
the need for a peer employee career ladder and consider full-time benefitted positions with job
mobility.
Status: Met
• The MHP has created a Consumer and Family Affairs Division Director position
reporting to the deputy director. The position requirements include lived experience.
The position will provide leadership, oversight, and direction for peer staff and peer
programs.
- 14 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
• The MHP director has met with the county consumer employees, who have researched
job classifications from other counties that support a peer career ladder.
• Discussions about a peer employee career ladder are reported between the MHP and
the county human resources, but no concrete steps towards a peer employee career
ladder have, so far, come from those discussions.
• Consumer staff are playing pivotal roles in the development of the Peer Respite Center
through
• WPC, and the development of a Peer Navigation program is planned for 2018.
- 15 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Changes in the MHP Environment and Within the MHP—Impact
and Implications
Discussed below are any changes since the last CalEQRO review that were identified as having a
significant effect on service provision or management of those services. This section emphasizes
systemic changes that affect access, timeliness, and quality, including any changes that provide
context to areas discussed later in this report.
Access to Care
• The MHP implemented two adult community-based crisis residential facilities. It is
preparing to launch county-operated mobile crisis teams and open a CSU in January
2018.
• The MHP implemented the Cisco phone system in the mental health Access Call Center
to link with SUTS Gateway Center to improve call response time and capture call data
for service analysis. However, the two call centers, as their names imply, remain
operationally and physically separate organizational units.
• The leadership participated in the Santa Clara Valley Health and Hospital Systems WPC
implementation of system-wide care coordination planning.
• Expanded initiatives include the Jail Diversion continuum of care to transition
consumers from custody into the community at the new Muriel Wright service complex.
• Children’s services integrated the community care reform program principles in
partnership with social services and juvenile probation.
• The executive team convened a professional shortage workgroup to identify innovative
approaches to recruit and retain licensed staff.
Timeliness of Services
• The expansion of the jail diversion continuum of care expedited timely treatment post
incarceration.
• The MHP launched four new mobile crisis teams and a CSU which expanded timely
response to urgent conditions.
Quality of Care
• The department developed its Integrated Organizational Structure combining multiple
departments.
- 16 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
• Management recruited and filled four new executive positions for Administrative
Services, Adult/Older Adult Services, Children, Youth and Family Services and Quality
Management.
• Leadership created a Consumer and Family Affairs Division Director position.
• The MHP selected Epic HealthLink and Netsmart myAvatar as the county electronic
health record and launched the discovery process.
• Data services developed system and operations dashboards to ensure compliance with
the CMS much-anticipated Mega-Rule update and to track and analyze data for
continuous quality improvement.
• The management took lead roles in the Santa Clara Valley Health and Hospital System’s
WPC implementation of system-wide care coordination planning.
Consumer Outcomes
• The MHP developed and launched an 18 to 24-month project plan and implementation
timeline for the adult/older adult redesign.
• Access activities integrated the mental health Access Call Center and substance use
SUTS Gateway Center on the same call center system and piloted one same-day access
clinic.
• Service expansion included plans and implementation timelines for MHSA innovation
projects. These were individual placement and supportive employment, faith-based
supports, PERTS and headspace which is based on the successful Australian headspace
model to treat emerging mental health problems in adolescents.
• The MHP created a Consumer and Family Affairs Division Director position.
- 17 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
PERFORMANCE MEASUREMENT As noted above, CalEQRO is required to validate the following PMs as defined by DHCS:
• Total beneficiaries served by each county MHP;
• Total costs per beneficiary served by each county MHP;
• Penetration rates in each county MHP;
• Count of TBS Beneficiaries Served Compared to the 4percent Emily Q. Benchmark (not
included in MHP reports; this information is included in the Annual Statewide Report
submitted to DHCS);
• Total psychiatric inpatient hospital episodes, costs, and average LOS;
• Psychiatric inpatient hospital 7-day and 30-day rehospitalization rates;
• Post-psychiatric inpatient hospital 7-day and 30-day SMHS follow-up service rates; and
• HCBs incurring $30,000 or higher in approved claims during a calendar year.
HIPAA Suppression Disclosure:
Values are suppressed to protect confidentiality of the individuals summarized in the data sets
where beneficiary count is less than or equal to eleven (*). Additionally, suppression may be
required to prevent calculation of initially suppressed data, corresponding penetration rate
percentages (n/a); and cells containing zero, missing data or dollar amounts (-).
- 18 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Total Beneficiaries Served
Table 1 provides detail on beneficiaries served by race/ethnicity.
Starting with CY16 performance measures, CalEQRO has incorporated the ACA Expansion data in
the total Medi-Cal enrollees and beneficiaries served. See Attachment C, Table C1 for the
penetration rate and approved claims per beneficiary for just the CY16 ACA Penetration Rate and
Approved Claims per Beneficiary.
Penetration Rates and Approved Claim Dollars per Beneficiary
The penetration rate is calculated by dividing the number of unduplicated beneficiaries served by
the monthly average enrollee count. The average approved claims per beneficiary served per year
is calculated by dividing the total annual dollar amount of Medi-Cal approved claims by the
unduplicated number of Medi-Cal beneficiaries served per year.
Regarding calculation of penetration rates, the Santa Clara MHP uses the same method used by
CalEQRO.
Race/Ethnicity
Average Monthly
Unduplicated
Medi-Cal Enrollees
% Enrollees
Unduplicated
Annual Count
of Beneficiaries
Served
% Served
White 58,907 13.0% 5,011 21.9%
Latino/Hispanic 168,992 37.2% 8,751 38.2%
African-American 15,085 3.3% 1,236 5.4%
Asian/Pacific Islander 159,007 35.0% 4,346 19.0%
Native American 1,602 0.4% 166 0.7%
Other 50,363 11.1% 3,391 14.8%
Total 453,954 100% 22,901 100%
Table 1: Santa Clara MHP Medi-Cal Enrollees and Beneficiaries Served in CY16,
by Race/Ethnicity
The total for Average Monthly Undupl icated Medi-Cal Enrol lees i s not a di rect sum of the averages above i t.
The averages are ca lculated independently.
- 19 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Figures 1A and 1B show 3-year (CY14-16) trends of the MHP’s overall approved claims per
beneficiary and penetration rates, compared to both the statewide average and the average for
large MHPs.
- 20 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Figures 2A and 2B show 3-year (CY14-16) trends of the MHP’s foster care (FC) approved claims per beneficiary and penetration rates, compared to both the statewide average and the average for large MHPs.
- 21 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Figures 3A and 3B show 3-year (CY14-16) trends of the MHP’s Latino/Hispanic approved claims
per beneficiary and penetration rates, compared to both the statewide average and the average for
large MHPs.
- 22 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
High-Cost Beneficiaries
Table 2 compares the statewide data for High-Cost Beneficiaries for CY16 with the MHP’s data for
CY16, as well as the prior two years. HCBs in this table are identified as those with approved claims
of more than $30,000 in a year.
See Attachment C, Table C2 for the distribution of the MHP beneficiaries served by approved claims
per beneficiary (ACB) range for three cost categories: under $20,000; $20,000 to $30,000; and
those above $30,000.
MHP YearHCB
Count
Total
Beneficiary
Count
HCB %
by
Count
Average
Approved
Claims
per HCB
HCB
Total Claims
HCB % by
Approved
Claims
Statewide CY16 19,019 609,608 3.12% $53,215 $1,012,099,960 28.90%
CY16 2,362 22,901 10.31% $56,592 $133,669,749 45.85%
CY15 1,315 22,707 5.79% $54,684 $71,909,163 35.71%
CY14 1,149 17,407 6.60% $56,985 $65,475,939 39.47%
Table 2: Santa Clara MHP High-Cost Beneficiaries
Santa Clara
- 23 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Timely Follow-up After Psychiatric Inpatient Discharge
Figures 4A and 4B show the statewide and MHP 7-day and 30-day outpatient follow-up and
rehospitalization rates for CY15 and CY16.
- 24 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Diagnostic Categories
Figures 5A and 5B compare the breakdown by diagnostic category of the statewide and MHP
number of beneficiaries served and total approved claims amount, respectively, for CY16.
MHP self-reported percent of consumers served with co-occurring (substance abuse and mental
health) diagnoses: 6.0percent.
- 25 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Performance Measures Findings—Impact and Implications
Access to Care
• Overall penetration rates are slightly above the rates for other large counties and the
state average.
• By race, the White Medi-Cal enrollees are 13 percent of the total enrolled population,
and 21.9 percent of beneficiaries.
• Asian Pacific Islanders are 35 percent of the enrolled population, and only 19 percent of
the beneficiaries.
• Penetration rates for children ages 0-5 and youth ages 6-17 are significantly above large
county and state averages.
• Statewide, high-cost beneficiaries (HCB) comprise 3.12 percent of the beneficiaries
served. For the MHP these comprise 10.31 percent of the beneficiaries served.
• For CY16, HCBs account for 45.85 percent of approved Medi-Cal claims in Santa Clara
County while statewide HCBs account for 28.90 percent of approved claims.
• By race or ethnicity, for all categories of beneficiaries, the percentage of HCBs served is
at least 2.2 times the statewide figure. For African-Americans, it is 3.1 times the
statewide percentage. For Hispanics, the percentage of HCBs in Santa Clara is 5.1 times
the statewide percentage for Hispanics.
• Hispanics are 38 percent of the beneficiaries served, but the account for 46 percent of
the MHP’s HCB approved claims.
• By age group, the percentage of HCB served is 6.3 times the statewide percentage for
the 0-5 age group and decreases steadily as age increases until it reaches 1.5 times the
statewide percentage for the 60 years and older group.
Timeliness of Services
• While the 7-day outpatient follow-up rate post-hospitalization has improved from about
30 percent in CY15 to more than 40 percent in CY16, and is above the state average rate,
the rehospitalization rate remains above the State average rehospitalization rate.
• The 30-day outpatient follow-up rate post-hospitalization follows the same pattern as
the 7-day outpatient follow-up rate post-hospitalization and the rehospitalization rate
again remains higher than the state average.
- 26 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Quality of Care
• Looking at the entire beneficiary population in Santa Clara County, average claims per
beneficiary (ACB) is more than twice what it is in other large counties and the statewide
average; and ACB has increased significantly in CY16 from both CY14 and CY15.
• The disparity between large county and the statewide ACB is greatest for children ages
0-5 and youth ages 6-17.
• The ACB for Hispanic and African-American beneficiaries is significantly higher than
other racial and ethnic groups.
• Based on eligibility category, approved claims per beneficiary range for three cost
categories: under $20,000; $20,000 to $30,000, and those above $30,000; it is the three
categories specific to children that have the highest ACB, with foster care having the
highest ACB by a wide margin.
• By service category, there are only two categories with ACB above $10,000 for large
counties and statewide: TBS and Day Treatment. For the MHP, four service categories
have ACB above $11,000: Inpatient ($15,580), Residential Services ($11,280), Mental
Health Services ($10,566), and TBS ($11,465).
• Beneficiaries receiving more than 15 services make up 39.57 percent of the statewide
beneficiary population; in Santa Clara, 51.29 percent of beneficiaries receive more than
15 services.
Consumer Outcomes
• Both 7-day and 30-day rehospitalization rates increased in CY16 from their comparative
CY15 rates and slightly exceed statewide averages.
- 27 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
PERFORMANCE IMPROVEMENT PROJECT
VALIDATION A Performance Improvement Project (PIP) is defined by CMS as “a project designed to assess and
improve processes and outcomes of care that is designed, conducted, and reported in a
methodologically sound manner.” The Validating Performance Improvement Projects Protocol
specifies that the EQRO validate two PIPs at each MHP that have been initiated, are underway, were
completed during the reporting year, or some combination of these three stages. DHCS elected to
examine projects that were underway during the preceding calendar year.
Santa Clara MHP PIPs Identified for Validation
Each MHP is required to conduct two PIPs during the 12 months preceding the review. CalEQRO
reviewed and validated two MHP-submitted PIPs, as shown below.
Table 3 lists the findings for each section of the evaluation of the PIPs, as required by the PIP
Protocols: Validation of Performance Improvement Projects.5
Table 3: PIPs Submitted by Santa Clara MHP
PIPs for Validation
# of PIPs PIP Titles
Clinical PIP 1 Welcome to Treatment Group
Non-clinical PIP 1 Contract Renewal Process
Table 4, on the following page, provides the overall rating for each PIP, based on the ratings given to
the validation items: Met (M), Partially Met (PM), Not Met (NM), Not Applicable (NA), Unable to
Determine (UTD), or Not Rated (NR).
5 2012 Department of Health and Human Services, Centers for Medicare and Medicaid Service Protocol 3 Version 2.0,
September 2012. EQR Protocol 3: Validating Performance Improvement Projects.
- 28 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Table 4: PIP Validation Review
Item Rating
Step PIP Section Validation Item Clinical Non-
clinical
1 Selected Study
Topics 1.1 Stakeholder input/multi-functional team M NR
1.2 Analysis of comprehensive aspects of enrollee needs, care, and services
M NR
1.3 Broad spectrum of key aspects of enrollee care and services M NR
1.4 All enrolled populations M NR
2 Study Question 2.1 Clearly stated M NR
3 Study 3.1 Clear definition of study population M NR
Population 3.2 Inclusion of the entire study population M NR
4 Study
Indicators 4.1 Objective, clearly defined, measurable indicators M NR
4.2 Changes in health status, functional status, enrollee satisfaction, or processes of care
M NR
5 Sampling Methods
5.1 Sampling technique specified true frequency, confidence interval and margin of error
NA NR
5.2 Valid sampling techniques that protected against bias were employed
NA NR
5.3 Sample contained sufficient number of enrollees NA NR
6 Data Collection 6.1 Clear specification of data M NR
Procedures 6.2 Clear specification of sources of data M NR
6.3 Systematic collection of reliable and valid data for the study population
M NR
6.4 Plan for consistent and accurate data collection M NR
6.5 Prospective data analysis plan including contingencies M NR
6.6 Qualified data collection personnel M NR
7 Assess
Improvement Strategies
7.1 Reasonable interventions were undertaken to address causes/barriers
M NR
8 Review Data Analysis and
8.1 Analysis of findings performed according to data analysis plan M NR
Interpretation
of Study Results 8.2 PIP results and findings presented clearly and accurately M NR
8.3 Threats to comparability, internal and external validity PM NR
8.4 Interpretation of results indicating the success of the PIP and follow-up
M NR
9 Validity of
Improvement 9.1 Consistent methodology throughout the study NA NR
9.2 Documented, quantitative improvement in processes or outcomes of care
NA NR
9.3 Improvement in performance linked to the PIP NA NR
9.4 Statistical evidence of true improvement NA NR
9.5 Sustained improvement demonstrated through repeated measures
NA NR
- 29 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Table 5 provides a summary of the PIP validation review.
Table 5: PIP Validation Review Summary
Summary Totals for PIP Validation Clinical PIP Non-clinical
PIP
Number Met 19 0
Number Partially Met 1 0
Number Not Met 0 0
Number Applicable (AP)
(Maximum = 28 with Sampling; 25 without Sampling) 20 0
Overall PIP Rating ((#Met*2) + (#Partially Met))/(AP*2) 97.5 % 0 %
Clinical PIP—Welcome to Treatment Group
The MHP presented its study question for the clinical PIP as follows:
“Will the development and implementation of Welcome to Treatment at Downtown Behavioral
Health (DTBH) increase timely access to care from 18.1 days to 14 days by December 2017,
engagement, and overall consumer perception within specialty behavioral health outpatient
services?”
Date PIP began: December 2015
Status of PIP: Completed
The goal of the PIP was to engage and improve timely delivery of services to individuals waiting for
specialty behavioral health treatment at Downtown Behavioral Health.
The problem facing the Santa Clara System of Care was increased demand in relation to service
provision capabilities, thereby creating delays to services. Research provided showed that delays in
providing timely access to services for consumers seeking specialty mental health reduces quality
of care and poor consumer outcomes.
To address improvements to timeliness, the MHP focused primarily on its welcoming strategies to
immediately attend to consumer requests and to enhance engagement. It initiated several
strategies in its Welcome to Treatment Group.
The MHP identified activities such as these to accomplish its goals:
- 30 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
• To improve outcomes of care and support engagement in the treatment services, a
consumer attends a group session in which information is provided in a welcome packet
that consumers can utilize when the clinic is closed.
• To familiarize the consumer with the treatment environment, to encourage consumer
willingness to engage, and to reduce the no-show rates, a clinician and a peer support
worker review the welcome packet and provide other information regarding the available
wellness and recovery services.
• The clinician determines/inquires whether a need for medication refill prior to intake is
warranted and whether to schedule a medication evaluation date.
• Consumers are guided on the process of obtaining medication from Urgent Care if necessary
prior to psychiatrist’s evaluation.
• Face-to-face interactions between consumers and their treatment team are coordinated to
encourage engagement and treatment success. The clerical staff will schedule an intake
appointment during the group and the consumer meets with assigned clinician the same
day if possible.
• Consumers are given a tour of Zephyr, a self-help center which is located on the grounds of
DTBH.
• The client perception data developed by the DTBH is collected from consumers using the
Welcoming Group Satisfaction survey with a 5-point Likert scale (strongly disagree,
disagree, neutral, agree, and strongly agree).
Results were reported which reflect the following:
• Improved timeliness to access for consumers showed a 4 percent improvement for
average calendar days to initial services at DTBH.
• A 7 percent improvement for consumers receiving initial services within 14 days.
• No statistical evidence on the data analysis table on engagement was collected,
however, anecdotal information noted engagement between staff and consumers has
improved as evidenced by decreased no-show rates and increase in utilization of
services at the clinic.
• The survey showed 81.8 percent of consumers found the Welcoming Group to be useful
and 89.1 percent reported they were able to receive adequate information.
• Future activities will consider providing a short therapy group to enhance consumer
coping skills.
The results from this PIP may be generalizable to other locations and the MHP is expanding the
program to another county clinic and potentially two contract providers.
- 31 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Relevant details of these issues and recommendations are included within the comments found in
the PIP validation tool.
The technical assistance provided to the MHP by CalEQRO consisted of discussing the value of
replicating this PIP in other settings. The value was emphasized and the unexpected results in the
ability to triage consumer needs was highlighted. The MHP determined this would conclude the
submission of this PIP and it would consider a new PIP for the next review cycle. Topics were not
discussed at this session. The MHP was encouraged to consult with EQRO early and often during
PIP formulations.
Non-clinical PIP—2018 Contract Renewal Process
The MHP presented its study question for the non-clinical PIP as follows:
“Will the implementation of the FY 2018 Contract Renewal Process Pilot Project, which includes the
following enhancements as listed below, improve timely execution of the county Short
Doyle/Mental Health Services Act (SD/MHSA) service contracts?
• The redesign of the Contracting Services Request (CSR) form to include only vital and
relevant information will help facilitate the approval process of the CSR in a timely manner.
• The implementation of the redesigned CSR form for SD/MHSA service contracts in digitized
format for use in DocuSign, a digital transaction management system that provides
electronic signature technology for facilitating electronic exchanges of signed documents,
will enable efficient tracking and review/approval of CSRs.
• The unbundling of service provider contracts into separate contracts and package by
program division: Family and Child, Adult and Older Adult, Criminal Justice Services,
Integrated Behavioral Health, and Supportive Housing Services specific program contracts
and expedite execution of contracts.
• Conducting various meetings related to the pilot project with all parties involved: pilot
project team, executive team, and CBO contract providers will ensure everyone knows the
goal of the pilot project and roles and responsibilities of each area.
• The creation a status tracking sheet to show current renewal status of all FY18 SD/MHSA
service contracts and making it available online/SharePoint site to all staff involved in the
pilot project so that everyone is aware and current on the status of contracts throughout the
renewal process.”
Date PIP began: February 2017
Status of PIP: Submission determined not to be a PIP (not rated)
- 32 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
The project’s aim, specific to SD/MHSA mental health service contracts only, was to pilot a new
approach to increase timely delivery of executed contracts, ensuring they are in place at the start of
the fiscal year for consumers.
It was determined that the non-clinical PIP did not meet the criteria for a PIP secondary to the lack
of consumer benefit. Although commendable efforts were done in the MHP’s process and business
procedures, there were no stated outcomes for the consumer.
This PIP does not reflect a data collection and analysis of comprehensive aspects of enrollee needs,
care and services. The improvements to the contract renewal procedures did prove successful,
however, it was deemed to be limited to the business practice change. This did not result in
producing outcomes that were consumer related. Services to consumers were not impacted if
contracts were not renewed timely. To qualify, the PIP has to be connected to consumer outcomes
and, as stated, this is a process change.
However, this PIP did reconstruct the method for timely completion of the MHP contract
agreements. The MHP managed to complete over 95 percent of its contracts with organizational
providers and introduced new efficiencies to do this. Discussion occurred on-site to emphasize
using EQRO for technical assistance early in its PIP discussions and throughout the development of
its PIPs.
In conclusion, this activity does not meet the requirements for a PIP; rather, it serves as more of a
practice improvement which focuses on contract renewal dates to validate effectiveness.
Relevant details of these issues and recommendations are included within the comments found in
the PIP validation tool for instructional purposes only.
The technical assistance provided to the MHP by CalEQRO consisted of discussion of this PIP and
the elements that would have helped in the PIP process. Topics were not discussed at the review
and TA was offered although it was not scheduled. The MHP was encouraged to consult with EQRO
early and often during PIP formulations and the MHP recognized this option.
PIP Findings—Impact and Implications
Access to Care
• The average calendar days for access to services was improved by 4 percent.
• Consumers were scheduled without delay for a Welcome to Treatment group upon
initial request.
Timeliness of Services
• Timeliness to the initial service within the 14-day standard was improved by 7 percent.
- 33 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
• Earlier initial appointments can lead to effective engagement for consumer benefit.
Quality of Care
• A secondary benefit emerging from this PIP allowed staff to immediately triage
consumer needs, potentially lending to earlier symptom interventions.
• Improved outcomes for timely access to care coupled with consumer satisfaction led the
MHP to replicate this approach in other clinic settings.
Consumer Outcomes
• Consumer satisfaction was reported at an improved rate and led to consideration of a
treatment group.
• Replicating this approach provides expanded quality for consumers.
PERFORMANCE AND QUALITY MANAGEMENT
KEY COMPONENTS CalEQRO emphasizes the MHP’s use of data to promote quality and improve performance.
Components widely recognized as critical to successful performance management include an
organizational culture with focused leadership and strong stakeholder involvement, effective use of
data to drive quality management, a comprehensive service delivery system, and workforce
development strategies that support system needs. These are discussed below, along with their
quality rating of Met (M), Partially Met (PM), or Not Met (NM).
Access to Care
Table 6 lists the components that CalEQRO considers representative of a broad service delivery
system that provides access to consumers and family members. An examination of capacity,
penetration rates, cultural competency, integration, and collaboration of services with other
providers forms the foundation of access to and delivery of quality services.
Table 6: Access to Care Components
Component Quality Rating
1A Service accessibility and availability are reflective of cultural competence principles and practices
M
- 34 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
The MHP strived to increase the penetration rate for services delivered with a stated goal of at
least one percentage point for each ethnic group. The MHP collects, reports, and reviews data
from regions, program level, age groups, and racial/ethnic groups to reflect culturally informed
activities.
The MHP has assumed the lead role in orchestrating WPC initiative with behavioral health
integrated activities. A group of 40-50 peer support workers convened with a kick-off for the
development of peer respite site that will launch later this year.
The Santa Clara Early Child Development Training Academy demonstrated the MHP’s
commitment to early years’ treatment for the 0-5 age group. Clinical staff attend monthly
seminars and use reflective supervision principles to apply this knowledge in their practice.
1B Manages and adapts its capacity to meet consumer service needs PM
The MHP screens all foster youth for mental health symptoms of which 1700 screenings were
completed that included the Katie A. subclass. Since February 2016 the MHP opened Katie A.
services ICC and IHBS to any youth that met medical necessity with full scope Medi-Cal. County
agencies use the same screening form and referrals can be made from child welfare, probation,
and community services.
The MHP designed an electronic capacity tracking tool to improve appointment availability
management at the Access Call Center, worked with rapid cycle teams and contracted with the
MTM Consulting firm to implement improvement ideas.
Recent contracts with Harbage Consulting will provide a new foundation for service delivery.
The MHP is moving from its “no wrong door” approach to a more centralized access system with
same day treatment as the goal.
The MHP continued work from its prior PIP focused on consumer engagement by defining the
current performance indicator for engagement to include at least one face-to-face visit within 30
days following hospitalization.
The MHP leadership was well-aware of network adequacy as its biggest challenge. The local area
poses high cost housing and a shortage of qualified professionals in community mental health
programs that must compete with lucrative private sector health care systems. Employee
retention at the contract providers is a continuing challenge in this well-funded system.
1C Integration and/or collaboration with community-based services to improve access
M
A partnership with Stanford University led to psychiatry residents working with the jail inmates.
The Valley Health system intends to use peer support workers at its call center and integrate the
systems for a coordinated and seamless entry to services.
With over 80 percent of its services delivered under contract, the MHP continues to engage with
its broad-based contract providers. Having implemented performance indicators at its
- 35 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
contracted providers, the MHP’s Pay for Success program completed the first year with positive
data and extended this incentive program for five years.
The MHP initiated monthly Clinical Record meetings to review data dashboards with
stakeholders for compliance with standards of care and review the State all-county letters and
discuss the impact.
Timeliness of Services
As shown in Table 7, CalEQRO identifies the following components as necessary to support a full-
service delivery system that provides timely access to mental health services. This ensures
successful engagement with consumers and family members and can improve overall outcomes,
while moving beneficiaries throughout the system of care to full recovery.
Table 7: Timeliness of Services Components
Component Quality Rating
2A Tracks and trends access data from initial contact to first appointment PM
The MHP tracks and reports the time from initial contact to first kept clinical appointment. The
MHP’s standard is 14 days, with most recent data indicating a mean of 14.7 days for adult
services, and 15.9 days for children and youth. Attainment of standard is 62.5 percent for adults
and 56.7 percent for children and youth.
The MHP has initiated multiple improvements focused on centralizing its access points with the
intention of increasing positive results for this metric yet it continues to struggle with
scheduling of initial appointments.
2B Tracks and trends access data from initial contact to first psychiatric appointment
M
The MHP utilizes a 30-day initial psychiatry contact standard. The adult services mean is 28.2
days, with 33.1 days the children and youth result. Attainment of standard is 54 percent for
adults and 44.7 percent for children and youth.
The MHP may wish to target initial psychiatry services with an improvement activity if the
interval between initial contact to first appointment continues to remain low in the achievement
of its standard.
2C Tracks and trends access data for timely appointments for urgent conditions
M
- 36 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
The MHP maintains a one-day standard for response to urgent conditions. It reports 100 percent
achievement of this for both the adult and children’s services. Given this is a large size county,
this achievement warrants recognition.
2D Tracks and trends timely access to follow-up appointments after hospitalization
M
The MHP tracks and reports a standard to follow-up post hospitalization of seven days. Of the
total 1,850 hospitalizations, 1,359 were adults and 491 were children. The adult services mean
is 7.3 days and for children it reports 5.3 days. Achievement of this is reported at 41.6 percent
for adults and 68.8 percent for children.
2E Tracks and trends data on rehospitalizations M
There were 216 rehospitalizations within 30 days of discharge from inpatient care. The MHP
reported a rehospitalization rate of 12.2 percent for adult services and 10.2 percent for children.
The data is inclusive of all hospital psychiatric admissions.
2F Tracks and trends no-shows M
The MHP tracks its no-show data for clinical staff and psychiatrists with a goal not to exceed 25
percent. The adult services indicated an 18.8 percent no-show rate for psychiatrists and a 4.5
percent no-show rate for clinicians. Children’s services indicated a 12.9 percent no-show rate for
psychiatrists and a 5.4 percent rate no-show rate for clinicians.
As discussed with the MHP on-site, it may benefit the MHP to revisit the 25 percent standard to
evaluate whether it aligns with its goals and is comparable to the no-show standard in other
similar-sized counties. Given the demand for psychiatric service, the no-show rates potentially
reflect expensive unused capacity.
Quality of Care
In Table 8, CalEQRO identifies the components of an organization that is dedicated to the overall
quality of care. Effective quality improvement activities and data-driven decision making require
strong collaboration among staff (including consumer/family member staff), working in
information systems, data analysis, clinical care, executive management, and program leadership.
Technology infrastructure, effective business processes, and staff skills in extracting and utilizing
data for analysis must be present in order to demonstrate that analytic findings are used to ensure
overall quality of the service delivery system and organizational operations.
- 37 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Table 8: Quality of Care Components
Component Quality Rating
3A Quality management and performance improvement are organizational priorities
M
The planning from its recent contracts with MTM Consulting and Harbage Consulting will guide
system revisions and improved efficiency. This will be the foundation for improvements in care
and delivery system quality. The goal is to achieve National Council of Quality Assurance (NCQA)
accreditation, a nationally recognized leader with a commitment to comprehensive research and
dedication to quality in healthcare.
The MHP created and filled its new Quality Management Director position. The MHP provided an
evaluation of the prior year and a current Quality Improvement Work Plan (QIWP). The MHP
had the foresight to include an infrastructure improvement plan to support its efforts. The
Quality Management redesign is underway to reflect the Quality Circle structure, which supports
system integration and an increased focus on providing value to its beneficiaries.
The QIWP had measurable, quantifiable goals focused on consumer outcomes. For example, the
focus of QI Grand Rounds in June and November 2017 was to increase client engagement with
goals focused on improved timely access.
The MHP created the Practice Standards Manual, intended to build a bridge between
documentation regulations and recovery-oriented best clinical practices. Monthly QI meetings
with all providers occur to discuss State notices and practice guidelines. This is a model of
developing the “why” regarding documentation with best practice and recovery orientation
thus, integrating Quality Improvement and Quality Assurance activities.
Among the MHP’s current initiatives, the implementation of its EHR system and conversion to
electronic data interchange with its contract providers, there are serious concerns about the
viability of its very aggressive schedule in the face of potentially significant constraints in the
technologically-skilled and experienced resources to support and sustain the implementation. .
Although this falls short for its quality care, the MHP has stated it has taken measures since the
review, these were not in place at the on-site review. CalEQRO anticipates evaluating these
activities at the next review cycle. This is a vital component of the continued quality care system.
3B Data are used to inform management and guide decisions M
Data reporting is extending to various components of the work force within the system of care.
The Data Decision team initiated a dashboard for adult and older adults. It continues to track
discharge success via the Milestones of Recovery Scale (MORS) and produces reporting for this
for all providers.
- 38 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Table 8: Quality of Care Components
Component Quality Rating
Data are published as a web link to provider staff. Training is planned for February 2018. This
process lines up with the quality management goals, streamlines the access to data, and allows
for stakeholders to manipulate the data to suit their needs.
With the new EHR system, all providers will enter data into Epic HealthLink. The data entered
into Epic HealthLink from county mental health and SUD providers will be transmitted
electronically to Netsmart myAvatar system for claims processing. Contract providers will
transmit their claiming data electronically from their local EHR to Netsmart myAvatar.
3C Evidence of effective communication from MHP administration, and stakeholder input and involvement on system planning and implementation
\M
Leadership established a comprehensive QI staff training program and opportunities to share
best practices to infuse quality care throughout the system.
The MHP combined the QI division and the Decision Support division and established its Quality
Circles which encouraged a unified cross-functional work style.
The MHP has made great efforts to communicate with staff in the past two years. This includes:
monthly staff bulletins, monthly all manager meetings, bi-annual all staff meetings, a staff-led
communication work group and continues to elicit stakeholder feedback for MHSA planning.
These are two-way exchanges of information that provide staff with an opportunity to suggest
ideas such as developing guidelines for e-mail reduction, track the number of meeting staff are
required to attend and develop a calendar of events important to the MHP. Each of these ideas is
being followed up and implemented at the time of the review.
The addition of the four new executive mangers has increased the visibility and staff access to
executive level decision making. Overall, these efforts improved transparency significantly with
staff being informed of changes, promotional opportunities, fiscal and political issues. In
addition, staff have been invited to participate in a number of improvement activities designed
to enhance their working conditions and increase job satisfaction.
The MHP contracted with Resource Development Associates (RDA) to conduct the MHSA System
Evaluation and conducting surveys/focus groups which included participation from CBO
partners/contract providers.
Contrasted against the cohesion developed among the executive team, the staff, family members,
and contract providers indicated information flow continues to be primarily one directional.
This may be an unintended result of multiple projects simultaneously absorbing the executive
team’s time. Stakeholder perception may be slow to feel the effects of the value leadership
- 39 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Table 8: Quality of Care Components
Component Quality Rating
places on its input. To its credit, the MHP leadership values its stakeholder relationships on
multiple levels and its efforts will continue to permeate through the system although this takes
sustained effort and time which is the MHP’s commitment. CalEQRO anticipates witnessing this
infusion at the next review.
3D Evidence of a systematic clinical continuum of care PM
The MHP continues to expand its service components with the SUTS, WPC initiative and its
Integrated Behavioral Health consolidation.
Enhanced efforts to foster integration of mental health and substance use treatment services are
evolving with its Practice Standards manual and QI workgroup addressing both units. A
documentation manual incorporated both requirements and continually emphasized the Golden
Thread linking assessment, diagnosis, treatment and documentation.
The family members and providers felt transitions in service were increasingly difficult citing
system barriers for providers to coordinate with one another. The parent focus group noted
transitions repeatedly resulted in large service gaps leading to significant challenges for both the
youth and family.
Most surprising is the continued challenge of establishing a bona fide clinical supervisory
position. Current practice accounts for clinical supervision yet it is an assigned responsibility
for licensed clinical staff who assume a lead role for their colleagues training. The current
practice also means that a supervisory assignment is not qualifying experience when applying
for management positions. This effectively negates opportunities within the career ladder for
some of the MHP’s most experienced clinicians and unintentionally contributes to feeling
undervalued.
Leadership values its assigned clinical supervisor employees and continues to advocate for
dedicated clinical supervisor positions, yet it appears the final decision lies beyond their direct
control. This is a situation atypical to other comparable MHPs and potentially undermines the
MHPs effectiveness. Several meetings have been held with the county Employee Services Agency
(ESA) deputy’s executive staff. These meetings have been received positively and the MHP has
been assured that the creation of a new supervisory position will be given top priority. This
specific situation is broader than the department, as similar codes supervisory codes are used by
other county departments. In addition, ESA is conducting a county-wide classification study of
the Senior Healthcare Program Manager classification, to review roles and functions and
consider impaction with other supervisory and management positions. CalEQRO anticipates a
status update at the next review cycle.
- 40 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Table 8: Quality of Care Components
Component Quality Rating
3E Evidence of consumer and family member employment in key roles throughout the system
PM
The MHP created a new position entitled Consumer and Family Affairs Division Director with an
open recruitment in process. The current mental health Peer Support Workers play a vital role
in many services especially within the contract providers. These staff have gained experience in
coordinating care, working with clients and providing home-linked services.
Consumers expressed regret that a clear career ladder with promotional opportunities were
limited. It may benefit the MHP for mental health Peer Support Workers to have an opportunity
to compete promotionally for entry level mental health worker positions. With the initiation of
the new Consumer and Family Affairs Division Director, the MHP could be taking the next steps
to a clear career path for the consumer employee track.
3F Consumer run and/or consumer driven programs exist to enhance wellness and recovery
PM
The wellness centers Zephyr and Narvaez Self-Help Centers are consumer-run recovery-
oriented programs. The hours are limited to Tuesday to Friday from 9:00 a.m. to 3:30 p.m. These
sites averaged 40 persons daily. The new welcoming process includes material about the Self-
Help Centers and often a visit to a Self-Help Center is conducted with a Peer Support Worker.
Given there exist two Self-Help Centers with limited hours, consumers and staff shared that an
additional center could benefit consumers in areas that do not yet have a wellness center.
Consideration for the addition a third center might bring increased attendance in other regions.
The MHP improved access to treatment by providing an alternative step-down level of care for
adults with its crisis residential treatment facility.
The MHP developed and promoted a Wellness Level of Care across it community-based
organizational providers.
3G Measures clinical and/or functional outcomes of consumers served M
Consumer focus groups reported that all persons were engaged in their own treatment planning.
Improvement goals are noted in the QIWP for successful outcomes. For example, the MHP noted
its goal to increase the number of adult clients who successfully discharge, as measured by
outcome scores at discharge of at least 35 percent and it supports Family and Child Services to
maintain its level at a minimum of 58 percent.
Several focus groups among staff, providers, and family members indicated that the system
appears impacted with quality affected due to the efforts to meet consumer demand within the
- 41 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Table 8: Quality of Care Components
Component Quality Rating
limits of available resources. This was evidenced by increased time to initial screenings and
large clinician caseloads potentially impacting the care provided to consumers.
Continued use of the MORS and Child and Adolescent Needs and Strengths (CANS) report scores
assist in the level of care decisions for consumers. Other instruments across the behavioral
health system are used such as the Patient Health Questionnaire 9 (PHQ-9) to monitor and
assess the severity of depressive symptoms, and the screening for anxiety with the Generalized
Anxiety Disorders (GAD) tool.
3H Utilizes information from Consumer Satisfaction Surveys M
The MHP solicits feedback on the statewide Consumer Perception Survey and conducts this
twice a year for a two-week period.
Various specialized surveys are distributed based on the program needs. The MHP developed a
Peer Support Satisfaction Survey, which collected information from clients on their satisfaction
with peer support services.
The data is published by the Decision Support team and shared among stakeholders.
Key Components Findings—Impact and Implications
Access to Care
• Access to appointments has been improved by the implementation of the CISCO call
center phone system at the Access Call Center. It is the same system used by the SUTS
Gateway Center. The two systems are linked for information sharing purposes which
facilitates transfers between the two organizational units when needed. Call response
times and accurate call data capture for service analysis have both improved.
• The MHP extended its continuum of care with the addition of two adult community-
based crisis residential facilities, deployment of mobile crisis teams, and the opening of
a community-based crisis stabilization unit.
• The MHP has contracted with MTM Consulting, a nationally recognized behavioral
health consulting group, with expertise in call center operations and same day
appointment access.
• The MHP has also contracted with Harbage Consulting. It plans to use this service to
assess organizational performance and upgrade communication, work efficiency, and
- 42 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
decision-making processes and align these with its mission and philosophy to meet
capacity.
• There are reports and some evidence of the MHP lacking sufficient capacity to meet
service demand. Anecdotal reports of an adult waiting list for initial appointments
reaching up to hundreds of beneficiaries are an area of concern.
Timeliness of Services
• The MHP developed an alert tracking and data management system to flag when a client
has been admitted to hospitals for a smoother transition of care from inpatient to
outpatient care.
• Timely service delivery was expanded with additional services for urgent response time
with the CSU and the mobile crisis teams.
• Screening all foster youth for mental health symptoms in a timely manner using a
collaborative approach with all stakeholders reflects the commitment to this target
group.
Quality of Care
• The MHP developed its Integrated Organizational Structure combining multiple
departments for efficiencies in service delivery.
• The MHP successfully recruited and filled four new executive positions for
Administrative Services, Adult/Older Adult Services, Children, Youth and Family
Services, and Quality Management.
• After selecting Epic HealthLink and Netsmart myAvatar as the County electronic health
record, the MHP launched its initiative to establish electronic data exchange among
contract providers.
• The MHP continued to develop system and operational dashboards to ensure
compliance with the CMS regulatory standards and to track and analyze data for
continuous quality improvement.
• Through its Quality Circles, the MHP intends to restructure components of the Quality
Management Committee to help build the capacity of staff and providers to initiate
quality improvement actions.
• The MHP Golden Thread treatment process starts with engaging the client so that an
assessment can be conducted that leads to formulating a treatment plan for services and
documenting progress towards treatment goals.
• A train-the-trainer workshop in collaboration with Public Health was offered to build
capacity of staff in the use of quality management tools and methods.
- 43 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
• The turnover rate for clinicians in the contract provider network raises concerns about
timeliness and quality of care. A high staff vacancy rate directly impacts accessibility
and timeliness of services.
• Substantial improvements have been made to provide consistent messages and
communication efforts regarding documentation standards. The Quality Assurance
Work Group established a forum for raising and discussing documentation issues with
staff and contract providers. The agenda, decision-making, and minutes are shared from
these activities. Stakeholders are encouraged to be present.
• Dispersing information to all stakeholders simultaneously may be indicated with the
intent to strengthen the link between leadership, contract providers and other
stakeholder groups.
Consumer Outcomes
• Dedication of resources to consumers was reflected with the creation of the Consumer
and Family Affairs Division Director position.
• Review of the transitions between providers to determine if successful discharge
procedures may warrant improvements.
• The MHP hired a full-time hospital liaison for adult services that is designated to track
client admissions and assist with a warm hand-off to outpatient clinics.
• Continued advocacy for Peer Support Workers to have an opportunity to promotionally
compete for entry level mental health worker positions may be indicated. Continued
advocacy for Peer Support Workers to have positions available within the county and to
have an opportunity to compete for entry level county mental health worker positions
would provide increased options.
• Consumer run self-help wellness centers have limited hours in comparison to the MHP
business hours. Zephyr is open Tuesday thru Friday from 9 a.m. to 3 p.m. The MHP has
indicated it has added additional resources to ensure current operational hours are met.
• The Self-Help Centers are important community wellness resources that can only
contribute to the extent they are properly staffed and resourced. The MHP has
indicated staffing has been added to support the operations.
• Consumers receive a quantity of services as demonstrated by beneficiary retention rates
showing more than 50 percent of beneficiaries receiving more than 15 services and
average claims per beneficiary (ACB) in some categories more than twice the large
county or State averages.
• It is not clear if these higher retention rates and ACBs are leading to better client
outcomes. The slightly higher rehospitalization rates in Santa Clara compared to the
- 44 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
statewide rates leave open the question of whether the higher costs are delivering the
desired results.
• It is unclear at this point whether this is evidence of an appropriate level of care for the
population served or an indicator that consumers are being retained in treatment
because of limited resources to refer to at the next lower level of care. Staffing capacity
potentially affects new consumers’ access to on-going treatment. This is a critical
question that requires attention.
- 45 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
CONSUMER AND FAMILY MEMBER
FOCUS GROUPS
CalEQRO conducted two 90-minute focus groups with consumers and family members during the
site review of the MHP. As part of the pre-site planning process, CalEQRO requested focus groups
with 8 to 10 participants each, the details of which can be found in each section below.
The consumer/family member focus group is an important component of the CalEQRO site review
process. Obtaining feedback from those who are receiving services provides significant information
regarding quality, access, timeliness, and outcomes. The focus group questions are specific to the
MHP being reviewed and emphasize the availability of timely access to care, recovery, peer support,
cultural competence, improved outcomes, and consumer and family member involvement. CalEQRO
provides gift certificates to thank the consumers and family members for their participation.
Consumer/Family Member Focus Group 1
• A culturally diverse group of adult beneficiaries with a mix of new consumers who have
initiated/utilized services within the past 12 months.
• This group was held at the MHP Administrative offices located at 1075 E Santa Clara Street,
San Jose.
Number of participants: 6
There were no participants who entered services within the past year.
Participants described their experience as the following:
• This group of consumers was very enthusiastic about the services they were receiving.
Consumers stated accessing services was straightforward and supportive. All described
being involved in their treatment planning.
• Participants received multiple services including medications support, counseling, and
case management.
• The majority of the participants had sessions with a psychiatrist either monthly or
every other month and a therapist on a regular basis with either twice monthly or every
three weeks. Most were also involved in group therapy and appreciated the group
process. They reported feeling support from the therapist as well as from their peers in
the group.
• In addition, all consumers had a case manager and were most enthused about the
support they gave to consumers. They described receiving assistance with housing,
- 46 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
transportation, paperwork, appointment reminders, food, and life-skills. As no one was
aware of services to assist them with either education or employment, they felt that this
resource might be helpful for them.
• Regular communication and home visits occurred with the case managers. Several
attended peer support groups and found them helpful. Group members mentioned that
having an ethnically matched case manager increased trust.
• No one had used the crisis service in the last year; however, all would recommend using
this service when necessary.
General comments regarding service delivery that were mentioned included the following:
• The consensus was services were consistently available and scheduled on a regular
basis.
• Staff provided support and indicated recovery was attainable.
• All felt that they had improved as a result of the services received.
Recommendations for improving care included the following:
• Increase availability of the group therapy sessions.
• Navigators would be useful at the onset of treatment to assist consumers in building
confidence in using the system services.
• Improve timeliness for the initial psychiatry appointment.
Interpreter used for focus group 1: No
Consumer/Family Member Focus Group 2
• A culturally diverse group of parents/caregivers of child/youth beneficiaries with a mix of
new consumers who have initiated/utilized services within the past 12 months.
• This group was held at the Kidscope Center located at 828 Bascom Avenue, San Jose.
Number of participants: 15
The participants who entered services within the past year described their experiences as the
following:
• Psychiatry services were provided seamlessly for the majority of participants.
• Initial entry into services was delayed and often confusing as to which service would be
provided.
- 47 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
General comments regarding service delivery that were mentioned included the following:
• Participants experienced different requirements and individual methods of paperwork
for each agency at initial entry.
• Group participants voiced concern about transfers from one agency to another
mentioning this resulting in undue delays in treatment.
• Treatment was thought to be provided as a package and consumers suggested this
potentially ended treatment prematurely due to limited symptom reduction.
Recommendations for improving care included the following:
• Provide standardized paperwork and procedures among the various agencies providing
care.
• Transition to a lower level of care gradually.
• Provide bridging to community referrals for support once treatment concludes.
• Review procedures for discharging consumers as youth may be inadvertently impacted.
Interpreter used for focus group 2: Yes Language(s): Spanish
Consumer/Family Member Focus Group Findings—
Implications
Access to Care
• The majority of participants indicated access to services was readily obtained.
• Transferring services often resulted in delays according to family members.
• Once service was obtained, case management was available.
Timeliness of Services
• Most of the participants received services in a timely manner and were served
consistently.
• Some experienced delays in change of providers.
Quality of Care
• Participants indicated staff were supportive and recognized recovery was possible.
- 48 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
• The majority indicated involvement in their treatment plans.
• Treatment was perceived to end prematurely for some participants.
• Entry paperwork varied among contract providers.
Consumer Outcomes
• Some participants were aware of a consumer survey and have submitted these in the
past.
• Participants were either involved in their own treatment plan or family members were
included.
• Group therapy for adults, often co-led by peers appeared to be beneficial.
- 49 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
INFORMATION SYSTEMS REVIEW Understanding an MHP’s information system’s capabilities is essential to evaluating its capacity to
manage the health care of its beneficiaries. CalEQRO used the written response to standard
questions posed in the California-specific ISCA, additional documents submitted by the MHP, and
information gathered in interviews to complete the information systems evaluation.
Key Information Systems Capabilities Assessment (ISCA)
Information Provided by the MHP
The following information is self-reported by the MHP through the ISCA and/or the site review.
Table 9 shows the percentage of services provided by type of service provider.
Table 9: Distribution of Services, by Type of Provider
Type of Provider Distribution
County-operated/staffed clinics 11 %
Contract providers 88 %
Network providers 1 %
Total 100 %
Percentage of total annual MHP budget dedicated to supporting information technology operations
(includes hardware, network, software license, IT staff): 3.73 percent
The budget determination process for information system operations is:
MHP currently provides services to consumers using a telepsychiatry application:
☐ Yes ☒ No ☐ In pilot phase
☒ Under MHP control
☐ Allocated to or managed by another County department
☐ Combination of MHP control and another County department or Agency
- 50 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Summary of Technology and Data Analytical Staffing
MHP self-reported technology staff changes (Full-time Equivalent [FTE]) since the previous
CalEQRO review are shown in Table 10.
MHP self-reported data analytical staff changes (in FTEs) that occurred since the previous CalEQRO
review are shown in Table 11.
The following should be noted with regard to the above information:
• Table 10 shows only FTEs directly employed by the MHP. The MHP also receives
information technology resources from the Health and Hospital System and Santa Clara
County IT, as well at vendors and consultants.
• The Epic HealthLink and Netsmart myAvatar implementation plan for simultaneous
implementations are ambitious undertakings given the level of available technical and
project management staffing in the MHP.
Current Operations
• The MHP makes extensive use of data to inform decision-making and in the last year has
begun producing information dashboards that are informative and widely available.
• The budget devoted to information systems is reported as 3.73 percent. This number
includes resources in the Health and Hospital System not directly under the control of
the MHP but is supporting its operations.
Table 10: Technology Staff
IS FTEs (Include Employees
and Contractors)
# of New FTEs
# Employees / Contractors Retired,
Transferred, Terminated
Current # Unfilled Positions
7 0 1 1
Table 11: Data Analytical Staff
IS FTEs (Include Employees
and Contractors)
# of New FTEs
# Employees / Contractors Retired,
Transferred, Terminated
Current # Unfilled Positions
2 2 0 0
- 51 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Table 12 lists the primary systems and applications the MHP uses to conduct business and manage
operations. These systems support data collection and storage, provide electronic health record
(EHR) functionality, produce Short-Doyle/Medi-Cal (SD/MC) and other third-party claims, track
revenue, perform managed care activities, and provide information for analyses and reporting.
Table 12: Primary EHR Systems/Applications
System/Application Function Vendor/Supplier Years Used
Operated By
Pro-Filer
Service recording, State reporting, billing, capacity management, assessment, referral, managed care
Co-Centrix 13 BHS IS
Diamond
Managed care service recording and billing
Dell 20 BHS IS
Epic HealthLink
EHR for BAP, EPS and BHS programs in FQHC clinics
Epic <1 HHS IS
Priorities for the Coming Year
• Implementation of Epic HealthLink clinical functions for MHP county-operated sites
(screening, scheduling, registration, clinical documentation, charge capture).
• Implementation of the Netsmart Practice Management for County and contract
providers (billing, claiming, reporting, QI and related functions).
• Preparing to initiate electronic exchange of service and billing data between contract
providers EHR systems and Netsmart myAvatar, effective July 2018.
Major Changes Since Prior Year
• Began the build for the implementation of the Epic HealthLink system.
• Began the build for the implementation of the Netsmart myAvatar system.
- 52 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
• Upgraded Cocentrix Pro-Filer from release 14 to release 15.
• Supported two Pay-For-Success projects, one called Project Welcome Home, an
innovative partnership approach to housing the homeless, and the other was to reduce
use of emergency psychiatric services and acute care services.
Other Significant Issues
• No currently active telepsychiatry implementation; still in the investigation stage.
• MHP self-reported co-occurring disorders rate is 6 percent, which is exceptionally low.
• High Cost Beneficiary (HCB) spending in some beneficiary groups reaches five or six
times statewide HCB percentages for those groups. This raises the question whether the
level of spending in these categories is driving outcomes that justify that level of
spending and whether that focus of spending is constraining spending in other areas of
need.
• MHP requested 4 additional FTEs to support Netsmart myAvatar and the contract
providers’ use of electronic data exchange with Netsmart myAvatar on a continuing
basis after the implementation is complete. This may prove to be a conservative request
although the MHP asserts it considers this adequate and will monitor staffing patterns
and adjust accordingly.”
• MHP requested six additional FTEs for continuing support of Epic HealthLink after the
implementation is complete.
• Netsmart is doing much of the work on the myAvatar build and data migration. Once the
implementation is complete, the vendor will be much less involved, leaving support and
maintenance to very limited local resources currently.
• The use of the integrated scheduling functionality in Epic HealthLink will be a huge
culture change for clinicians who are accustomed to having direct control of their own
appointment schedule in the current systems environment. This is a culture change that
is important to the success of Epic HealthLink implementation and, therefore, it needs to
be carefully managed. Clinical leadership needs to be especially clear and consistent in
their communication on this subject.
• The MHP stated when they move contract providers to the electronic data exchange
with Netsmart myAvatar, a steering committee previously used will be revived. Given
the stated target date of July 2018, to begin EDI with the contract providers, the revival
of that steering committee is overdue, and it should be engaged with all possible haste.
• At present, the contract provider’s awareness of this project seems limited to the broad
concepts and the target date. There does not seem to be an awareness of what
transactions will be exchanged, what exactly the contract providers and their vendors
- 53 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
need to do, in what order, and when they need to do it to make a coordinated transition
to electronic data exchange with Netsmart myAvatar.
• MHP staff described a detailed communications plan for this project yet it remains
unclear that the right information at the right level of detail is reaching the stakeholders
most responsible for making this a successful project.
Plans for Information Systems Change
• Currently implementing Epic HealthLink for clinical functionality.
• Currently implementing Netsmart myAvatar for billing and State reporting mandates.
Current Electronic Health Record Status
Table 13 summarizes the ratings given to the MHP for EHR functionality.
Table 13: EHR Functionality
Rating
Function System/Application Present Partially Present
Not Present
Not Rated
Alerts Epic (FQHC only) x
Assessments Pro-filer x
Care Coordination x
Document imaging/storage Pro-filer x
Electronic signature—consumer
x
Laboratory results (eLab) Epic (FQHC only) x
Level of Care/Level of Service
x
Outcomes x
Prescriptions (eRx) Epic (FQHC only) x
Progress notes Pro-filer x
Referral Management Pro-filer x
Treatment plans Pro-filer x
Summary Totals for EHR Functionality: 5 3 0 4
Progress and issues associated with implementing an electronic health record over the past year
are discussed below:
• No significant functional improvements were implemented to the legacy EHR (Pro-filer)
system during the past year.
- 54 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Consumer’s Chart of Record for county-operated programs (self-reported by MHP):
☐ Paper ☐ Electronic ☒ Combination
Personal Health Record
Do consumers have online access to their health records either through a Personal Health Record
(PHR) feature provided within the EHR, consumer portal, or third-party PHR?
☐ Yes ☒ No
If no, provide the expected implementation timeline.
☒ Within 6 months ☐ Within the next year
☐ Within the next two years ☐ Longer than 2 years
Medi-Cal Claims Processing
MHP performs end-to-end (837/835) claim transaction reconciliations:
If yes, product or application:
Pro-Filer Version 15.3.2
Method used to submit Medicare Part B claims:
☐ Paper ☐ Electronic ☒ Clearinghouse
Table 14 summarizes the MHP’s SDMC claims.
Table 15 summarizes the most frequently cited reasons for claim denial.
Number
Submitted
Gross Dollars
Billed
Number
Denied
Dollars
Denied
Percent
Denied
Gross Dollars
Adjudicated
Claim
Adjustments
Gross Dollars
Approved
706,691 $508,351,045 26,105 $20,659,099 4.06% $487,691,946 $221,389,285 $266,302,661
Table 14: Santa Clara MHP Summary of CY16 Short Doyle/Medi-Cal Claims
Includes services provided during CY16 with the most recent DHCS process ing date of May 19, 2017.
The statewide average denia l rate for CY2016 was 4.48 percent.
Change to the FFP reimbursement percentage for ACA a id codes delayed a l l cla im payments between the months of January-May 2017.
☒ Yes ☐ No
- 55 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
• Denied claim transactions listed with the denial reason description as “Missing,
incomplete, invalid ICD-10 diagnosis or condition” are generally re-billable within the
State claims resubmission guidelines.
Information Systems Review Findings—Implications
Access to Care
• The MHP’s current systems do not offer the mental health Access Call Center or SUTS
Gateway Center on-line real-time information about available appointment slots at
MHP, including contract provider sites.
• Access Call Center employees continue an outdated method to schedule appointments.
This involves walking to a group of hand-maintained “white boards” to identify
available treatment slots while placing the caller on hold. Note, the MHP plans to retire
the “white boards” once Netsmart myAvatar is fully operational.
• Reviewers heard repeated references to a concern regarding waiting lists at the Access
Call Center for adults and children. The number of adults and children awaiting
treatment slots through the Access Call Center should be numbers known to MHP
leadership in real-time every day. This can be a constant indicator for how the MHP is
doing with treatment capacity and access to services.
Timeliness of Services
• MHP staff stated with the Epic HealthLink implementation end-user testing was just
beginning and intended to go into production use on February 28, 2018. Configuring
behavioral health functionality in what is a hospital information system (Epic) is a
challenge. This is new territory for the MHP and possibly Epic.
• The concern here is that with testing just starting, the MHP may not yet have a final,
locked-down, training environment and fully-populated final training materials with
five weeks left to go-live.
Denial Reason DescriptionNumber
Denied
Dollars
Denied
Percent
of Total
DeniedBeneficiary not eligible or aid code invalid or restricted service indicator must be "Y" 14,603 $10,013,338 48%Missing, incomplete, invalid ICD-10 diagnosis or condition 3,123 $3,804,765 18%Other coverage must be billed prior to submission of this claim 4,180 $3,418,426 17%Total Denied Claims 26,105 $20,659,099 100%
Table 15: Santa Clara MHP Summary of CY16 Top Three Reasons for Claim Denial
- 56 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
• With what appears to be minimal staffing on the project team and a small amount of
training delivered, this could be either a very challenging go-live or a delayed go-live. It
can be expected, in either case that during the adoption phase clinicians will be slower
in Epic HealthLink than they were in Pro-filer and that may impact workflow in the
clinics.
Quality of Care
• Epic HealthLink is, at its core, an inpatient and outpatient hospital information system.
The MHP should be prepared for the need to make adjustments after the system goes
into production use. This will require resources that are very customer-support
oriented and experts in Epic HealthLink and either conversant with or at least have
access to staff who are conversant with the MHP’s clinical service delivery policies and
procedures.
• The MHP will be dependent for its success on the reliable seamless coordinated
operation of two highly complex information systems working together in a way that
has little precedent in California.
• For some time, likely measured in years, this new information systems infrastructure
will require considerable expert support. The number and level of resources currently
available to support mission critical operations and data exchange processes will need
to continue to be monitored for the impact on the system.
• The MHP can expect a considerable number of requests for either modifying reporting
out of Netsmart myAvatar and Epic HealthLink or creating new reports to help manage
services in this new and very complex information environment. It is unclear that the
MHP has the depth of staffing resources to provide timely turnaround on these requests.
Consumer Outcomes
• With a potential delay in start-up secondary to the monumental undertaking in the
conversion process, consumer appointments may be adversely affected as staff attend
to necessary adjustments in learning the system.
- 57 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
SITE REVIEW PROCESS BARRIERS The following conditions significantly affected CalEQRO’s ability to prepare for and/or conduct a
comprehensive review:
No barriers were encountered during this review.
- 58 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
CONCLUSIONS During the FY17-18 annual review, CalEQRO found strengths in the MHP’s programs, practices, or
information systems that have a significant impact on the overall delivery system and its
supporting structure. In those same areas, CalEQRO also noted opportunities for quality
improvement. The findings presented below relate to the operation of an effective managed care
organization, reflecting the MHP’s processes for ensuring access to and timeliness of services and
improving the quality of care.
Strengths and Opportunities
Access to Care
Strengths:
• Implementation of the CISCO call system strengthens the ability to streamline and
coordinate initial consumer contact.
• The new Integrated Organizational Structure potentially created a clean structure with
room for growth; all age groups share administrative, operational and fiscal integration.
• The MHP is currently working to introduce same day Access service.
• Hiring consultants, as the MHP has done, may contribute to quicker implementation of
efficiencies across the MHP system.
Opportunities:
• Continue to monitor its new call center processes for efficiencies and timeliness to meet
its standards.
• Network adequacy across the system remains a challenge in regard to serving
consumers. Primary challenges are hiring and retention, especially with the contract
providers.
• External factors are the shortage of qualified staff across the Bay Area, coupled with
high housing costs and low unemployment rates that continue to impact services,
recruitment and retention.
• With the continued integration of the mental health Access Call Center and SUTS
Gateway Center to streamline operations, and coordinate services to beneficiaries with
co-occurring disorders, consumers benefit.
• With the implementation of Netsmart myAvatar, retiring the “white boards” that track
appointment slots and provide Access Call Center employees on-line real-time access to
available appointment slots lead to updated access processes.
- 59 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Timeliness of Services
Strengths:
• The clinical PIP was successful in its focus on strategies to improve timeliness and
subsequently improve timely triage and identification of consumer needs.
• The 7-day outpatient follow-up rate post-hospitalization improved, and the 30-day
outpatient follow-up rate post-hospitalization shows greater improvement. This may be
a result of an inpatient liaison to assure consumers receive a follow-up session.
Opportunities:
• Review the timeline and business plan for the MHP’s contract providers’ conversion to
the EHR to ensure efficiencies in the workflow and prevent delays in claims submissions
as well.
• Investigate the anecdotal references to a call-back list at the Access Call Center for
numerous consumers waiting for appointments. Issues regarding unused capacity may
be indicated. This is further supported by an average clinical staff productivity of 49
percent.
• Potentially assigning a peer support worker at the inpatient follow-up, rather than using
the clinical staff, as resources are scarce, could improve follow-up. The WPC initiative
includes plans to establish peer navigators to assist with this follow-up.
Quality of Care
Strengths:
• Renewed vision has emerged with leadership and its efforts to integrate care among
mental health, quality management, physical health, and substance use services.
• The MHP has strengthened its infrastructure with additional management positions to
provide oversight to operationalize its integration efforts.
• The expanded efforts and an integrated vision among leadership appears to have
resulted in a cohesive management team.
• Consistently analyzing the impact of the effectiveness of the MHP initiatives to improve
service timeliness and quality are underway with the Data Dashboards.
Opportunities:
- 60 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
• Reviewing the MHP EHR business plan and implementation timeline can ensure that its
decisions fully support the integrity and realistic capability of a roll-out among contract
providers to achieve functionality and to be successful is duly warranted.
• Potentially unintended results were reported from focus groups that expressed the
reorganization resulted in stakeholders feeling distanced from leadership and some
indicated they felt their input had been devalued.
• Opportunities to include various levels of stakeholders, and to continue transparent
communication are encouraged.
• Differences in communications and decisions among contract monitors were expressed
in focus groups. Contract providers suggest a standardized policy so that contract
monitors have more clear expectations. The new contract process may expedite this
effort.
• Provider to provider transfer is awkward and reportedly is frayed. This suggests a
burden to both consumer and providers in pursuing seamless service delivery.
• Publishing announcements and changes in processes through a portal or web page for
providers rather than relying on emails lists could assist with consistent messaging.
• While effective communication is apparent at the executive level, evaluation of methods
to spread to various levels is encouraged.
• The MHP is overdue in establishing a Clinical Supervisor classification. Although this
topic has been a focus, results were not yet achieved. The class might prove to be an
important channel of communication to employees in addition to other benefits.
Consumer Outcomes
Strengths:
• The clinical PIP indicated improvements to both access and timeliness for which the
MHP intends to replicate in additional clinics.
• Creation and current recruiting for the Consumer and Family Affairs Division Director
position indicates commitment to strengthen the consumer voice.
• The MHP has relatively admirable penetration rates and based on ACB figures appears
to deliver a more than usual level of service to its beneficiaries.
Opportunities:
• With only two wellness self-help centers in the county, Zephyr in San Jose and
Esperanza in Gilroy, evaluating the option of increasing the number is suggested. Both
centers appear underfunded for staff and other resources.
- 61 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
• While the high penetration rates and average claim per beneficiary figures are
significant, analyze whether this is translating into equally admirable outcomes.
Rehospitalization rates, for instance, remain somewhat above the state average.
Recommendations
• Immediately apply additional resources to the implementation of electronic data
exchange between the contract providers and the MHP using Netsmart myAvatar.
o Release thorough Companion Guides that detail the transactions or web services to
be implemented with full detail of data content and formatting specifications as
soon as possible.
o Institute a formal process for determining the readiness of trading partners, that is,
contract providers and their vendors and take a low-risk approach to on-boarding
contract providers. Start with one or two contract providers that are technically
capable, with vendors that have experience with electronic data integration, and
who are eager to participate in this transition.
o Limit the number of contract providers on-boarding in any month to what your
existing staff can realistically support through a challenging transition.
• Strongly advocate for the four additional full-time employees for the Netsmart
myAvatar support team and the six additional staff for the Epic HealthLink support
team. Either of these systems require considerable daily support and oversight. With
them working together, the support need will be even greater. Do not underestimate the
continuing support need involved in managing systems of this scale and complexity.
• Continue to advocate and strengthen the potential of creating a clinical supervisory
position or equivalent. Consider a comparison with other comparable MHPs. Consider
the effects this classification would also provide as a career path for talented clinicians
that could lead to management and possibly reduce turnover in key positions. At a
minimum, determine whether other compensatory benefits could be applicable for the
additional responsibilities.
• Review the recommendation to increase wellness Self-Help Center funding which
contribute to daily activities at the current sites. Study the need for a third center given
the large geographic region of the county.
• Investigate the exceptionally high level of high cost beneficiary (HCB)spending for
Hispanics and children ages 6-17 to determine if it reflects management priorities and
treatment needs of this population versus lack of step-down services. Evaluate the
clinical outcomes for this group. Analyze the transition process or other factors that
may increase costs without improving outcomes.
- 62 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
ATTACHMENTS
Attachment A: CalEQRO On-site Review Agenda
Attachment B: On-site Review Participants
Attachment C: Approved Claims Source Data
Attachment D: CalEQRO Performance Improvement Plan (PIP) Validation Tools
- 63 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Attachment A—On-site Review Agenda
The following sessions were held during the MHP on-site review, either individually or in
combination with other sessions.
Table A1—EQRO Review Sessions - Santa Clara MHP
Opening Session – Changes in the past year; current initiatives; and status of previous year’s recommendations
Use of Data to Support Program Operations
Disparities and Performance Measures/ Timeliness Performance Measures
Quality Improvement and Outcomes
Performance Improvement Projects
Health Plan and Mental Health Plan Collaboration Initiatives
Clinical Line Staff Group Interview
Clinical Supervisors Group Interview
Consumer Employee Group Interview
Consumer Family Member Focus Group(s)
Contract Provider Group Interview – Administration and Operations
Contract Provider Group Interview –Quality Management
Validation of Findings for Pathways to Mental Health Services (Katie A./CCR)
ISCA/Billing/Fiscal
EHR Deployment
Access Call Center Site Visit
Wellness Center Site Visit
Site Visit to Innovative Clinical Programs: Innovative program/clinic that serve special populations or offer special/new outpatient services.
- 64 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Attachment B—Review Participants
CalEQRO Reviewers
Jovonne Price, Quality Reviewer Maureen Bauman, Quality Reviewer Bill Ullom, Chief, Information Systems Reviewer Robert Greenless, Information Systems Reviewer Walter Shwe, Consumer/Family Member Consultant
Additional CalEQRO staff members were involved in the review process, assessments, and
recommendations. They provided significant contributions to the overall review by participating in
both the pre-site and the post-site meetings and in preparing the recommendations within this
report.
Sites of MHP Review
MHP Sites
2325 Enborg Lane San Jose, CA 95128
1075 E. Santa Clara Street San Jose, CA 95116 Kidscope Center 828 Bascom Avenue San Jose, CA 95116 Contract Provider Sites
None
- 65 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Table B1 - Participants Representing the MHP
Last Name First Name Position Agency
Tullys Toni Director
Santa Clara
County
Behavioral Health
Services
Department
(BHSD)
Wiley Deane Deputy Director BHSD
Copley Bruce Director, Alcohol &
Drug Services, Access BHSD
Webber Whitney Quality Management
Director BHSD
Landreneau Todd Adult/Older Adult
Director BHSD
Terao Sherri Family & Children’s
Director BHSD
Ho Tiffany Medical Director BHSD
Luna Laura
Senior Health Care
Program Manager-
Health Plan
BHSD
Olivares Gabby Division Director-
Criminal Justice
BHSD
Hernandez Sandra Division Director-
Integrated Health BHSD
Obilor Margaret Division Director-
Adult-Older Adult BHSD
Juarez Maretta Division Director-
Family & Children BHSD
Garcia Pat Director of
Administration BHSD
Le Mikelle
Senior Manager-
MHUC, Call Center,
CWBC
BHSD
Anderson Alicia
Sr. Health Care
Program Manager-
WPC
BHSD
- 66 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Table B1 - Participants Representing the MHP
Last Name First Name Position Agency
Chu Dinh
Sr. Health Care
Program Manager-
WPC
BHSD
Faria-Costa Zelia Senior Manager-F&C
Division BHSD
Jung Soo Senior Manager-
Housing BHSD
Harnish Mary Compliance & Privacy
Mgr., MHS
BHSD
Berman Cheryl
SUTS Division
Director-Adult-Older
Adult
BHSD
Garrison Victoria Senior Manager-CJS BHSD
Yuter Sheila Senior Manager-
HealthLink BHSD
Ledesma Margaret Senior Manager-Katie
A.
BHSD
Acevedo Domingo QA Manager BHSD
Hogan Sheryl Senior Health Care
Analyst BHSD
Nguyen Hung Senior Manager-
Decision Support BHSD
Ferris Jeannette Senior Manager-
Learning Partnership BHSD
Pangilinan MariaEva Cultural Competency BHSD
Nguyen Thuhien ECCAC BHSD
Rocco Bob Quality Imp. Coord II BHSD
Morale Jeanne Senior Health Care
Program Manager BHSD
Chiu Suzanne LCSW, Clinical Lead
Downtown MHC BHSD
Iwu Loretta
Contract Monitor,
Adult-Older Adult
Programs BHSD
- 67 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Table B1 - Participants Representing the MHP
Last Name First Name Position Agency
Ruiz Enrique MHPSW BHSD
Lai Evonne Contracts Manager BHSD
Robben Vince Finance BHSD
Janini Yasmina Decision Support BHSD
Marquez Veronica Decision Support BHSD
Rodriguez Arisve Clinician Kidscope-BHSD
Nghe Diem Clinician Call Center-BHSD
Picazo Connie Clinician Call Center-BHSD
Ortiz Yesenia Clinician CWBC-BHSD
Rodriguez Luis Clinician CWBC-BHSD
Ponce Mario Clinician CalWorks-BHSD
Urrutia Manuel Clinician CalWorks-BHSD
Trugman Paul Clinician CalWorks-BHSD
Cowan Lance Clinician Downtown-BHSD
Lopez-Molina Mauricio Clinician Las Plumas-BHSD
Bravo Wendy Clinician Las Plumas-BHSD
Koomson James Community Worker Las Plumas-BHSD
Vierra Amanda Clinical Standards-MH BHSD
Powell Larry Clinical Standards-MH BHSD
- 68 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Nelson Tianna
Clinical Standards-
SUTS BHSD
Jones Jennifer Program Manager-
Consumer Affairs BHSD
Kibel Sharla Clinician Kidscope-BHSD
Cornejo Vanessa Clinician Call Center-BHSD
Estrada Jose Clinician Juvenile Hall-
BHSD
Ascue Yukie Clinician Sunnyvale-BHSD
Salada Brian Program Manager-
Sunnyvale Sunnyvale-BHSD
Antons Peter
Program Manager-Las
Plumas/Receiving
Center
Las Plumas-BHSD
Brewer Martha Program Manager-
CalWorks CalWorks-BHSD
Talamantez Rachel Program Manager-
Kidscope Kidscope-BHSD
Bueno Lidia Program Manager-
Kidscope Kidscope-BHSD
DeLeon Judy Program Manager-
CWBC & MHUC BHSD
Lownsbery Steve Clinical Standards-
SUTS BHSD
Hardy John MHPSW BHSD
Blade Michelle MHPSW BHSD
Huang Sophia MHPSW BHSD
Guido Diana MHPSW BHSD
Rahim Hussein MHPSW BHSD
Torres Sharon ECCAC BHSD
De Castro Eramelisse ECCAC BHSD
Wardle Billy Clinician Call Center-BHSD
- 69 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Attachment C—Approved Claims Source Data
Approved Claims Summaries are provided separately to the MHP in a HIPAA-compliant manner.
Values are suppressed to protect confidentiality of the individuals summarized in the data sets
where beneficiary count is less than or equal to eleven (*). Additionally, suppression may be
required to prevent calculation of initially suppressed data, corresponding penetration rate
percentages (n/a); and cells containing zero, missing data or dollar amounts (-).
Table C1 shows the penetration rate and approved claims per beneficiary for just the CY16 ACA
Penetration Rate and Approved Claims per Beneficiary. Starting with CY16 performance measures,
CalEQRO has incorporated the ACA Expansion data in the total Medi-Cal enrollees and beneficiaries
served.
Table C2 shows the distribution of the MHP beneficiaries served by approved claims per beneficiary
range for three cost categories: under $20,000; $20,000 to $30,000, and those above $30,000.
Entity
Average
Monthly ACA
Enrollees
Number of
Beneficiaries
Served
Penetration
Rate
Total Approved
Claims
Approved
Claims per
Beneficiary
Statewide 3,674,069 141,926 3.86% $611,752,899 $4,310
Large 1,778,582 67,721 3.81% $318,050,214 $4,696
Santa Clara 136,481 4,939 3.62% $41,406,080 $8,383
Table C1: Santa Clara MHP CY16 Medi-Cal Expansion (ACA) Penetration Rate
and Approved Claims per Beneficiary
Range of
ACB
MHP Count of
Beneficiaries
Served
MHP
Percentage of
Beneficiaries
Statewide
Percentage of
Beneficiaries
MHP Total
Approved
Claims
MHP
Approved
Claims per
Beneficiary
Statewide
Approved
Claims per
Beneficiary
MHP
Percentage
of Total
Approved
Claims
Statewide
Percentage
of Total
Approved
Claims
< $20K 18,592 81.18% 94.05% $110,403,052 $5,938 $3,612 37.87% 59.13%
>$20K -
$30K1,947 8.50% 2.83% $47,443,332 $24,367 $24,282 16.27% 11.98%
>$30K 2,362 10.31% 3.12% $133,669,749 $56,592 $53,215 45.85% 28.90%
Table C2: Santa Clara MHP CY16 Distribution of Beneficiaries by ACB Range
- 70 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Attachment D—PIP Validation Tool
PERFORMANCE IMPROVEMENT PROJECT (PIP) VALIDATION WORKSHEET FY17-18 CLINICAL PIP
GENERAL INFORMATION
MHP: Santa Clara
PIP Title: Welcome to Treatment Group
Start Date (MM/DD/YY): 12/15/16
Completion Date (MM/DD/YY): 12/31/17
Projected Study Period (#of Months): 12
Completed: Yes ☒ No ☐
Date(s) of On-Site Review (MM/DD/YY):
01/23-25/17
Name of Reviewer: Jovonne Price
Status of PIP (Only Active and ongoing, and completed PIPs are rated):
Rated
☐ Active and ongoing (baseline established and interventions started)
☒ Completed since the prior External Quality Review (EQR)
Not rated. Comments provided in the PIP Validation Tool for technical assistance purposes only.
☐ Concept only, not yet active (interventions not started)
☐ Inactive, developed in a prior year
☐ Submission determined not to be a PIP
☐ No Non-Clinical PIP was submitted
Brief Description of PIP (including goal and what PIP is attempting to accomplish):
- 71 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
The goal of the PIP was to engage and improve timely delivery of services to individuals waiting for specialty behavioral health treatment at Downtown Behavioral Health.
ACTIVITY 1: ASSESS THE STUDY METHODOLOGY
STEP 1: Review the Selected Study Topic(s)
Component/Standard Score Comments
1.1 Was the PIP topic selected using stakeholder input? Did the MHP develop a multi-functional team compiled of stakeholders invested in this issue?
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
The multi-functional team was made up of representatives from outpatient Contracted and County Clinics (manager and staff), Central Wellness (manager), and Narvaez Behavioral Health (manager and staff). Suzanne Chiu (Lead DTBH) and Enrique Ruiz (Peer Support Worker DTBH) were the group facilitators. There were 2 consumers involved in development of the Welcome to Treatment Group, Peer Support Workers.
- 72 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
1.2 Was the topic selected through data collection and analysis of comprehensive aspects of enrollee needs, care, and services?
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
The Performance Improvement Project is focused on timeliness to outpatient services. The Santa Clara County Behavioral Health Department (BHSD) was having difficulty meeting its timeliness metrics. Some factors that might have contributed to the problem are as follows:
• Limited outpatient capacity
• Lack of a step-down process for clients to transition to a lower level of care
• Limited coordination of referral process among the County, contracted agency clinics and Call Center
• Lack of follow up and outreach to support a warm hand-off and engagement with outpatient services
• Lack of client knowledge of behavioral health services and community resources.
Behavioral Health Call Center specialty behavioral health report validates the number of consumers needing services.
An average of 54 days had elapsed from initial request for these consumers who had not yet been assigned to specialty mental health services.
Select the category for each PIP:
Clinical:
☐ Prevention of an acute or chronic condition ☐ High volume services
☒ Care for an acute or chronic condition ☐ High risk conditions
Non-Clinical:
☐ Process of accessing or delivering care
- 73 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
1.3 Did the Plan’s PIP, over time, address a broad spectrum of key aspects of enrollee care and services?
Project must be clearly focused on identifying and correcting deficiencies in care or services, rather than on utilization or cost alone.
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
The problem facing the Santa Clara System of care was increased demand in relation to service provision capabilities, thereby creating delays to services. Delays in providing timely access to services for consumers seeking specialty mental health reduces quality of care and poor consumer outcomes (Evans N, 2014).
• To improve outcomes of care and support engagement in the treatment services, information that consumers can utilize when the clinic is closed is provided to the attendees in a welcome packet.
• To familiarize the consumer with the treatment environment, encourage the consumers’ willingness to engage and stay in treatment, help the consumer feel safe and secure, and reduce the no-show rates, a clinician and a peer support worker reviews the welcome packet and provides other information regarding the available wellness and recovery services.
• The clinician checks in with consumers for brief intervention to see if there will be need for medication refill prior to intake and estimated medication evaluation date by the psychiatrist. This process makes the consumer feel heard and supported with their individual needs are taken into consideration as evident by some of the surveys completed by the consumers at the end of the group. This improves outcome by ensuring medication stability for consumers and decreases the likelihood for consumer decompensation due to being out of medication prior to their appointment.
• The clinician triages the needs of the clients with a brief assessment to determine if the consumers need to be connected to urgent care for medication support or an earlier appointment for stabilization. This process facilitates treatment timeliness, which helps to prevent further decompensation and possible hospitalization. Whereby, further stabilization is necessary, a 5150 protocol is initiated,
- 74 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
and the consumer is transported to emergency psychiatry services.
• Consumers are guided on the process of obtaining medication from BH Urgent Care if necessary prior to psychiatrist’s evaluation. This process encourages staff/consumer interactions for better support and good quality care.
• Face to face interactions between consumers and their treatment team are coordinated to encourage engagement and treatment success. The clerical staff will schedule an intake appointment during the group and the consumer meets with assigned clinician the same day if possible.
• In addition, individuals are informed about and given a tour of Zephyr, a self-help center which is located on the grounds of the Downtown Behavioral Health Clinic. This center provides peer run groups, computer classes, consumer rights workshops, and other wellness and recovery activities, which is available to consumers as a treatment option. The center provides additional peer support to help improve interactions and socialization with other consumers, which increases attendance and utilization of the Self-Help Center by the attendees of the Welcome to Treatment Group. This improves outcomes by increasing peer support interactions and resources.
1.4 Did the Plan’s PIPs, over time, include all enrolled populations (i.e., did not exclude certain enrollees such as those with special health care needs)?
Demographics:
☐ Age Range ☐ Race/Ethnicity ☐ Gender ☐ Language ☐ Other
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
The population for this project are those consumers who are referred through the Call Center and includes the following categories:
• Those who call the BHS Call Center requesting services
• Those referred by their Primary Care Physician for mental health services
• Those who are attempting to reenter the system (either because they were previously closed or some other reasons).
Totals 4 Met 0 Partially Met 0 Not Met 0 UTD
- 75 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
STEP 2: Review the Study Question(s)
2.1 Was the study question(s) stated clearly in writing?
Does the question have a measurable impact for the defined study population?
Include study question as stated in narrative:
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
Will the development and implementation of Welcome to Treatment at Downtown Behavioral Health increase timely access to care from 18.1 days to 14 days by December 2017, engagement, and overall consumer perception within specialty behavioral health outpatient services?
Totals 1 Met 0 Partially Met 0 Not Met 0 UTD
STEP 3: Review the Identified Study Population
3.1 Did the Plan clearly define all Medi-Cal enrollees to whom the study question and indicators are relevant?
Demographics:
☐ Age Range ☐ Race/Ethnicity ☐ Gender ☐ Language ☐ Other
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
The population at DTMH is predominantly Hispanic and White. The gender distribution is somewhat evenly distributed, with slightly more males. The population is mainly adults ages 26-59.
See PIP submission for graphs indicating the demographics.
3.2 If the study included the entire population, did its data collection approach capture all enrollees to whom the study question applied?
Methods of identifying participants:
☒ Utilization data ☒ Referral ☐ Self-identification
☐ Other:
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
Totals 2 Met 0 Partially Met 0 Not Met 0 UTD
- 76 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
STEP 4: Review Selected Study Indicators
4.1 Did the study use objective, clearly defined, measurable indicators?
List indicators: 1. Average calendar days to initial service at DTBH 2. Percent of consumers at DTBH who have their initial service within
14 days 3. Percent of consumers opened to DTBH after referral from the Call
Center 4. Percent of consumers who were not opened to DTBH after
referral from the Call Center because they ended up receiving crisis services
5. Average number of face to face services within 60 days of admit for consumers who are opened to Downtown Mental Health
6. Percent of new consumer admits with 4 visits within 30 calendar days at DTBH
7. Percent of consumers who strongly agreed or disagreed Welcome to Treatment Group was useful at DTBH
8. Percent of consumers who strongly agreed or disagreed they were able to receive adequate information about the services they would be receiving at DTBH
9. Percent of consumers who strongly agreed or disagreed they felt more knowledgeable in how to better navigate their treatment at DTBH
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
Goals for each indicator:
1. 14 days
2. 2 percentage points improvement over baseline
3. 20percent increase from baseline with appropriate openings
4. n/a
5. 75 percent (MHP standards: A minimum of 2 hours per visit and 4 contacts per 30 days)
6. 75 percent
7. Client perception data
8. 75percent
9. 75percent
- 77 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
4.2 Did the indicators measure changes in: health status, functional status, or enrollee satisfaction, or processes of care with strong associations with improved outcomes? All outcomes should be consumer focused.
☐ Health Status ☒ Functional Status
☒ Member Satisfaction ☐ Provider Satisfaction
Are long-term outcomes clearly stated? ☒ Yes ☐ No
Are long-term outcomes implied? ☒ Yes ☐ No
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
• The client perception data was collected using a 5-question Welcoming Group Satisfaction survey with a 5-point Likert scale (strongly disagree, disagree, neutral, agree, strongly agree) developed by DTBH.
• Timely access to service within 14 days to understand how to improve the effectiveness of consumer’s early engagement and support, which is a proven method of keeping consumer’s in treatment.
• To show an increase in the number of consumers attending DTBH.
• To demonstrate we can keep people out of crisis settings through the interventions.
• To demonstrate consumer engagement through face to face contacts within 60 days after being opened at DTBH
• Percentage of new consumer admits with 4 visits within 30 days is a standard set forth for ongoing rapport and engagement during the first 30 days
• To obtain consumers’ perception on usefulness of the Group to determine if providing a Welcome to treatment group is beneficial to the clients
• To obtain consumers’ perception on adequacy of information received about services from the Group. To determine areas of improvement in the information provided to clients during the group.
• To obtain consumers’ perception on their ability to navigate treatment at DTBH.
Totals 2 Met 0 Partially Met 0 Not Met 0 UTD
- 78 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
STEP 5: Review Sampling Methods
5.1 Did the sampling technique consider and specify the:
a) True (or estimated) frequency of occurrence of the event?
b) Confidence interval to be used?
c) Margin of error that will be acceptable?
☐ Met
☐ Partially Met
☐ Not Met
☒ Not Applicable
☐ Unable to Determine
5.2 Were valid sampling techniques that protected against bias employed?
Specify the type of sampling or census used:
<Text>
☐ Met
☐ Partially Met
☐ Not Met
☒ Not Applicable
☐ Unable to Determine
5.3 Did the sample contain a sufficient number of enrollees?
______N of enrollees in sampling frame
______N of sample
______N of participants (i.e. – return rate)
☐ Met
☐ Partially Met
☐ Not Met
☒ Not Applicable
☐ Unable to Determine
Totals 0 Met 0 Partially Met 0 Not Met 3 NA 0 UTD
STEP 6: Review Data Collection Procedures
6.1 Did the study design clearly specify the data to be collected?
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
See Indicators in Item 4.1 above.
6.2 Did the study design clearly specify the sources of data?
Sources of data:
☒ Member ☐ Claims ☐ Provider
☒ Other: Unicare
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
All of the access, engagement, and outcomes data for this PIP is collected in Unicare, the MHP’s electronic health record. The access data is captured by BHS Call Center staff for all consumers.
The second set of data, service documentation related to
engagement and outcomes, is also collected in Unicare. This
- 79 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
information is double-checked for accuracy because wrong
information may result in payment denial by the insurer or rejection
by the state reporting (CSI) for errors.
By combining these two sets of data in Unicare, the Decision Support
team in the Quality Management Departments has the ability to
calculate the performance indicators for access, engagement, and
outcomes for this PIP.
DTBH clinic staff asked all consumers at the conclusion of the
Welcoming Group to complete a survey on their experience.
- 80 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
6.3 Did the study design specify a systematic method of collecting valid and reliable data that represents the entire population to which the study’s indicators apply?
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
Once the form is filled out and saved, it is available for other Call Center staff and the receiving treatment providers to review. Call Center staff collects the data on an ongoing basis, the data is analyzed monthly by the Decision Support team and reported quarterly.
When the consumers or family members call the toll-free number, BHS Call Center staff begin the screening process to capture the individual’s demographic and contact information. In addition, the staff make an initial screening to determine the consumer level of care. Based on all the collected information and availability of capacity, the BHS Call Center staff make appropriate treatment referrals. The access data is captured in a User Defined Data (UDD) Mental Health Referral Form. Once the form is filled out and saved, it is available for other Call Center staff and the receiving treatment providers to review.
Once the consumer shows up to their appointment at the treatment location, clerical staff verify and document the consumer demographics, contact and insurance information in Unicare. After this process, the consumer then meets with their assigned clinician. The clinician documents all services provided to the consumer along with progress notes in Unicare.
The Unicare data provided is routinely reviewed monthly and quarter by the management team, quality performance committee and A/OA system of care committee. The client perception data was collected at the end session.
- 81 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
6.4 Did the instruments used for data collection provide for consistent, accurate data collection over the time periods studied?
Instruments used:
☒ Survey ☒ Medical record abstraction tool
☐ Outcomes tool ☐ Level of Care tools
☐ Other:
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
Unicare and consumer survey were used consistently.
6.5 Did the study design prospectively specify a data analysis plan?
Did the plan include contingencies for untoward results?
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
Daily log-in with real-time notes of consumer calls; monthly and quarterly reports to management for review.
Untoward results would be addressed on-going by the appropriate unit, i.e., staff, QM, or Data as applicable.
6.6 Were qualified staff and personnel used to collect the data?
Project leader:
Name: Loretta Iwu and Suzanne Chiu
Title: MHPS PSW
Role: Contract Manager/Lead Clinician
Other team members:
Names: See comments section.
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
Call Center staff including clerical and licensed clinicians collected data from consumers. Unicare data was collected by staff in the Decision Support team and the Quality Management staff. All staff are qualified per job specifications.
Totals 6 Met 0 Partially Met 0 Not Met 0 UTD
- 82 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
STEP 7: Assess Improvement Strategies
7.1 Were reasonable interventions undertaken to address causes/barriers identified through data analysis and QI processes undertaken?
Describe Interventions:
• The clinician triages the needs of the clients with a brief assessment to determine if the consumers need to be connected to urgent care for medication support or an earlier appointment for stabilization
• The clinician checks in with consumers for brief intervention to see if there will be need for medication refill.
• Provide a welcoming pamphlet and packet to consumers who are seeking services for additional education and behavioral health resources, which enables consumers to make informed decisions about treatment options to improve engagement
• Create therapeutic, supportive environment that facilitates client-staff therapeutic alliance, as consumers, families, and other support systems connects with the case manager and clinician the same day of group for better support and engagement.
• Also streamline workload and workflow for staff from documentation to caseload.
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
Totals 1 Met 0 Partially Met 0 Not Met 0 NA 0 UTD
- 83 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
STEP 8: Review Data Analysis and Interpretation of Study Results
8.1 Was an analysis of the findings performed according to the data analysis plan?
This element is “Not Met” if there is no indication of a data analysis plan (see Step 6.5)
☒ Met
☐ Partially Met
☐ Not Met
☐ Not Applicable
☐ Unable to Determine
See Item 6.3 and Item 6.5 above for details.
- 84 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
8.2 Were the PIP results and findings presented accurately and clearly?
Are tables and figures labeled? ☒ Yes ☐ No
Are they labeled clearly and accurately? ☒ Yes ☐ No
☒ Met
☐ Partially Met
☐ Not Met
☐ Not Applicable
☐ Unable to Determine
Results are clear and labeled.
• The Unicare data provided is routinely reviewed monthly and quarter by the management team, quality performance committee and A/OA system of care committee. The client perception data was collected using a 5-question Welcoming Group Satisfaction survey with a 5-point Likert scale (strongly disagree, disagree, neutral, agree, strongly agree) developed by DTBH.
• The average calendar days to initial service at DTBH improved by 4percent with 19percent improvement of level 2 consumers participating in Welcoming Group being opened within 14 days of initial service. This result may trigger the need to include the call center, all county clinics, and contract providers to further improve timely access to care.
• The project obtained real-time referrals during the 12-month period which lead to improved results as evidenced by a reduction of an average wait time from 54 days to 32 days. There may be a need to continue real-time referrals which will result in zero wait time for consumers seeking specialty mental health.
• Engagement between staff/consumers as shown increased face-to-face interactions, an improvement of 664percent for DTBH and 820percent for the Welcoming Group. This may trigger additional clinicians to participate in the facilitation of the group to improve engagement and to provide clinical support to consumers.
- 85 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
• Based on consumer perception data, 81.8percent of consumers found the group useful. There may be a need to implement a short therapy group as part of the welcome to treatment to help enhance consumers’ coping skills to add value to the group.
8.3 Did the analysis identify: initial and repeat measurements, statistical significance, factors that influence comparability of initial and repeat measurements, and factors that threaten internal and external validity?
Indicate the time periods of measurements: ___________________
Indicate the statistical analysis used: _________________________
Indicate the statistical significance level or confidence level if available/known: _______percent ______Unable to determine
☐ Met
☒ Partially Met
☐ Not Met
☐ Not Applicable
☐ Unable to Determine
The MHP did not analyze with respect to the demographic parameters of our study population before and after the intervention. The measures for the study populations were taken at different times of the year.
Data was collected regularly throughout the year.
See Item 8.2 above.
8.4 Did the analysis of the study data include an interpretation of the extent to which this PIP was successful and recommend any follow-up activities?
Limitations described:
Conclusions regarding the success of the interpretation:
Recommendations for follow-up:
☒ Met
☐ Partially Met
☐ Not Met
☐ Not Applicable
☐ Unable to Determine
• Based on the data analysis results, the PIP is successful as evidenced by improved timeliness to access for consumers which is reflected in the 4percent improvement for average calendar days to initial services at DTBH.
• There has been an improvement for percent of consumers at DTBH who have their initial services within 14 days with 7percent increase after the project.
• There is no statistical evidence on the data analysis table on engagement, however, engagement between staff and consumers has improved since the implementation of the group as evidence by decreased no show rates and increase in utilization of services at the clinic.
• There was 81.8percent of consumers who reported the Welcoming Group to be useful and 89.1percent who reported they were able to receive adequate information.
See Item 8.2 above.
Totals 3 Met 1 Partially Met 0 Not Met 0 NA 0 UTD
- 86 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
STEP 9: Assess Whether Improvement is “Real” Improvement
9.1 Was the same methodology as the baseline measurement used when measurement was repeated?
Ask: At what interval(s) was the data measurement repeated?
Were the same sources of data used?
Did they use the same method of data collection?
Were the same participants examined?
Did they utilize the same measurement tools?
☐ Met
☐ Partially Met
☐ Not Met
☒ Not Applicable
☐ Unable to Determine
Some baseline data was not available as some interventions were newly created. The MHP now has baseline data for these indicators.
Since it is early into the phase of determining real improvement this section is not applicable at this time.
The MHP intends to replicate the group in other centers as it has improved consumers information regarding navigating the system and allowed staff to intervene in a triage manner.
- 87 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
9.2 Was there any documented, quantitative improvement in processes or outcomes of care?
Was there: ☒ Improvement ☐ Deterioration
Statistical significance: ☒ Yes ☐ No
Clinical significance: ☒ Yes ☐ No
☐ Met
☐ Partially Met
☐ Not Met
☒ Not Applicable
☐ Unable to Determine
See Item 8.4 above.
9.3 Does the reported improvement in performance have internal validity; i.e., does the improvement in performance appear to be the result of the planned quality improvement intervention?
Degree to which the intervention was the reason for change:
☐ No relevance ☐ Small ☐ Fair ☒ High
☐ Met
☐ Partially Met
☐ Not Met
☒ Not Applicable
☐ Unable to Determine
These results from this PIP may be generalizable to other locations and the MHP is expanding the program to another County clinic and potentially two contract providers.
9.4 Is there any statistical evidence that any observed performance improvement is true improvement?
☐ Weak ☐ Moderate ☐ Strong
☐ Met
☐ Partially Met
☐ Not Met
☒ Not Applicable
☐ Unable to Determine
No statistical testing was performed; data suggests improved outcomes related to the goals.
9.5 Was sustained improvement demonstrated through repeated measurements over comparable time periods?
☐ Met
☒ Partially Met
☐ Not Met
☒ Not Applicable
☐ Unable to Determine
This was completed with a first year of data which can be used for repeat measures. The results are promising and show success. This PIP is intended to continue and the MHP will calculate future results for comparisons.
Totals 0 Met 0 Partially Met 0 Not Met 5 NA 0 UTD
- 88 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
ACTIVITY 2: VERIFYING STUDY FINDINGS (OPTIONAL)
Component/Standard Score Comments
Were the initial study findings verified (recalculated by CalEQRO) upon repeat measurement?
☐ Yes
☒ No
ACTIVITY 3: OVERALL VALIDITY AND RELIABILITY OF STUDY RESULTS: SUMMARY OF AGGREGATE VALIDATION FINDINGS
Conclusions:
The MHP was successful with this PIP in improving timeliness to services within the 14-day standard. The Welcome to Treatment group posed success based on consumer feedback and the intent is to expand the PIP to other clinics and contract provider services.
Recommendations:
Based on the initial success, the MHP is encouraged to continue this PIP and to expand it to improve timeliness to services.
Check one: ☒ High confidence in reported Plan PIP results ☐ Low confidence in reported Plan PIP results
☐ Confidence in reported Plan PIP results ☐ Reported Plan PIP results not credible
☐ Confidence in PIP results cannot be determined at this time
- 89 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
PERFORMANCE IMPROVEMENT PROJECT (PIP) VALIDATION WORKSHEET FY17-18 NON-CLINICAL PIP
GENERAL INFORMATION
MHP: Santa Clara
PIP Title: FY2018 Contract Renewal Process Pilot Project
Start Date (MM/DD/YY): 02/01/2017
Completion Date (MM/DD/YY): 09/30/2017
Projected Study Period (#of Months): 12
Completed: Yes ☒ No ☐
Date(s) of On-Site Review (MM/DD/YY):
01/23-25/2018
Name of Reviewer: Jovonne Price
Status of PIP (Only Active and ongoing, and completed PIPs are rated):
Rated
☐ Active and ongoing (baseline established and interventions started)
☐ Completed since the prior External Quality Review (EQR)
Not rated. Comments provided in the PIP Validation Tool for technical assistance purposes only.
☐ Concept only, not yet active (interventions not started)
☐ Inactive, developed in a prior year
☒ Submission determined not to be a PIP
☐ No Clinical PIP was submitted
Brief Description of PIP (including goal and what PIP is attempting to accomplish):
- 90 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
The project’s aim, specific to Short-Doyle (SD) /Mental Health Services Act (MHSA) mental health service contracts only, was to pilot a new approach to increase timely delivery of executed contracts, ensuring they are in place at the start of the fiscal year for consumers.
It was determined that the non-clinical PIP did not meet the criteria for a PIP secondary to the lack of consumer benefit. Although commendable efforts were done in the MHP’s process and business procedures, there were no stated outcomes for the consumer. However, this PIP did reconstruct the method for timely completion of the MHP contract agreements. The MHP managed to complete over 95percent of its contracts with organizational providers and introduced new efficiencies to do this. Discussion occurred on-site to emphasize using EQRO for technical assistance in developing its PIPs.
- 91 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
ACTIVITY 1: ASSESS THE STUDY METHODOLOGY
STEP 1: Review the Selected Study Topic(s)
Component/Standard Score Comments
1.1 Was the PIP topic selected using stakeholder input? Did the MHP develop a multi-functional team compiled of stakeholders invested in this issue?
☐ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
Although the multi-functional team does not include direct representation from consumer/family members the pilot project was conducted with the County Mental Health Plan (MHP) beneficiaries/enrollees in mind. The County MHP in the beneficiary/recipient’s assigned county is responsible for providing MHP-covered services for eligible recipients in that county. Timely execution of SD/MHSA Community Based Organization service provider contracts which ensures services are in place for Santa Clara County MHP clients throughout the year. Given the PIP is focused on improving operational management of contracts administration, the pilot project does include participants, specifically BHSD staff from specific areas of the Department, who directly contribute to the development and execution of SD/MHSA contracts: Administration, Programs-CYF and A/OA, Finance and Contracts.
- 92 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
1.2 Was the topic selected through data collection and analysis of comprehensive aspects of enrollee needs, care, and services?
☐ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
This does not reflect a data collection and analysis of comprehensive
aspects of enrollee needs, care and services. Services were not
stopped because contracts were not renewed timely. To qualify, the
PIP has to be connected to consumer outcomes as stated this is a
process change.
Prior to the implementation of the FY2018 Contract Renewal Process Pilot Project, BHSD has not been able to execute SD/MHSA renewal contracts by the start of the fiscal year in recent years and has had to execute contracts on a retroactive basis-after the start date of the contract effective date:
In FY2016, BHSD conducted two contract amendment processes: a) Term 1-renew existing SD/MHSA contracts and b) Term 2-amend contracts to reflect contractual changes resulting from Request for Proposal (RFP) activities/awardee results.
For Term 1, 48 of the 51 SD/MHSA contracts renewed on time, but for Term 2, the second contract process rolled out in FY2016, only 10 of the 39 SD/MHSA contracts, which represents about 26percent of the SD/MHSA contracts, renewed on time.
In FY2017, only one of the 36 SD/MHSA CBO service contracts renewed on time.
Select the category for each PIP:
Clinical:
☐ Prevention of an acute or chronic condition ☐ High volume services
☐ Care for an acute or chronic condition ☐ High risk conditions
Non-Clinical:
☐ Process of accessing or delivering care
- 93 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
1.3 Did the Plan’s PIP, over time, address a broad spectrum of key aspects of enrollee care and services?
Project must be clearly focused on identifying and correcting deficiencies in care or services, rather than on utilization or cost alone.
☐ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
The PIP does not discuss in depth or measure consumer outcomes as required to meet criteria.
The goal of the project was to ensure timely execution of service contracts utilized by the County’s Medi-Cal beneficiaries/enrollees. A service contract is an agreement between the Behavioral Health Services Department (County) and a service provider (contractor) in which a contractor agrees to provide mental health services to eligible Medi-Cal beneficiaries of Santa Clara County based on the terms reflected in the contract.
BHSD serves an estimated 30,000 consumers annually with about 30percent of the services being provided by the County and the majority of the services at 70percent are provided by contracted, community-based agencies, who are working in the communities where clients/consumers live.
1.4 Did the Plan’s PIPs, over time, include all enrolled populations (i.e., did not exclude certain enrollees such as those with special health care needs)?
Demographics:
☐ Age Range ☐ Race/Ethnicity ☐ Gender ☐ Language ☐ Other
☐ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
A service contract is an agreement between the Behavioral Health Services Department (County) and a service provider (contractor) in which a contractor agrees to provide mental health services to eligible Medi-Cal beneficiaries of Santa Clara County based on the terms reflected in the contract. The FY2018 Contract Renewal Process Pilot Project’s goal was to improve the timely execution of SD/MHSA service contracts provided by community- based organizations, contractors, who serve the County’s Medi-Cal beneficiaries across the age continuum from 0-5 to older adults.
Totals 0 Met 0 Partially Met 0 Not Met 0 UTD
- 94 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
STEP 2: Review the Study Question(s)
2.1 Was the study question(s) stated clearly in writing? Does the question have a measurable impact for the defined study population?
Include study question as stated in narrative: See comments. There is no indication of consumer outcomes included in the study question. It appears to be strictly a business process change which could inadvertently benefit consumers. To date, there is no evidence that is the case.
☐ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
See item 2.1. The MHP presented this study question. This is focused on a business process change and is not consumer focused. “Will the implementation of the FY2018 Contract Renewal Process Pilot Project, which includes the following enhancements as listed below, improve timely execution of the County’s SD/MHSA service contracts?
• The redesign of the Contracting Services Request (CSR) Form to include only vital/relevant information will help facilitate the approval process of the CSR in a timely manner.
• The implementation of the redesigned CSR Form for SD/MHSA service contracts in digitized format for use in DocuSign, a digital transaction management system that provides electronic signature technology for facilitating electronic exchanges of signed documents, will enable efficient tracking and review/approval of CSRs.
• The unbundling of service provider contracts into separate contracts and package by program division: F&C, A/OA, CJS, IBH, and SHS, will provide each program division better control over their specific program contracts and expedite execution of contracts.
• Conducting various meetings related to the pilot project with all parties involved: pilot project team, executive team, and CBO contract providers will ensure everyone knows the goal of the pilot project and roles and responsibilities of each area.
• The creation a status tracking sheet to show current renewal status of all FY18 SD/MHSA service contracts and making it available online/SharePoint site to all staff involved in the pilot project so that everyone is aware and current on the status of contracts throughout the renewal process. “
- 95 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Totals 0 Met 0 Partially Met 0 Not Met 0 UTD
STEP 3: Review the Identified Study Population
3.1 Did the Plan clearly define all Medi-Cal enrollees to whom the study question and indicators are relevant?
Demographics:
☐ Age Range ☐ Race/Ethnicity ☐ Gender ☐ Language ☐ Other
☐ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
The MHP has not included a target group of consumers. This appears to be a business procedural shift which will lead to timely contract completions.
A service contract is an agreement between the Behavioral Health Services Department (County) and a service provider (contractor) in which a contractor agrees to provide mental health services to eligible Medi-Cal beneficiaries of Santa Clara County based on the terms reflected in the contract.
3.2 If the study included the entire population, did its data collection approach capture all enrollees to whom the study question applied?
Methods of identifying participants:
☐ Utilization data ☐ Referral ☐ Self-identification
☒ Other: Contract renewal process/completion dates
☐ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
Totals 0 Met 0 Partially Met 0 Not Met 0 UTD
STEP 4: Review Selected Study Indicators
4.1 Did the study use objective, clearly defined, measurable indicators?
List indicators:
• Turnaround Time (TAT) of SD/MHSA CSR Form Approval
• percent of SD/MHSA contracts executed and effective at the start of FY2018
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
- 96 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
4.2 Did the indicators measure changes in: health status, functional status, or enrollee satisfaction, or processes of care with strong associations with improved outcomes? All outcomes should be consumer focused.
☐ Health Status ☐ Functional Status
☐ Member Satisfaction ☐ Provider Satisfaction
Are long-term outcomes clearly stated? ☐ Yes ☐ No
Are long-term outcomes implied? ☐ Yes ☐ No
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
The MHP does not provide evidence that consumer care was impacted as when contracts were amended due to timeliness, services continued. This appears to be focused on the process changes.
Process measures, while acceptable, must offer strong clinical evidence that the process being measured is meaningfully associated with outcomes. This determination should be based on published guidelines, including citations from randomized clinical trials, case control studies, or cohort studies. At a minimum, the PIP should be able to demonstrate a consensus among relevant practitioners with expertise in the defined area who attest to the importance of a given process.
While enrollee satisfaction is an important outcome of care in clinical areas, improvement in satisfaction should not be the only measured outcome of a clinical project. Some improvement in health or functional status should be addressed
The goal of the project was to ensure timely execution of service contracts utilized by the County’s Medi-Cal beneficiaries/enrollees.
This enables consistent and efficient continued care to consumers served who are served by 70percent contract providers. However, there is no evidence that consumers were not served.
Totals 0 Met 0 Partially Met 0 Not Met 0 UTD
STEP 5: Review Sampling Methods
5.1 Did the sampling technique consider and specify the:
a) True (or estimated) frequency of occurrence of the event?
b) Confidence interval to be used?
c) Margin of error that will be acceptable?
☐ Met
☐ Partially Met
☐ Not Met
☒ Not Applicable
☐ Unable to Determine
No sampling used.
- 97 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
5.2 Were valid sampling techniques that protected against bias employed?
Specify the type of sampling or census used:
<Text>
☐ Met
☐ Partially Met
☐ Not Met
☒ Not Applicable
☐ Unable to Determine
5.3 Did the sample contain a sufficient number of enrollees?
______N of enrollees in sampling frame
______N of sample
______N of participants (i.e. – return rate)
☐ Met
☐ Partially Met
☐ Not Met
☒ Not Applicable
☐ Unable to Determine
Totals 0 Met 0 Partially Met 0 Not Met 0 Not Applicable 0 UTD
- 98 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
STEP 6: Review Data Collection Procedures
6.1 Did the study design clearly specify the data to be collected?
The MHP would need to define: Who will collect data, how often it will be collected, who will analyze the data, how often will it be analyzed, what do they expect the results to be, what will they do if the results are different than what they expect.
☐ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
Data is related to the business processes and does not reflect consumer outcomes.
• Turnaround Time (TAT) of SD/MHSA CSR Form Approval
• percent of SD/MHSA contracts executed and effective at the start of FY2018
6.2 Did the study design clearly specify the sources of data?
Sources of data:
☐ Member ☐ Claims ☐ Provider
☒ Other: contract renewal process-see comments
☐ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
The FY2018 Contract Renewal Process Pilot Project Data Set includes:
• All completed and approved SD/MHSA CSR Forms. As part of the CSR set-up in DocuSign, a copy of a submitted CSR is automatically emailed to the Contracts Manager to give notice that a request has come through and pending approval. A final approved copy of the CSR is also emailed to the Contracts Manager at the end of the process. The CSR Form includes submission date and approval date to enable the team to calculate the CSR approval turnaround time (TAT) from start to finish with the approval of the form.
• The number of contracts approved/executed through the Master Contracts List (MCL) process or by the Board of Supervisors through a Legislative File which are public documents.
• A contract renewal status tracking sheet of all FY18 SD/MHSA service contracts was developed based on input on the needs of all the areas: Programs, Finance, and Contracts and ensure up-to-date contract renewal status information. The file was posted online/SharePoint site to all pilot project participants, and the Contracts team was requested to maintain and update the data file.
- 99 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
6.3 Did the study design specify a systematic method of collecting valid and reliable data that represents the entire population to which the study’s indicators apply?
☐ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
In the past, the original CSR Form did not have a date field for the CSR submission date, and therefore BHSD was unable to track the submission date to determine the average approval turnaround time (TAT) of CSRs and assess performance from year to year.
With the new SD/MHSA CSR Form, the CSR submission date field is included along with the approval date of the CSR which enabled BHSD to determine and calculate the approval TAT per form, calculate the average TAT during the pilot project, as well as TAT by program division level.
6.4 Did the instruments used for data collection provide for consistent, accurate data collection over the time periods studied?
Instruments used:
☐ Survey ☐ Medical record abstraction tool
☐ Outcomes tool ☐ Level of Care tools
☒ Other: ongoing collection during the contract renewal period
☐ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
- 100 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
6.5 Did the study design prospectively specify a data analysis plan?
Did the plan include contingencies for untoward results?
☐ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
The FY2018 Contract Renewal Process Pilot Project Data Set includes:
• All completed and approved SD/MHSA CSR Forms. As part of the CSR set-up in DocuSign, a copy of a submitted CSR is automatically emailed to the Contracts Manager to give notice that a request has come through and pending approval. A final approved copy of the CSR is also emailed to the Contracts Manager at the end of the process. The CSR Form includes submission date and approval date to enable the team to calculate the CSR approval turnaround time (TAT) from start to finish with the approval of the form.
• The number of contracts approved/executed through the Master Contracts List (MCL) process or by the Board of Supervisors through a Legislative File which are public documents.
• A contract renewal status tracking sheet of all FY18 SD/MHSA service contracts was developed based on input on the needs of all the areas: Programs, Finance, and Contracts and ensure up-to-date contract renewal status information. The file was posted online/SharePoint site to all pilot project participants, and the Contracts team was requested to maintain and update the data file.
Limited information was included in the event of untoward results. A defined process to ensure timely completion was not presented. Oversight regarding meeting the timelines was not clear, rather it was hoped staff would adhere to the renewal time.
Contingency plans were limited to “encouraged team members to be proactive to ensure that contracts are moving through the process in an effective, timely manner”.
- 101 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
6.6 Were qualified staff and personnel used to collect the data?
Project leader
Name: Jeanne Moral
Title: Sr. Heath Care Program Manager, System Initiatives
Role: Lead facilitator
Other team members:
Names: various members of the Contracts Team, IT staff
☐ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
Totals 0 Met 0 Partially Met 0 Not Met 0 UTD
- 102 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
STEP 7: Assess Improvement Strategies
- 103 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
7.1 Were reasonable interventions undertaken to address causes/barriers identified through data analysis and QI processes undertaken?
Describe Interventions:
• Unbundle service provider contracts into separate contracts and package by program division: F&C, A/OA, CJS, IBH, and SHS.
• Redesign the Contracting Service Request (CSR) Form, digitize it and make it available in DocuSign.
• Create a tracking sheet to show current renewal status of all FY18 SD/MHSA service contracts and make it available online/SharePoint site to all pilot project participants. Include the following status categories for each contract:
1. CSR Received
2. Exhibit A’s Received
3. Exhibit B’s Received
4. Drafting Contract
5. Contract's Unit Review of Draft Contract
6. Finance's Unit Review of Draft Contract
7. Program's Unit Review of Draft Contract
8. With Contractor/Agency for Signature
9. With County Counsel/OBA for Signature
10. With BSHD Department Head for Signature
11. With General Fund Financial Services Director and HHS CFO for signature
12. With HHS CEO for signature (if applicable)
13. Procurement (if applicable)
14. Finalized for BOS Meeting
15. With HHS Accounts Payable Team
☐ Met
☐ Partially Met
☐ Not Met
☐ Unable to Determine
Interventions were focused on the business processes; no interventions were focused on the consumer benefit.
- 104 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
• Request the County Office of Budget Analysis (OBA) to consider BHSD’s FY18 SD/MHSA contracts for the County’s Master Contract List (MCL).
Totals 0 Met 0 Partially Met 0 Not Met 0 NA 0 UTD
- 105 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
STEP 8: Review Data Analysis and Interpretation of Study Results
- 106 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
8.1 Was an analysis of the findings performed according to the data analysis plan?
This element is “Not Met” if there is no indication of a data analysis plan (see Step 6.5)
☐ Met
☐ Partially Met
☐ Not Met
☐ Not Applicable
☐ Unable to Determine
There are no consumer outcomes evaluated.
Graphs and charts were included in the submission document as well as a narrative of the data analysis.
Average Approval Turnaround Time of FY2018 SD/MHSA CSRs:
In the past, the original CSR Form did not have a date field for the CSR submission date, and therefore BHSD was unable to track the submission date to determine the average approval turnaround time (TAT) of CSRs and assess performance from year to year. With the new SD/MHSA CSR Form, the CSR submission date field is included along with the approval date of the CSR which enabled BHSD to determine and calculate the approval TAT per form, calculate the average TAT during the pilot project, as well as TAT by program division level.
For the FY2018 SD/MHSA contract renewal process, the average approval TAT for the 61 SD/MHSA contracts was three days, see attached file for details. In the past, based on staff’s experience CSR approval took weeks and sometimes months but as illustrated the pilot project surpassed the two-week CSR approval TAT goal. In addition, the majority of the CSRs were approved within two days as pictured here. At the top of the range, only one CSR reflected a 10-day approval TAT.
Number of Completed Contracts by Execution Type:
The County of Santa Clara has two contract execution types: MCL = Master Contract List or Board = Board of Supervisors. Agreements approved for the MCL get Delegation of Authority (DOA) and does not require a Legislative File (LF) for full execution of the contract while contracts that exceed $5 million require execution of the contract through the regular Board of Supervisors (BOS) process which requires an LF which is a lengthy process. With the unbundling of the service contracts as described in prior sections into five distinct program divisions, the Contracts Unit analyzed the updated maximum
- 107 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
financial obligation (MFO) for each contract and found the majority of the FY18 SD/MHSA contracts were below $5 Million MFO cap each which made them eligible for the MCL.
The MCL request submitted by BHSD was initially submitted to the Office of Budget Analysis (OBA) to ensure the requested contracts met MCL requirements. 48 of the 61 FY2018 SD/MHSA contracts were eventually approved for the MCL by the BOS on May 9, 2017: the BOS adopted a resolution delegating authority to Agency and/or Department Heads to implement the MCL for FY2018. The recommendation of the Contracts Unit to request MCL approval of the eligible SD/MHSA contracts in April 2017 helped facilitate the completion of the FY2018 SD/MHSA contracts by the target date since it largely minimized the number of contracts that had to go through the LF portion of the process. The other contracts-13 that required BOS approval were approved in June 2017.
Only 3 of the 61 SD/MHSA contracts were not executed at the start of the contract term date while the vast majority, 95percent of the contracts, were renewed on time. As illustrated, the FY2018 completion rate is again closer to the 94percent completion rate achieved for the FY2016 Term 1 SD/MHSA renewal process and within the 90-100percent target goal of the pilot project. Refer to the excel file for details.
Percent of SD/MHSA contracts executed and effective at the start of FY2018:
Of the three contracts executed after the start of the fiscal year: one contract was with the City of San Jose and required BHSD to follow the contract approval process required by the City, which involved the approval of the contract by their City Council; the contract item could not be included in the agenda until after the start of the fiscal year. As for the other two contracts, the signature process took longer than expected: one with the service provider-vendor and other with County Counsel.
- 108 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
8.2 Were the PIP results and findings presented accurately and clearly?
Are tables and figures labeled? ☐ Yes ☐ No
Are they labeled clearly and accurately? ☐ Yes ☐ No
☐ Met
☐ Partially Met
☐ Not Met
☐ Not Applicable
☐ Unable to Determine
8.3 Did the analysis identify: initial and repeat measurements, statistical significance, factors that influence comparability of initial and repeat measurements, and factors that threaten internal and external validity?
Indicate the time periods of measurements: contract renewal process
Indicate the statistical analysis used: _________________________
Indicate the statistical significance level or confidence level if available/known: _______percent ______Unable to determine
☐ Met
☐ Partially Met
☐ Not Met
☐ Not Applicable
☐ Unable to Determine
The MHP provided limited information regarding the factors that influence comparability and factors that threaten internal and external validity.
Although these may be implied, such as staff adhering to timelines and contract provider completion of contracts, it would benefit the MHP to include these for future elements to consider.
- 109 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
8.4 Did the analysis of the study data include an interpretation of the extent to which this PIP was successful and recommend any follow-up activities?
Limitations described:
Conclusions regarding the success of the interpretation:
Recommendations for follow-up:
☐ Met
☐ Partially Met
☐ Not Met
☐ Not Applicable
☐ Unable to Determine
The focus was on the contracting process, this does not meet criteria without any consumer focus.
Overall, the FY2018 Contract Renewal Process Pilot Project improved the Department’s administration and execution of the SD/MHSA contracting process.
BHSD has significantly improved the completion rate of the execution of SD/MHSA CBO service provider contracts for FY2018 with the new approach and enhancements described in the prior sections. In addition, the SD/MHSA CSR approval TAT was quicker than expected. Although the Department did not have a TAT baseline, since this information was not previously recorded, the average TAT approval of the SD/MHSA CSR Forms during the pilot project was three days. In addition, 95percent of the FY2018 SD/MHSA service contracts were executed at the start of the contract term date. The FY2018 renewal completion rate is closer to the 94percent completion rate achieved during the FY2016 Term 1 SD/MHSA contract renewal process and within the 90-100percent target goal of the pilot project. Please refer to Item 8 section for additional details.
Follow-up activities have included these:
• Due to the successful implementation and ease of use of the digitized SD/MHSA CSR Form and updated CSR Form review/approval process, the Contracts Team and Project Lead developed a redesigned CSR Form for non-SD/MHSA service agreements for use in DocuSign which was implemented in Fall 2017.
• For the FY2019 SD/MHSA contract renewal process, BHSD plans to continue the approach utilized in this pilot project, encourage new recommendations from team members involved in the contracting work and promote continuous operational enhancements that would benefit the County’s
- 110 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
Medi-Cal beneficiaries-clients/consumers, ensuring service contracts are in place at the start of every fiscal year.
• In addition, the Contracts Team worked with SCVHHS Information Services Department to include minor adjustments to the SD/MHSA CSR Form based on feedback/input provided by staff/users of the new SD/MHSA CSR Form. The updated SD/MHSA CSR Form will be rolled out as part of the FY2019 SD/MHSA contract renewal process. The Contracts Team plans to review and update the SD/MHSA CSR Form periodically to ensure the functionality/purpose of the form is relevant, up-to-date, in collaboration with Programs, Finance, and Administration.
• The Department will continue to track CSR Form approval TAT and completion rates of SD/MHSA contract renewals by fiscal year and perform any necessary adjustments to the contracting process to ensure the Department is on track for timely execution of SD/MHSA contracts.
Totals 0 Met 0 Partially Met 0 Not Met 0 NA 0 UTD
STEP 9: Assess Whether Improvement is “Real” Improvement
9.1 Was the same methodology as the baseline measurement used when measurement was repeated?
Ask: At what interval(s) was the data measurement repeated?
Were the same sources of data used?
Did they use the same method of data collection?
Were the same participants examined?
Did they utilize the same measurement tools?
☐ Met
☐ Partially Met
☐ Not Met
☐ Not Applicable
☐ Unable to Determine
The Department was unable to set a baseline CSR turnaround time (TAT) at the start of the project. Prior to the pilot project’s rollout, the initial submission date of the CSR Form by the CSR requester was not previously recorded.
Through the pilot project, BHSD is now able to document/record CSR submission dates and going forward the team can use the CSR approval TAT results from the FY18 pilot project as a baseline for future renewal contracting processes and can assist the Department to set standards and expectations around turnaround times for future contract renewal activities and update/make improvements as needed.
- 111 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
9.2 Was there any documented, quantitative improvement in processes or outcomes of care?
Was there: ☒ Improvement ☐ Deterioration
Statistical significance: ☒ Yes ☐ No
Clinical significance: ☒ Yes ☐ No
☐ Met
☐ Partially Met
☐ Not Met
☐ Not Applicable
☐ Unable to Determine
Overall, the FY2018 Contract Renewal Process Pilot Project improved the Department’s administration and execution of the SD/MHSA contracting process. BHSD has significantly improved the completion rate of the execution of SD/MHSA CBO service provider contracts for FY2018 with the new approach and enhancements described in the prior sections.
In addition, the SD/MHSA CSR approval TAT was quicker than expected. Although the Department did not have a TAT baseline, since this information was not previously recorded, the average TAT approval of the SD/MHSA CSR Forms during the pilot project was three days.
In addition, 95percent of the FY2018 SD/MHSA service contracts were executed at the start of the contract term date. The FY2018 renewal completion rate is closer to the 94percent completion rate achieved during the FY2016 Term 1 SD/MHSA contract renewal process and within the 90-100percent target goal of the pilot project. Please refer to Item 8 section for additional details.
- 112 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
9.3 Does the reported improvement in performance have internal validity; i.e., does the improvement in performance appear to be the result of the planned quality improvement intervention?
Degree to which the intervention was the reason for change:
☐ No relevance ☐ Small ☐ Fair ☒ High
☐ Met
☐ Partially Met
☐ Not Met
☐ Not Applicable
☐ Unable to Determine
The formation of a core team with representation from each of the contributing areas: Programs-CYF and A/OA System of Care, Finance, and Contracts helped facilitate the renewal process, troubleshoot with fellow team members, and represent their areas needs/concerns throughout the process.
The redesign of the Contracting Services Request (CSR) Form to include only vital/relevant information helped facilitate the approval process of the CSR.
The implementation of the redesigned CSR Form in digitized format for use in DocuSign, a digital transaction management system that provides electronic signature technology for facilitating electronic exchanges of signed documents, enabled efficient tracking and review/approval of CSR Forms.
The unbundling of service provider contracts into separate contracts and package by program division: F&C, A/OA, CJS, IBH, and SHS, provided each program area better control over their specific program contracts and expedite execution of contracts.
Conducting various meetings related to the pilot project with all parties involved: pilot project team, executive team, and CBO contract providers ensured everyone - staff/CBOS were all aware of the goal of the pilot project and roles and responsibilities of each area.
- 113 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
9.4 Is there any statistical evidence that any observed performance improvement is true improvement?
☐ Weak ☐ Moderate ☒ Strong
☐ Met
☐ Partially Met
☐ Not Met
☐ Not Applicable
☐ Unable to Determine
Through the pilot project, the Department was able to improve the operational efficiency in the administration and execution of SD/MHSA service contracts for FY2018: the approval turnaround time (TAT) of SD/MHSA CSR Forms during the pilot project was three days and 95percent of SD/MHSA contracts were in place at the start of FY2018.
This is the first time BHSD has been able to measure approval turnaround time (TAT) for each SD/MHSA CSR submission and calculate the TAT.
As illustrated, the FY2018 completion rate is again closer to the 94percent completion rate achieved for the FY2016 Term 1 SD/MHSA renewal process and within the 90-100percent target goal of the pilot project
9.5 Was sustained improvement demonstrated through repeated measurements over comparable time periods?
☐ Met
☐ Partially Met
☐ Not Met
☐ Not Applicable
☐ Unable to Determine
Due to the productive timeline with successful results, this was the first data collection. BHSD plans to continue to track and analyze timely execution of SD/MHSA contracts annually and also perform any necessary adjustments to the process in collaboration with programs, finance and contract teams to ensure contracts are executed timely every fiscal year and in place for the County’s MHP beneficiaries and enrollees.
Totals 0 Met 0 Partially Met 0 Not Met 0 NA 0 UTD
ACTIVITY 2: VERIFYING STUDY FINDINGS (OPTIONAL)
Component/Standard Score Comments
Were the initial study findings verified (recalculated by CalEQRO) upon repeat measurement?
☐ Yes
☒ No
- 114 -
Santa Clara County MHP CalEQRO Report Fiscal Year 2017–18
ACTIVITY 3: OVERALL VALIDITY AND RELIABILITY OF STUDY RESULTS: SUMMARY OF AGGREGATE VALIDATION FINDINGS
Conclusion:
This PIP was determined not to meet the criteria for a PIP secondary to the lack of consumer outcomes and the limited benefit to the consumer. The details were discussed on-site at the time of the review. The MHP was encouraged to continue to consult at the earliest stage with EQRO to determine the validity of its new PIP for the next review cycle.
The MHP was able to demonstrate a successful PIP in a brief time with greatly improved results as far as the contract process.
Recommendations:
Due to the success of the contract renewal process and its efficiencies the MHP intends to continue the efforts of the PIP.
The MHP demonstrated successful results and is encouraged that the MHP intends to continue these efforts at the improved processes for contract renewal.
Check one: ☐ High confidence in reported Plan PIP results ☐ Low confidence in reported Plan PIP results
☐ Confidence in reported Plan PIP results ☐ Reported Plan PIP results not credible
☐ Confidence in PIP results cannot be determined at this time