Agenda
• Introduction
• Overview of Human Services Performance Management System
• Overview of Measure Development Process
• Overview of Long Term Services & Supports
• Overview of Balanced Sets of Measures
• Measure Development Exercises2
System History and Timeline
Service Delivery Act Legislative Report Development Implementation
2009 legislation establishing Steering Committee on Performance and Outcome Reforms
Legislative report issued in 2012 with specific recommendations; legislation establishing system and Council in 2013
Development of the system begins, including hiring staff, creating a framework, and issuing baseline reports
PIPs are implemented and improvement assistance has begun; new measures and partnerships are being developed
4
Mission and Values
Mission: to improve outcomes for people through creativity, flexibility, accountability,
collaboration, and performance management
Values: accountability, collaboration, continuous improvement, equity, flexibility,
inclusiveness, reliance on data, sustainability, and transparency
No single entity can achieve client outcomes alone.
We need to work together to improve lives for the people we serve.
5
Performance Management System Goals
The Human Services Performance
Management System creates an
opportunity for the Minnesota
Department of Human Services, counties,
and community partners to work more
closely together to improve the lives of
people served.
6
Performance Management System Components
• Measure Development: Create balanced sets of program measures that provide a true picture of how we are doing.
• Performance: Assure performance standards are met.
• Improvement: Provide improvement assistance and strategies for counties.
• Advance Equity: Maintain an inclusive process which is considerate of diverse perspectives and reduces disparities in the Human Services system.
7
Outcomes and Measures, Continued
Measures do not currently exist for outcomes five and six. The Performance Management team continues to work with stakeholders to develop additional system measures for these outcomes.
Measure Development Approach
• Bring together stakeholders from DHS, Counties, and the community to develop shared measures.
• Performance management is more than developing measures. A successful system integrates the use of data into all facets of the organization.
• We meet our partners where they are on their performance management journey to develop a system that meets their organizations unique needs.
• We leverage our partners expertise throughout the process of developing a performance management system.
11
Team Roles and Responsibilities
Team Team Members Team Responsibilities
Steering Committee
• Representatives from DHS• Representatives from Counties• Agency & County Performance
• Provide background information and understanding of DHS Long Term Services and Supports
• Provide feedback on key deliverables• Guide measure development work
Data Team • Representatives from Policy, Program, and Data areas
• Conduct research and provide information about DHS Long Term Services and Supports
• Collect information needed to review DHS Long Term Services and Supports
Measure Development Team
• Representatives from DHS• Representatives from Counties• Provider representatives• Partners and advocates
• Provide subject matter expertise, experience, and strategic thinking to develop Performance Measures for DHS Long Term Services and Supports
Office of Strategy and Performance Project Team
• Carol Becker, project manager• Olufemi Fajolu, county data• Gary Mortensen, manager,
Human Services Performance Management
• Manage the measurement development, gaps analysis, and data infrastructure phases of the project
• Facilitate the outcome measures development conversations• Collect and manage project information from County and DHS teams• Synthesize key findings to develop measurement inventory, data inventory and
program inventory.
12
Project Overview (page 1 of 2)
• Steering committee of DHS and county staff identifies participants and reviews meeting agenda.
• Host meeting of DHS, county and community folks to discussion current measures, the Balanced Framework and potential new measures. • Outcome: List of potential new measures.
• Give list of potential new measures to data team to determine what is feasible.
• Get participants from meeting back together to discuss which potential new measures:• are feasible.• have data quality issues and what could be done about that.• could be built into future system development.• will not be possible anytime soon.
13
Project Overview (page 2 of 2)
• For measures that are currently possible, move them into the existing measurement processes.
• Develop draft measures for each county. Give them a year to adjust their processes before moving the measure into the PIP process.
• For measures that have data quality issues, carry out discussions among DHS, counties and providers to identify what is needed to improve data quality.
• For measures that could be included in future system development, ensure that these needs are included in planning documents.
• For measures that are not possible, hold further discussions on how to address these needs.
• Repeat process as necessary.
14
Statute Language
Statutory Requirement - 402A.10 Definitions
Subd. 1a.Balanced set of program measures.
A "balanced set of program measures" is a set of measures that, together, adequately quantify achievement toward a particular program's outcome. As directed by section 402A.16, the Human Services Performance Council must recommend to the commissioner when a particular program has a balanced set of program measures.
Subd. 4d.Performance management system for human services.
A "performance management system for human services" means a process by which performance data for essential human services is collected from counties or service delivery authorities and used to inform a variety of stakeholders and to improve performance over time.
17
Research and Planning
Research
• Partnered with the Management Analysis and Development team at MN Management and Budget to guide us in the development of planning Balanced Sets of Measures
• Researched scorecards and performance measures used by other organizations
• Interviewed organizations with strong performance management systems
• Presented to MACSSA Forum for feedback
18
Balanced Sets of Measures – Guiding Principles
• A list of ten guiding principles was created to ensure measures are developed with people and communities at the core, the stakeholders involved, and using the principles of Results-Based Accountability.
• The measures developed will represent the core components that create a strong program so that we can identify and promote what is working, identify systemic issues and work with counties to improve performance.
19
Balanced Sets of Measures – Categories
Measure categories used to assess balanced set of program measuresCategory How it will be used
Client Engagement
How satisfied/ respected are the people receiving services?
EquityImprove diverse groups experiences and outcomes? (i.e. racial disparities measures)
FinancialWhat are the costs of providing these activities? (i.e. program ROI)
Operations How efficiently do we do our work? (e.g. staff training, staffing levels, data collection, etc.)
Program Effectiveness
How well do we do our work? (e.g. application processing times, quality of services, impact on individuals)
20
Balanced Sets of Measures – Sample Measures
Category Sample Measures
Client Engagement
• Were you treated with dignity and respect?• Were staff members courteous and helpful?• Staff had the knowledge to handle my request?
Equity• County staff reflects population served• Rates of out-of-home placement by race• Family Assessment versus Investigation by race
Financial• Cost effectiveness of services• Return on Investment• Timeliness of payments / submitting requests for reimbursement
Operations • Percentage of staff trained in a timely manner• Staff to case load ratio• Data integrity
Program Effectiveness
• Percent of applications processed within one business day• Number of hours between initial contact and crisis assessment• Percent of people with paying jobs
21
Balanced Sets of Measures – Criteria
Measure criteria used to select and create a balanced set of measures
Criteria Description of Criteria
DataPower
Timely, reliable, currently exists, easily accessible, high validity, low
human error
CommunicationPower
Easily understood, compelling to stakeholders
ProxyPower
Says something of central importance, matches direction of other
measures in the mix
22
Key RBA concepts
• Two levels of accountability: population and program
• Three questions for program accountability
• How much did we do?
• How well did we do it?
• Is anyone better off?
26
Results accountability - Two Parts
• Population accountability:
about the well-being of
whole populations
For Communities – Cities – Counties – States – Nations
• Performance accountability:
about the well-being of
client populations
For Programs – Agencies – Service Systems
27
Definitions
Population
• Result or OutcomeA condition of well-being for children, adults, families or communitiesChildren born healthy, children ready for school, safe communities, clean environment, prosperous economy
• Indicator or BenchmarkA measure that helps quantify the achievement of a resultRate of low-birthweight babies, Percent ready at K entry, crime rate, air quality index, unemployment rate
Performance
• Performance MeasureA measure of how well a program, agency or service system is working
Three types:
• 1. How much did we do? • 2. How well did we do it? • 3. Is anyone better off?
=Customer Results
28
Performance measure quadrants
Quadrants
Quantity Quality
Effo
rt How much did we do?
How welldid we do it?
Effe
ct Is anyone better off?
(How many?)
Is anyone better off?
(How much?)
29
Example performance measures by quadrant
Quadrants
Quantity Quality
Effo
rt Number of Students
Student-teacher ratio
Effe
ct Number of high school graduates
Percent of high school
graduates.
30
Types of measures found in each quadrant
Quadrants
Quantity Quality
Effo
rt
How much did we do?
• Number of clients/customersserved
• Number of activities (by type of activity)
How well did we do it?
• Staffing: client/staff ratio, workload ratio, staff turnover rate, staff morale, percent of staff fully trained, percent clients seen in their own language, worker safety, unit cost
• Service delivery: percent timely, percent of clients completing activity, percent correct and complete, percent meeting standard
Effe
ct
Is anyone better off?• Number of clients improved
(point-in-time vs. point-to-point)
Is anyone better off?
• Percent with skills/knowledge (e.g. parenting skills)
• Percent with an attitude/opinion (e.g. toward drugs)
• Percent with behavior (e.g. school attendance)
• Percent in desired circumstance (e.g. working, in stable housing)
31
The matter of use
1. The first purpose of performance measurement is to IMPROVE PERFORMANCE.
2. Avoid the “performance measurement equals punishment trap.”
• Create a healthy organizational environment.
• Start small.
• Build bottom-up and top-down simultaneously.
32
Different Descriptions of Progress
1. Data
a) Population indicators: movement for the better away from the baseline
b) Program performance measures: customer progress and better service
How much did we do?How well did we do it?Is anyone better off?
2. Accomplishments: Positive activities, not included above
3. Stories behind the statistics that show how individuals are better off
33
Ground Rules
•All voices hold equal weight
•Respect others, respect yourself
• This is only one step in the process
•Do not assume we are all on the same page
35
Table Introductions
• Share your name
•What “hat” you’re wearing today (your role or roles)
• Ice Breaker
36
Desired Measures for Long-term Services and Supports
Services Measures
Timeliness
Percent of initial assessments scheduled within 20 days of person accepting the assessment
Percent of people with financial application review completed in 10 days
Percent of people with coordinated services and supports plan in 10 days
Percent of clients going through the State Medical Review Team process in X number of days
Percent of waivered services authentications within X number of days
Percent of people that access safe plan services with 30 days of assessments
Percent of MA liability completed in X number of days
Percent calls or emails returned in 2 days
Percent MN choices assessments completely in X number of days
Percent of cases with time from intake to waiver within X number of days
Acceptance of service
Percent of accepted assessment
Informed Choice
Percent of clients made aware of informed choice
Care Plan Created
Percent of cases with a care plan created
Plans documenting goals
Percent of people’s care plans document their goals
Assessment Components Complete
Percent of assessment components completed
Informal Supports
Percent of people with informal, non-eligibility based supports are incorporated into their plans
People bringing Representatives
Percent of people that bring a representative(s) of their choice to an assessment or meeting
Assessed Needs versus Services Received
Percent of assessed needs that services were actually delivered
Spending
Percent of authorized spending versus amount claimed.
Employment
Percent of people employed
Percent of people earning more than $600/month
Percent of people employed 6 months or longer
Percent of people with income increases
Living Arrangements
Percent of people living in their own home versus other settings
Percent of people with complexity/higher needs behavior medical characteristics (e.g. autism)
served in own home
Percent of people living in their preferred setting
Percent of peoples moving involuntarily
Percent of individuals moved that wanted to move
Reduction in supports
Percent of cases with reductions in supports after engagement (such as reductions in ER visits)
Continuity of Services (Service Gaps)
Percent of cases with no break in services
LTSS tool
Percent of cases where the case manager completes the LTSS evaluation tool
Exiting
Percent of people who didn’t need assistance within a certain timeline
Operational Measures
LTSS Survey Tool Impacts
Percent of cases where issues were identified with the LTSS tools and corrective action was
taken.
Percent of services implemented that were evaluated using the LTSS improvement tool to
ensure effectiveness
Cost
Cost of providing services per case
Timeliness of Data Entry
Percent of SA entry into MMIS prior to previous year’s SA end date
Appeals
Percent of cases resulting in appeals
Availability of Services
Ratio of services needed within a geographic area to the amount of provider capacity within that
geographic area
Number of waivers denied services due to lack of availability of service
Language other than English
Percent of people served by a person that spoke their language?
Number/proportion that need/request interpretive services that received those services (in-
person, language live, tele-interpretation)
Number of translated documents/info available
Percent of services provided in native language
Number of forms translated
Client Contacts
Percent of clients whose case manager met face to face at least two times in a year
Percent of phone calls returned
Minimum number of client contacts provided
Percent of clients who keep metal health services and case management into waiver services
Average number of case managers a client has over a given amount of time
Outreach Efforts
How many outreach activities were undertaken in a year
Timely Provider Payments
Percent of providers paid within the timelines in their contracts
Planned versus actual services
Cost of planned services (CSSP) versus actual utilization of services
Data Entry
Percent of SA entry into MMIS prior to previous year’s SA end date
Costs
Average cost per day (by program, residential versus nonresidential)
Timely payments – Individuals
People get their money (personal needs allowance) in a timely manner
Children transitioning to Adult Services
Number of family who receive a culturally specific explanation of adult service system/plan
before child exits school-based services
Staff Measures
Staffing Diversity
Staff diversity representative of county demographics
Staff Retention
Staff turnover rates
Staff Training
Number of staff trained in the core curriculum (CM 101)
Average length of time to train new employees from hiring to case load
Percent of case managers capable of doing waiver management
Percent of case managers who completed mental health training
Percent of staff trained in person-centered approaches
Percent of supervisors that have competency across LTSS programs
Case Load
Caseload ratio to staffing and a threshold of acceptable caseload
Client Satisfaction Measures
Client Survey Questions
Percent of people that learned of a new support that they felt could help them
Percent of people who felt they could take reasonable risk in their life
Percent of people who feel they have control over their life
Percent of people who feel they have control & choice over the support they receive
Percent of people who feel they have the options available they want
Percent of people who felt they had the right level of support
Percent of people who felt their support are culturally responsive
Percent of people reporting information was helpful
Percent of surveys completed
Percent of people from under-represented groups tell us they feel respected at same/similar
ration as general population or as compared to other sub-groups
Percent satisfaction: housing, work, socialization, living in an environment of choice
Percent of people who report that their plan helps them achieve their goals aspirations
Percent of people who know who to contact for what
Percent of clients reporting increased satisfaction with case management support
Percent of people who know how to access help if they want or need it
Percent of people who report that their support plans meet their needs
Percent of people who report that their plans help them achieve their goals/aspirations
Top Related