Bill P. Martone President WPM Consulting - CTACNY P. Martone . President WPM Consulting . ......

The Community Technical Assistance Center of New York

Bill P. Martone President WPM Consulting


To obtain Continuing Education Unit credits, please 1. Please log into the webinar system via your computer, and view the

entire presentation 2. Go to https://sswforms.es.its.nyu.edu/ 3. Create a profile 4. Select today’s webinar 5. Pay the $15 processing fee 6. Take the Knowledge Test* *The Knowledge Test will be made available to you within the week. You will receive an email with this information within 24 hours of this webinar.

https://sswforms.es.its.nyu.edu/


December 18th Embarking on a New Path

http://www.ctacny.com/embarking-on-a-new-path-registration.html

January 15th

Culture- Driven Leadership http://www.ctacny.com/culture-driven-leadership-

registration.html



http://www.ctacny.com/culture-driven-leadership-registration.html










Performance Excellence: Measuring What Matters

A Leadership Webinar Series

Session 3

February 19, 2016


• Support you in identifying the strengths of your organization specific to achieving sustained positive outcomes for youth and families.

• Support you in increasing your understanding about your organizational capacity for developing or expanding community based services

• Hopefully increase your enthusiasm and understanding regarding specific strategies you can go back and utilize to improve services in your own programs


Line of Sight Management Use of Data Outcome Measurement Benchmarking Tools Q&A


We Covered: 1) Organizational Culture 2) Performance-based Culture 3) Performance Excellence – An Agency Journey 4) Leading Cultural Adaptation Materials in Appendix

Areas of Focus

Performance Excellence Areas of Focus

Proc

ess M

anag

emen

t

Leadership & Team Development

The Nature of Teams The Team Maturity Matrix

Sponsors, Coaches, Team Leaders Communication That Works

360° Evaluations Directive Supervision

Leadership as a System Leadership Assessment & Measurement

Process Management

The Process Organization Process Measures

Suppliers, Inputs, Process, Output, Customers (SIPOC) Rapid Improvement Technology

5 Step Improvement Process Plan, Do, Check, Act (PDCA)

So Today We Will Look at Line of Sight Management Data, Outcomes and Data Collecting Tools

Line Of Sight Management

The Vital Few People Aligned with Goals & Measures Linkage to Business Plans Fact Based Decision Making

Catch Ball Useful Information vs. Data Overload Reporting and Presentation

Div/Dept Plans, Goals & Metrics

Mgmt. Plans, Goals & Metrics

Team Plans, Goals & Metrics

Company Metrics

And Results

Sr. Exec Team

Executive Teams

Management Teams

Teams

Org/Dept Metrics

And Results

Mgmt. Metrics And

Results

Team Metrics And

Results

Strategic Plan

Company Plan, Goals &

Metrics

Line of Sight Architecture

Are You Drowning in Data?

Data Rich – Information Poor

Focus on Data • Data Collection

• What to Measure • Operational Definitions • Data Sources • Collection Methods • Frequency • Data Display

Data-driven decision making

Meaningful Data and Measures

Satisfaction Survey

Training Pre-Post Tests

Consumer Focus

Satisfaction Surveys

Safety Data

Financial Results

IT Systems

Turnover and

Leaves

Audit & Compliance

Results

Mindful Measurement Drill down to Customer Focus

Youth and Family

Engagement

Staff Deployment of Principles

Fidelity to Core Practice

Principles

Youth and Family

Satisfaction

Focus Groups


Utilization Data

Phone Calls w/ Families

Complaints & Grievances

Follow-Up Studies

MEASUREMENT DOMAINS DATA SOURCES

19

Our Cycles of Improvement

Line of Sight Reporting at

Leadership Meeting

Annual and Strategic Planning

Improvement Recommendations

Deploy Implementation

Plans

Data Monitoring, Analysis, and Consolidation

Satisfaction Survey

Training Pre-Post Tests

Consumer Focus


Safety Data

Financial Results

Turnover and

Leaves

IT Systems

Audit & Compliance

Results

October/November Individual Performance

Development Plan Data is Gathered

Environmental Scan Completed by ELT

December/January Review Environmental Scan Information and

Data* with LT

February LT Members Present Data and Conduct a S.W.O.T. with Each

Agency Department Then LT Review S.W.O.T. Data and Chooses Top 18

in Each Category

March/April LT and Business

Performance Begin Workforce Planning and

Budgeting ELT Selects Organizational Goals and Strategies and

Shares with LT

May/June Budget is Reviewed with

Board of Directors Departments Select Their

Annual Goals Aligned with Organizational Goals

July Budget and

Organizational Goals Approved by Board of

Directors

September Organizational Goals and Budget Communicated

Throughout Organization

Strategic Planning Cycle

Strategic Planning

California Benchmarking Initiative

Actually Two Initiatives

California Alliance

Benchmarking for Financial S w vival And Organizational

Excellence

A Practice lmprovement Project Why Benchmarking • Benchmarking is a powerfu l management tool for moving organizations toward financial, operational and

practice optimi zation Benchm:uking Objectives • Data for Advocacy • Individualized benchmarking reports organizations can use to compare their performance to others in the

state and throughout the U.S . • Organizational improvement by identifying potential best practices What is Benchmarking • Measuring and comparing results of key work processes with those of best performers • Giving performa nce data meani ng by providing an external context for understanding the information,

identifying both strengths and opportun ities for improvemcnt, and suggesting best practices by distinguishing top performe rs from others

Benchmarking Data • Confidential, easy to complete on-line surveys already in use by the National Alliance for Strong and Familie s

and Communities as well as CCCMHA • National Alliance survey contains administrntivc, residential, foster care/treatment foster care, outpatient,

family preservation, juvenile justice, and e arly chi ldhood/education items • CCCMHA survey contains financial, organizational, and clinical i tems • New item development for practice areas not already included • Organizlltioal climate survey also available for confidential online completion by employees

Benchmllrking Reports • National norms and California norms if at least 15 Alliance agencies in a specific area participate • Easy to use, individualized reports provide detailed percentile r:mkings for each din1ension mellsured, a very

clear picture of strengths and weaknesses relative to other organizations quickly emerges • Timel y standard and executive reports

Favorabllity Dashboard Overall Clinical Benchmarks

Favorable Unfavor.ble 16.67'4 22.58 Unt.vo11ble

33.33

Neutral Nr:utrel S0.00%

58.06"

Fina ncial Benchmarks Operational Benchmarks favonble Unfavorable

9.09 Unfavorable 14.29 Favorable

27.27'4 28.57

Neutr.I 57.14%

Percent Metrics Submitted 45.50%

Program ID: 999 Page 1of 3 Ma_rch 27, 2008

Sample Report - Restraint/Seculsion

Relevant Agency Information which may

affect the outcome Agencies

identif ied w ith a confidential

ID number

Data provided over a rolling 3-year period for trending

Reported data converted to make for an easy comparison

McKinley Bench Marking Project

Sample Report – Filtering Data

McKinley Bench Marking Project


Quality Indicator Metric Definition Performance Rating Trend Ratings

Favorable Comparable

Unfavorable Q1 Q2 Q3 Q4

Reliable Outcomes Restrictiveness of Living

Environment Scale (ROLES)

Percentage of youth discharged to a home-like environment. 80-100% 60-79% Less than 60%

Consumer Satisfaction

Youth, Family, and Adult Satisfaction Survey

Consumer’s overall mean score on satisfaction with services where 1=low satisfaction and 5=high

satisfaction.

4.0-5.0

3.0-3.99

2.0-2.99 Collected annually

Practice Process Incident Report Overall total incident rate, includes all incident types compared to the

historical average rate.

Below 1 standard deviation

Within 1 standard deviation

Above 1 standard deviation

Fiscal

Annual Forecast

Agency overall net operating income based on preliminary data.

Break even or surplus

Not Applicable

Deficit

Hathaway-Sycamores’ 4-Square of Overall Quality Indicators FY 2013-2014 – Quarter 2 (Oct - Dec)

Reliable Outcomes Consumer Satisfaction

Fiscal Practice Process

Report prepared by the Evaluation and Research Department

The following indicate the overall favorability trends for items:

A Data Dictionary is an Absolute Must!

Possible Residential Treatment Center Measures

• Examples

• Recidivism • Success in Community • Seclusion & Restraints • Satisfaction – Youth/Families • CANS/CAFAS/ROLES • Family Finding/Natural Supports • Follow-Up Studies


Residentially Based Services

Outcome Measures

Outcome Measures

Data Sources/ Data Collection

Frequency of data collection

Data Providers

Outcomes for Children and Families

1. Achievement of permanency: Children at RBS exit with legal permanency (adoption, guardianship and reunification), and any type of placement episode termination

CWS/CMS

Annual

BIS

2. Average lengths of stay (in group care and entire RBS period)

CWS/CMS Annual BIS

3. Rates of re-entry into group care and foster care of children enrolled in the RBS program

CWS/CMS Annual BIS

4. Analyses of the involvement of children or youth and their families in services planning and treatment (Do children and families have a sense of “voice and choice” in their treatment experience?

CANS, YSS, YSS-F

Semi-annual

RUM/Provider

5. Client satisfaction YSS, YSS-F Semi-annual Provider

6. Child safety: Substantiated maltreatment while at home or in group care during RBS period

CWS/CMS Annual

7. Child well-being: Positive placement changes and # of placement moves

CWS/CMS Annual

8. Child educational progress Child’s case file Annual Provider

9. Child and family voice and choice Child’s case file Annual Provider

10. The existence of a connection with a caring adult Child’s case file Annual Provider

Outcome Measures



Data Providers

Systems Operation

11. Use of the program by the County On-going process County

12. The operation of the program by the private nonprofit

On-going process Provider

Fiscal Outcomes

13. Payments made to the private nonprofit agency by the County

Fiscal Workgroup

14. Actual costs incurred by the nonprofit agency for the operation of the program

Fiscal Workgroup

15. The impact of the program on State and County AFDC-FC program costs

Fiscal Workgroup

16. The impact of the program on State and County Early Periodic Screening, Diagnosis, and Treatment (EPSDT) Program costs

DMH

17. The impact of the program on State and County Mental Health Services Act (MHSA-Proposition 63)

DMH

Outcome Measures



Data Providers

Safety S1. 99.6% of the children/youth who are enrolled in RBS do not have any new substantiated allegations as specified in California Health & Safety Code, Section 1522(b) while receiving services under this contract

CWS/CMS

Quarterly

BIS

S2. 100% of Corrective Action Plans (CAPs) are submitted on time and successfully implemented, including facility and safety deficiencies

Corrective Action Plans/Auditor Controller

Reports

Annual

Provider

S3. 98% of children/youth are free from child-to-child injuries while in the residential site.

Child’s Case File/Facility Review Reports/

SIR/I-Track

Annual

Provider & Wrap Admin

S4. 94% of the children/youth who are enrolled in RBS do not have any new substantiated allegations within one (1) year after graduating from RBS

CWS/CMS

Annual

BIS

Permanency P1. 75% of youth that graduate from RBS will not have a subsequent out-of-home placement after six (6) months

CWS/CMS Semi-annual BIS

P2. 85% of families whose children/youth graduating from the RBS Demonstration Project continue using community based services and supports six (6) months after graduation

Follow-up Reports/

POC

Bi-annually

(Dec & June)

Provider

P3. CONTRACTOR will maintain an overall average length of stay of ten (10) months or less (in Residential)

CWS/CMS Child’s Case File

BIS Provider

P4. 80% of children/youth enrolled will have at least five (5) adult family members and fictive kin (non-relative) identified within ten (10) months of enrollment

POC Child’s Case File

Provider

P5. CONTRACTOR will facilitate 100% contact of approved connections


Provider

Outcome Measures



Data Providers

Well Being

WB1. 70% rating of family and youth satisfaction with services

YSS/YSS-F WFI-4/CAFAS

Semi-annual

Provider

WB2. 70% of youth demonstrate improvement on the behavioral/well-being measures

CAFAS

Semi-annual

Provider

WB3. At least 51% of CFT is comprised of informal supports


Semi-annual

Provider

WB4. 75% of children/youth maintain at least an 80% school attendance rate or improved attendance rate from previous quarter.

Child’s Case File

School Report Card

Quarterly

Provider

Displaying Data

Tools Built and/or Utilized

Meeting Minutes Format

Fish Bones

Charters

Five Step Model

Storyboard

Holding the Gains Four

Panel

Root Cause Prioritization

Matrix

Project Life Cycle

Meeting Agenda Format

Project Out Briefing

Templates Kaizen

Action Item List

Improvement Plan Format

Meeting Evaluation Form & Graphing Tool

Project Evaluation Worksheet

SIPOC

Kaizen (改善)

Kaizen is the practice of continuous improvement. Kaizen was originally introduced to the West by Masaaki Imai in his book Kaizen: The Key to Japan’s Competitive Success in 1986. Today Kaizen is recognized worldwide as an important pillar of an organization’s long-term competitive strategy. Kaizen is continuous improvement that is based on certain guiding principles:

Good processes bring good results Go see for yourself to grasp the current situation Speak with data, manage by facts Take action to contain and correct root causes of problems Work as a team Kaizen is everybody’s business And much more!

One of the most notable features of kaizen is that big results come from many small changes accumulated over time. However this has been misunderstood to mean that kaizen equals small changes. In fact, kaizen means everyone involved in making improvements. While the majority of changes may be small, the greatest impact may be kaizens that are led by senior management as transformational projects, or by cross-functional teams as kaizen events.

https://en.wikipedia.org/wiki/Help:Installing_Japanese_character_sets

https://en.wikipedia.org/wiki/Help:Installing_Japanese_character_sets

https://en.wikipedia.org/wiki/Japanese_language








Kaizen Rapid Business Improvement Process (1- 5 weeks)

Data Collection

Identify Waste And Defects

(Opportunities For Improvement)

Analysis of Causes

(Fishbone)

Root Cause Analysis

Leverage Points

Preliminary Solutions

Improvement Plan Alternative Solutions

1.

2.

3.

4.

Solution

Savings $$$

Cost

Plan

Why?

Why?

Why?

Why?

Why?

OFIs OFI

Tools People

Layout Structure

Root Cause

OFIs

Cause Cause

Cause Cause

Cause

Cause

5 Whys

Final Solutions

Story Board

57 57

We Utilized The Team Handbook To Learn About Tools The Following Examples Are From The Handbook

Ishikawa Cause & Effective Diagram - Fishbone

The Tool • Developed by Kaoru Ishikawa • Diagram called fishbone because it looks like a fish skeleton with head spine and bones • Unlike others this tool does not analyze data

• Helps people organize their ideas and theories about causes • These theories later become verified with data

The Fishbone • The problem under investigation is described in a box at the head of the diagram • A long arrow pointing to the head is the spine and indicates the problems feed into

the spine that might cause the problem at the head • The larger bones feeding the spine represent the main potential causes of the problem • The smaller bones represent deeper causes of the larger bones • Greatest challenge is to have the bones show cause and effect relationships


An Easy Problem – Bad Tasting Pizza


Problem – Appointment No Show

Tools Most Likely Used In The 7 Step Method Discussed Last Time

Had to focus the organization around the importance of using data and figure out what was

important for us to know – vital few – data dictionary Data Collection & Outcomes Reporting Takes Resources

• Dedicated Research & Evaluation Staff • Funded primarily through our Mental Health Contract • Utilized a number of graduate students in the department with small stipends

Key focus coming out of 2008 was assuring adequate resources for the agency

• Paying attention to staff’s billing became labeled as “Its all about the money”

As reports were produced and shared transparency throughout the agency occurred • Everyone knew how everyone else was doing

While the tools were extremely useful it took time to train staff in their use and help staff

understanding the right tools to use for a given analysis

The Kaizen process took a significant investment of staff time up front but cut significant amounts of time off the duration our our projects

Obstacles & Challenges To Using Data/Tools/Outcomes/Line of Sight Reporting

Contact Information

William P. Martone President WPM Consulting, Inc. 1516 Beech Street South Pasadena, CA 91030 Cell Phone: 626-831-6850 E-Mail: [email protected]

mailto:[email protected]


Yvette Kelly [email protected]

mailto:[email protected]

Appendix

Some Excerpts From Hathaway-Sycamores Website Comprehensive Outcomes Report

www.hathaway-sycamores.org

Bill P. Martone President WPM Consulting - CTACNY P. Martone . President WPM Consulting . ......

Documents

Transcript of Bill P. Martone President WPM Consulting - CTACNY P. Martone . President WPM Consulting . ......