Components of a Learning System

David M. Williams, Ph.D.Improvement Advisor

Institute for Healthcare Improvement

D

D

The Early Years Collaborative • “Scotland will be the best place to grow up.”• Draws upon existing community infrastructure across the country• Focused on age-based workstreams:

• 15% reduction in the rates of still-births and infant mortality by 2015.

• 85% of all children within each CPP will reached all of the expected developmental milestones at the time of the child’s 27-30 month child health review, by end-2016.

• 90% of all children within each CPP have reached all of the expected developmental milestones at the time the child starts primary school, by end-2017.

5

D

Components of a Learning System 1. System level measures2. Explicit theory or rationale for system changes3. Segmentation of the population4. Learn by testing changes sequentially5. Use informative cases: “Act for the individual learn for

the population” 6. Learning during scale-up and spread with a production

plan to go to scale7. Periodic review8. People to manage and oversee the learning system

From Tom Nolan PhD, IHI

Workstream 1 AimTo ensure that women experience positive

pregnancies which result in the birth of morehealthy babies as evidenced by a reduction of

15% in the rates of stillbirths (from 4.9 per 1,000 births in 2010 to 4.3 per 1,000

births in 2015) and infant mortality

(from 3.7 per 1,000 live births in 2010 to 3.1 per 1,000 live births in 2015).

Workstream 2 AimTo ensure that 85% of all children within

each Community Planning Partnership havereached all of the expected developmentalmilestones at the time of the child’s 27 30‐month child health review, by end 2016.‐

Workstream 3 AimTo ensure that 90% of all children within

each Community Planning Partnership havereached all of the expected developmental

milestones at the time the child starts primary school, by end 2017.‐

Source: Brandon Bennett, IA

Big Aims

We are here!

Overall Project Measures vs. PDSA Cycle Measures

AchievingAim

Data for Project Measures: - Overall results related to the project aim (outcome, process, and balancing measures) for the life of the project

Adapting ChangesDuringPDSA Cycles

Data for PDSA Measures: - Just enough data- Quantitative data on the impact of a particular change - Qualitative data to help refine the change - Subsets or stratification of project measures for particular patients or providers- Collect only during cycles

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

Point Number

Pou

nds

of R

ed B

ag W

aste

3.25

3.50

3.75

4.00

4.25

4.50

4.75

5.00

5.25

5.50

5.75

6.00

Median=4.610

Mea

sure

Time

Run Chart - Data for Learning & Improvement

PDSA Measures Guide Learning about our testing.

Process Measures Guide Learning about how our testing is improving reliability of the process.

Outcome Measures Guide Learning about how the reliability of the process is achieving our aim.

Components of a Learning System 1. System level measures2. Explicit theory or rationale for system changes3. Segmentation of the population4. Learn by testing changes sequentially5. Use informative cases: “Act for the individual learn for

the population” 6. Learning during scale-up and spread with a production

plan to go to scale7. Periodic review8. People to manage and oversee the learning system

From Tom Nolan PhD, IHI

Early Years Collaborative

2 day LSExpert

meetingsCluster

meetingsSupports

Expert QI & Early Years faculty

Networking events

Listserv Site Visits

Phone conf Assessments

Monthly Reports via web

2 dayLS

A

P

D

S

A D

P

S

LS1 2 day Kickoff

D

S

P

A

2 dayLS

Key ChangesImprovementMeasures

May 2013 TBC Oct 2013Jan 2013

Pre-work

Oct 2012

Etc…

TBC

R

Reduce infant mortality

Societal Issues

Post-birth actions

Pre-birth actions

Smoking / Alcohol & Drug Misuse

Improved sharing of information

Improved management and quality of care

Improved joint working

Improved identification

Aim 1⁰ 2⁰

Theory of what actions will reduce infant mortality

Health

Improved rate of breastfed babies

Quicker diagnoses of Neonatal Abstinence Syndrome

Nutrition

Mental health & wellbeing

Attachment Improved stability / permanence for LAC

Improved money management

Improved access

Improved family centred response

Improved leadership, culture and planning

Improved teamwork, communication, skills and collaboration

Parenting skills

Identification & reasons for current resilience

WORKSTREAM 1 (conception to 1 year)

Theory of what drives infant mortality

Poverty

Quality Of Home Environment

Domestic Abuse & Violence

Workforce Issues

Transport, Community Capacity & Cultures

Access To Services

EmploymentDetailed aim:To ensure that women experience positive pregnancies which result in the birth of more healthy babies as evidenced by a reduction of 15% in the rates of:• stillbirths (from 4.9

per 1000 births in 2010 to 4.3 per 1000 births in 2015)

• infant mortality (from 3.7 per 1000 live births in 2010 to 3.1 per 1000 live births in 2015)

Version: 06/03/2013

R

Children have all the developmental skills and abilities expected of a 27-30 month old

Societal Issues

Child’s physical & mental health and emotional development

Carer’s physical & mental health and skills

Level of education Improved sharing of information

Improved joint working

Improved early identification

Detailed Aim:85% of all children within each CPP have reached all of the expected developmental milestones at the time of the child’s 27-30 month child health review by end-2016

Misuse of alcohol & drugs

Health

Improved child’s dental health

Improving child nutrition

Poverty

Quality Of Home Environment

Domestic Abuse & Violence

NutritionDisabilities & Mental health

Attachment

Improved stability / permanence for LAC

Improved money management

Improved management, planning and quality of services

Improved family centred response

Improved leadership, culture and planning

Improved teamwork, communication, skills and collaboration

Improving brain development and physical play

Parenting skills & knowledge

WORKSTREAM 2 (1 year to 30 months)

Identification & reasons for current resilience

Aim 1⁰ 2⁰

Theory of what drives developmental milestonesTheory of what actions will

ensure developmental milestones are reached

Workforce Issues

Transport, Community Capacity & Cultures

Early Learning & Play

Access To Services

Employment

Additional Support

Version: 06/03/2013

R

Children have all the developmental skills and abilities expected at the start of primary school

Societal Issues

Child’s physical & mental health and emotional development

Improved sharing of information

Improved management, planning and quality of services

Improved joint working

Improved identification

Detailed Aim:90% of all children within each CPP have reached all of the expected developmental milestones at the time the child starts primary school, by end-2017

Health

Improved child’s dental health

Improving child nutrition

Attachment

Improved stability / permanence for LAC

Improved uptake of benefits

Improved family centred response

Improved leadership, culture & planning`

Improved teamwork, communication and collaboration

Improving brain development and physical play

Early Learning & Play

WORKSTREAM 3 (30 months to start of primary school)

Identification & reasons for current resilience

Aim 1⁰ 2⁰

Theory of what drives developmental milestones

Theory of what actions will ensure developmental

milestones are reached at the start of primary school

Poverty

Quality Of Home Environment

Domestic Abuse & Violence

Workforce Issues

Transport, Community Capacity & Cultures

Access To Services

Employment

Carer’s physical & mental health and skills

Level of education

Misuse of alcohol & drugs

NutritionDisabilities & Mental healthParenting skills & knowledge

Additional Support

Version: 06/03/2013

R

Provide the Leadership System to support quality improvement across the Early Years Collaborative

Early Years Collaborative is a strategic priority & underpins all policy planning and operational activity

Early Years Collaborative values, culture and behaviours are modelled by all leaders at all levels

Infrastructure to support delivery of Early Years Collaborative

Place quality issues at the top of senior leader meeting agendas

Ensure that the senior team participates in Walk-rounds

Display data that depicts progress towards aimDetailed Aim:

Timely delivery of all three workstream “stretch aims”

Early years executive and operational leads are identified

Leaders can describe how they personally maintain early years focus within their working environment

Ensure the development of a measurement system used to understand and drive quality indicators

Build commitment with partners to focus on delivery

CPPs communicate the EYC with enthusiasm and consistency

Leaders illustrate how users are included in design, improvement, and delivery of Early Years

Measurement plan and priorities are established and triangulation with other key data

Leaders demonstrate their ability to set direction and engage and mobilise staff to constantly improve quality of service

Meet regularly with the Implementation Committee to track progress and remove barriers

Ensure a feedback mechanism for issues raised in Walk-rounds

Establish Programme Management and remove barriers

Add Early Years Collaborative and outcomes to the CPP agenda

Establish an EYC Implementation Committee

Assign a senior leader to each improvement area (Workstreams 1-3 and measurement)

Spread plan is in place for core and innovative work

Leaders facilitate change by cultivating innovation from intelligence, insights and wisdom of people working together

WORKSTREAM 4 (Leadership)

Aim 1⁰ 2⁰Theory of what drives leadership support Theory of what actions will

ensure leadership support

Strategy for capturing, celebrating and spreading innovation

Version: 06/03/2013

R

Components of a Learning System 1. System level measures2. Explicit theory or rationale for system changes3. Segmentation of the population4. Learn by testing changes sequentially5. Use informative cases: “Act for the individual learn for

the population” 6. Learning during scale-up and spread with a production

plan to go to scale7. Periodic review8. People to manage and oversee the learning system

From Tom Nolan PhD, IHI

A population, for which claims data exists and achieving Triple Aim results will not result in perverse economic loss.

Cannot be defined by a clinical condition (Diabetes) or issue (readmissions).

An approach to achieving the IHI’s The Triple Aim for a given population - from the perspective of a consumer health plan-less, FFS based, Medicare participating, not for profit, hospital->health system…

Note: The size of the rectangles is meant to be indicative of population size, not cost.

A population, for which claims data exists and achieving Triple Aim results will not result in perverse economic loss.

A sub-population, high cost and or high utilization people from the larger population.

People who have “fallen through the cracks” of our “rescue-care” system.

Cannot be defined by a clinical condition (Diabetes) or issue (readmissions). Note: The size of the rectangles is meant to be indicative of

population size, not cost.

An approach to achieving the IHI’s The Triple Aim for a given population - from the perspective of a consumer health plan-less, FFS based, Medicare participating, not for profit, hospital->health system…

A population, for which claims data exists and achieving Triple Aim results will not result in perverse economic loss.

A sub-population, high cost and or high utilization people from the larger population.

Sub-groups, people from the high cost high utilization sub-population that can be stratified based upon relatively similar needs.

People who have “fallen through the cracks” of our “rescue-care” system.

Cannot be defined by a clinical condition (Diabetes) or issue (readmissions).

Sub-groups based more on needs and less on conditions.

Note: The size of the rectangles is meant to be indicative of population size, not cost.

An approach to achieving the IHI’s The Triple Aim for a given population - from the perspective of a consumer health plan-less, FFS based, Medicare participating, not for profit, hospital->health system…

A population, for which claims data exists and achieving Triple Aim results will not result in perverse economic loss.

A sub-population, high cost and or high utilization people from the larger population.

Sub-groups, people from the high cost high utilization sub-population that can be stratified based upon relatively similar needs.

People who have “fallen through the cracks” of our “rescue-care” system.

Interventions intended to address the needs of high cost high utilization sub-groups, Plan Do Study Act cycles.

Cannot be defined by a clinical condition (Diabetes) or issue (readmissions).

Sub-groups based more on needs and less on conditions.

Some interventions will work and some will not. All should result in learning and start on the smallest practical scale.

Note: The size of the rectangles is meant to be indicative of population size, not cost.

An approach to achieving the IHI’s The Triple Aim for a given population - from the perspective of a consumer health plan-less, FFS based, Medicare participating, not for profit, hospital->health system…

Components of a Learning System 1. System level measures2. Explicit theory or rationale for system changes3. Segmentation of the population4. Learn by testing changes sequentially5. Use informative cases: “Act for the individual learn for

the population” 6. Learning during scale-up and spread with a production

plan to go to scale7. Periodic review8. People to manage and oversee the learning system

From Tom Nolan PhD, IHI

Repeated Use of the PDSA Cycle for TestingChanges That Result in

Improvement

Hunches Theories

Ideas

DATA

Very Small Scale Test

Follow-up Tests

Wide-Scale Tests of Change

Implementation of Change

Sequential building of knowledge under a wide range

of conditions

Spreading

AP D

S

AP

D S

APD

S

A P

DS

Sustaining the gains

48

Percent of Surgeries with Intraoperative Temp Control

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25Obs

Percent

Daily % cases in control

12

3

45

6

Mini-measure tracks improvement cycles

Scottish Borders - Run chart of run charts!R

Scottish Borders - PDSAs completedR

Test! Test! Test!• Burt Sandeman’s Story Post LS1:

– The Challenge To Be Quick• Burt Sandeman’s Story Post LS2:

– The F-Word• Looked after two-year olds:

– My Prediction Was Wrong

30

Components of a Learning System 1. System level measures2. Explicit theory or rationale for system changes3. Segmentation of the population4. Learn by testing changes sequentially5. Use informative cases: “Act for the individual learn for

the population” 6. Learning during scale-up and spread with a production

plan to go to scale7. Periodic review8. People to manage and oversee the learning system

From Tom Nolan PhD, IHI

Act with the Individual

• Dad’s Care• Asset Based Community Development in N. Ay

rshire

32

Components of a Learning System 1. System level measures2. Explicit theory or rationale for system changes3. Segmentation of the population4. Learn by testing changes sequentially5. Use informative cases: “Act for the individual learn for

the population” 6. Learning during scale-up and spread with a production

plan to go to scale7. Periodic review8. People to manage and oversee the learning system

From Tom Nolan PhD, IHI

Things to Consider to “Scale-up”• Determine full scale at project setup and milestones to reach full scale• Different changes may require different scale-up strategies • Consider different dimensions of structure

– Information technology– Physical (e.g. space, equipment, capacity)– Human resources (workforce organization and capabilities)– Financial– Learning system

• Use “5x” (5--25--125--625--3125---) thinking to predict/define the structural issues and set a path forward for testing– (What is working when testing with x that probably won’t work with 5x, ...?)

• Standardize processes (e.g. training, referral)• Understand oversight requirements as the system grows

Components of a Learning System 1. System level measures2. Explicit theory or rationale for system changes3. Segmentation of the population4. Learn by testing changes sequentially5. Use informative cases: “Act for the individual learn for

the population” 6. Learning during scale-up and spread with a production

plan to go to scale7. Periodic review8. People to manage and oversee the learning system

From Tom Nolan PhD, IHI

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12

0

10

20

30

40

50

60

70

80

Avg. Patient Satisfaction - Clinic B

Last 12 Quarters

Avg

% S

atis

fact

ion

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12

0

10

20

30

40

50

60

70

80

Avg. Patient Satisfaction - Clinic C

Last 12 Quarters

Avg

% S

atis

fact

ion

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12

0

10

20

30

40

50

60

70

80

Avg. Patient Satisfaction - Clinic D

Last 12 Quarters

Avg

% S

atis

fact

ion

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12

0

10

20

30

40

50

60

70

80

Avg. Patient Satisfaction - Clinic E

Last 12 Quarters

Avg

% S

atis

fact

ion

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12

0

10

20

30

40

50

60

70

80

Avg. Patient Satisfaction - Clinic F

Last 12 Quarters

Avg

% S

atis

fact

ion

Period Review

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12

0

10

20

30

40

50

60

70

80

Avg. Patient Satisfaction - Clinic G

Last 12 Quarters

Avg

% S

atis

fact

ion

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12

0

10

20

30

40

50

60

70

80

Avg. Patient Satisfaction - Clinic H

Last 12 Quarters

Avg

% S

atis

fact

ion

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12

0

10

20

30

40

50

60

70

80

Avg. Patient Satisfaction - Clinic A

Last 12 Quarters

Avg

% S

atis

fact

ion

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12

0

10

20

30

40

50

60

70

80

Avg. Patient Satisfaction - Clinic I

Last 12 Quarters

Avg

% S

atis

fact

ion

You are here! AimThe Work Remaining to Do

Keeping an eye on the journey

Components of a Learning System 1. System level measures2. Explicit theory or rationale for system changes3. Segmentation of the population4. Learn by testing changes sequentially5. Use informative cases: “Act for the individual learn for

the population” 6. Learning during scale-up and spread with a production

plan to go to scale7. Periodic review8. People to manage and oversee the learning system

From Tom Nolan PhD, IHI

40