Measuring Quality and Excellence:Identifying Indicators

April 2019

Margaret Fitch RN PhD

Why We Need Measurement

Measurement allows identification of those with difficulties and monitoring of their status over time

Important to be able to measure the impact of interventions in a standardized way

Measurement facilitates comparisons and quality improvement

“If you can not measure it, you cannot improve it” ~Sir William Thomson

Measuring ‘service’ performance

Access

Quality

‘Client’ satisfaction

Efficiency

Outcomes (service, client)

Measuring quality

“Sensitive” indicators used to measure quality of {named} care delivered

defined as “ those outcomes that are relevant, based on

program/discipline scope and domain of practice, and for which there is empirical evidence linking inputs and interventions to the outcome for patients”

Reflection

Make a list of 5-8 variables you think ought to be measured to show that nursing care occurred and made a difference.

Share your list with your neighbour…

Quality of Care (CIHI)

Quality of care contributes to patient safety and health outcomes; considers how well health care services are provided to patients.

Quality of Care addresses these questions: Do these services measure up to health care evidence?

Are they patient-centred?

Do they produce desired health outcomes?

Do they contribute to patient safety?

Outcomes of care are the direct results of the care patients receive

Urgency to measure outcomes and demonstrate quality care

Driving influences: Patient safety, fiscal concerns, patent expectations

Mounting expectation to demonstrate quality of nursing care delivered

{discipline} sensitive indicators are proposed as a way to meet this demand

Patient reported outcomes are useful to report on quality

Patient reported outcomes can be used to guide clinical care, assist in quality improvement, allow benchmarking and comparative effectiveness research

Instruments need to be reliable, measures sensitive to change, and outcomes associated with nursing interventions

Performance Indicator

Definitionnumerical measures that can

be used as a guide to monitor process performance

data often requires further investigation to isolate actual problem/pinpoint solution

What do we need to measure?

Indicator Framework

Structure Process Outcomes

Safe

Effective

Person Centred

Access

Equitable

Efficient

Qu

ali

ty D

ime

nsi

on

s

Program Design Dimensions

Indicators needed at all levels

Examples

Where data are

collected

Examples of variables

At point of patient care Pain, constipation, delirium,

nausea

Quality of life

At program or service

level

Experiences, perspectives on

care, satisfaction

What worked well

At system level

(regional, national)

Emergency visits

ICU admissions

Home care visits

Completing a standardized symptom screening tool

Examples of indicators (CIHI) Accessibility

Comprehensiveness

Continuity of care

Integration

Appropriateness of care

Effectiveness

Safety

Competence or capability

Patient experiences

Equity

Productivity or technical efficiency

Expenditure or cost

Responsiveness/trust in the health system

Efficiency

Healthy lives or health status improvement

Efficient allocation of resources

Innovation and capacity to improve

Effectiveness of specializedpalliative care teams: lit reviewZimmerman et al, JAMA, 2008

Indicators Caregiver satisfaction, Quality of life

Patient satisfaction, quality of life

Symptom distress: Pain, dyspnea, constipation, sleep

Place of death

Cost

Advanced directives

Medications

Treatment planning attitude

Psychosocial, depression , anxiety

Spiritual well being

Health care professional knowledge

Social support

Referral

Candidate National Indicators (Canada draft)

Place of death (home, hospital, preferred location)

# days in acute care setting during last 30 days of life

% cancer deaths occurring in hospital

New chemotherapy within the last two weeks of life

ICU Admission within the last 30 days of life

Emergency department visit in the last two weeks of life

Home care within 6 months of death

Nurse sensitive indicators:Literature review(Burston et al., 2013)

Exploration of nurse sensitive indicators has primarily focused on:

Relationship between structural variables and patient outcomes in acute care settings

Potential indicators for specific patient groups and nursing roles

Selection, reporting and sustained use are key aspects

Evidence inconsistent due to disparity in definitions used, data collected and analysis methods

Consensus regarding indicators to be used and integration into daily practice are required

Indicators must

Measure important phenomena

Be scientifically sound

Provide useable information

Be feasible to collect

Be clear regarding context in which they are measured

Nurse sensitive indicators

Indicators must be measureable

Data must be obtained at relatively low cost

Data gathering must not place undue additional burden on the clinical team

Significant variation in the indicator must be attributable to nursing

There must be wide applicability within settings, applying to a large number of patients

Variation attributable to nursing

The phenomenon is recognized as important

There is a recognized contribution of nursing

There is evidence to support sensitivity to nursing

Nurses must ‘own’ responsibility (in terms of legitimate authority, self perception, and sphere of practice)

Variation in outcome attributable to nursing must be substantial or the link between a nursing structure or process and the outcome must be strong

Example

Nursing sensitive outcomes and indicators in chemotherapy

Griffiths, Richardson & Blackwell

United Kingdom

November, 2009

Conclusions

Wide number of possible indicators but evidence weak regarding link to outcomes

Clearest direct impact of nurses was to safety and experience dimensions of quality

Impact on treatment effectiveness is indirect and mediated by the ability to support patients in managing toxicities (in turn based on link to accurate problem identification and access to therapies)

Attributes of nursing sensitive indicators identified via concept analysis(Frequency>10) (Heslop & Lu, 2014)

Structural – setting related

Hours of nursing care per patient day

Nurse staffing (staff mix, skill mix, staff ratio)

Process (none)

Outcome – patient related

Safety

Pressure ulcer

Falls and falls with injury

Nosocomil selective infection

Perception

Patient/family satisfaction with nursing care

Nursing sensitive indicators(Heslop & Lu, 2014)

Structural

Patient related :

patient characteristics

Nursing related:

RN Education level, years of experience

Setting related:

Hours of nursing care per patient day

Nursing staffing (staff mix, skill mix, staff ratio)

Patient acuity

Patient turnover

Workload intensity

% hours suppled by RNs

Organizational factors of nursing practice environment

Support for nursing education

Manager ability leadership and support

Relationships with other practitioners

Nursing sensitive indicators(Heslop & Lu, 2014)

Process

Nursing related

Nursing intervention/nursing practice

Setting related

Nursing documentation/nursing care plan

Outcome-patient related

Safety Pressure ulcer

Falls and falls with injury

Nosocomial selective infection

Nosocomial urinary tract infection

Medication error

Pneumonia

Vein system complication

Failure to rescue

Perception

Patient/family satisfaction with nursing care

Patient/family satisfaction with pain management

Use of health care

Length of stay

Waiting time for nursing care

Unplanned hospital visits post-discharge

Functional status

Symptom resolution/reduction

Clinical management

Nursing sensitive indicators(Heslop & Lu, 2014)

Creating/analyzing nursing quality database(Aydin et al., 2004)

Created a large ongoing reliable and valid quality database to examine nurse staffing and patient care outcomes in acute care hospitals

Contains prospective nurse staffing, process of care, and patient outcomes based on ANA nursing quality indicators

Variables: Hospital nursing staffing, patient days, patient falls, pressure ulcer and restraint prevalence, RN education, and patient perceptions of care

Voluntary contribution; largest state database

Steps in indicator development

Establishing the purpose of the indicators

Designing the conceptual framework

Selecting and designing the indicators

Interpreting and reporting the indicators

Maintaining and reviewing the indicators

Identify purpose and audience for indicator

Who will be responsible for the final selection and publication of the indicators?

How will key stakeholders be involved?

Will an expert group be established to provide specialist advice?

Will public consultation be undertaken?

How will the indicators be sustained and funded over time?

Design conceptual framework

Offers a guide for developing indicators

Provides a formal way of thinking about the topic area

Helps ensure relevant and balanced selection of indicators

Aids in understanding links between indicators

Provides useful device for organizing and reporting in a structured and meaningful way

Selecting/designing indicators Valid and meaningful

Sensitive and specific to the underlying phenomenon

Statistically sound

Intelligible and easily interpreted

Related where appropriate to other indicators

Allows international comparison

Ability to be disaggregated over time

Consistency over time

Timeliness

Linked to policy or emerging issues

Compel interest and excite

Indicators of APN on outcomesIngersoll et al, 2000

TOP TEN: Delphi with practicing APNs in USA

satisfaction with care delivery,

symptom resolution/reduction,

perception of being well cared for,

compliance/adherence with treatment plan,

knowledge of patients and families,

trust of care provider,

collaboration among care providers,

frequency and

type of procedures ordered

quality of life.

Example (www.hope.org)

Strategy

Nurses conduct bedside change of shift reporting

Measurement

Nursing staff and physician satisfaction scores

Patient satisfaction scores

Outcomes

Increase in staff satisfaction scores

Increase in patient satisfaction scores

Improved ability of nurses to prioritize work

Decrease in staff time

Decrease in handoff errors

Example (www.hope.org)

Strategy

Patients and families participate in rounds

Measurement

% families that participate in rounds

Number of stories in which new information is discovered from family

Length o time for rounds

Patient satisfaction scores

Staff satisfaction scores

Length of stay

Outcome

Length of stay decreased

Increase in satisfaction scores

Decrease in readmissions rate, safety outcomes

Change in percentage of misses versus errors

Example (www.hope.org)

Strategy

Patients and families have access to medical records or online portals for personal health information

Measurement

Number of times the portals are used

Surveys of patient’s use of portals

Outcome

Increase in patient satisfaction scores

Increased medication adherence

Nursing sensitive screening measures (Stalpers et al., 2016)

Deliberative screening – detection of risks otherwise may not be noticeable

Nursing (mandatory) screening -Delirium malnutrition, pain and pressure ulcers

In hospital records ranged low of 59% for delirium to high of 94% for pain

Interpreting and reporting indicators

Bridges the gap between measurement and understanding

Report objectively and in policy neutral manner; over context as appropriate

Keep audience in focus

Ensure dissemination strategy facilitates data report reaching intended audience

Examples of indicators

% patients with a written care plan

my health care provider and I worked together to set personal goals to manage my health care

My health care provider listened carefully to me at today’s visit

I understand my health care provider’s advice and what I need to do to manage my illness

My health visit helped me to gain confidence in managing my health problems

You and your family were able to participate in decisions about your care

Clinician and staff respected your choice of whether or not to have a family member or friends with you during your care

Clinician staff respected your family’s cultural and spiritual needs

Maintaining and reviewing indicators

Indicators need on-going assessment

Be open to discussion and refinement or modification

Consultation with stakeholders is critically important to obtain feedback about the relevancy and usefulness of the indicator

What do you think now?

Has you thinking about your list of indicators changed in any way?

Talk it over with your colleague beside you?

Top ten measures that matter:nursing hospice and palliative care

March 2017

Facilitator: Margaret Fitch

Measure 1

NAME:

hospice and palliative care – comprehensive assessment

Definition:

Percentage of patients for whom a comprehensive assessment was completed

Measure 2

NAME:

Screening for physical symptoms

Definition:

Percentage of seriously ill patients receiving palliative care in an acute hospital setting >1 day or patients enrolled in a hospice >7 days who had a screening for physical symptoms (pain, dyspnea, nausea, and constipation) completed

Measure 3

NAME:

Pain treatment (any)

Definition:

Percentage of seriously ill patients receiving specialty palliative care in an acute hospital setting >1 day or patients enrolled in a hospice >7 days who screened positive for moderate to severe pain on admission, and the percent receiving medication or nonmedication, within 24 hours of screening

Measure 4

NAME:

Dyspnea Screening and Management

Definition:

Percentage of patients with advanced chronic or serious life-threatening illnesses that are screened for dyspnea, for those who are diagnosed with moderate or severe dyspnea, a documented plan of care to manage dyspnea exists

Measure 5

NAME:

Discussion of emotional or psychological needs

Definition:

Percentage of seriously ill patients receiving specialty palliative care in an acute hospital setting >1 day or patients enrolled in a hospice >7 days with chart documentation of a discussion regarding emotional or psychological needs

Measure 6

NAME:

Discussion of spiritual/religious concerns

Definition:

Percentage of hospice patients with documentation in the clinical record of a discussion of spiritual and religious concerns or documentation that the patient or caregiver did not want to discuss these issues

Measure 7

NAME:

Documentation of surrogate

Definition:

Percentage of seriously ill patients receiving specialty palliative care in an acute hospital setting >1 day or enrolled in a hospice >7 days with the name and contact information for the patient’s surrogate decision-maker in the chart or documentation that there is no surrogate

Measure 8

NAME:

Treatment preferences

Definition:

Percentage of seriously ill patients receiving specialty palliative care in an acute care hospital setting >1 day or enrolled in a hospice for >7 days with chart documentation of preferences for life-sustaining treatments

Measure 9

NAME:

Care consistency with documented care preferences

Definition:

If a vulnerable elder has documented treatment preferences to withhold or withdraw life-sustaining treatment (e.g., a do-not-resuscitate order, no tube-feeding, no hospital transfer), then these treatment preferences should be followed

Measure 10

NAME:

Global measure

Definition:

Patient and/or family assessments of the quality of care is a key part of measuring quality for any setting caring for palliative or hospice patients