Outcome Measurement in Palliative Care The Essentials Bausewein C, Daveson B, Benalia H, Simon ST, Higginson IJ

Acknowledgements: The contributionof all PRISMA members in relation to thedevelopment of this work is gratefullyacknowledged: Gwenda Albers, BarbaraAntunes,Ana Barros Pinto, DorotheeBechinger-English, Emma Bennett, LucyBradley, Lucas Ceulemans, Luc Deliens,Noël Derycke, Martine de Vlieger, LetDillen, Julia Downing, Michael Echteld,Natalie Evans, Dagny Faksvåg Haugen,Nancy Gikaara, Barbara Gomes, MarjoleinGysels, Sue Hall, Richard Harding, SteinKaasa, Jonathan Koffman, Pedro LopesFerreira,Arantza Meñaca, Johan Menten,Natalia Monteiro Calanzani, Fliss Murtagh,Bregje Onwuteaka-Philipsen, RoelinePasman, Francesca Pettenati, Robert Pool,Richard A. Powell, Miel Ribbe, KatrinSigurdardottir, Franco Toscani, Bart Vanden Eynden, Paul Vanden Berghe,Trudievan Iersel.PRISMA also thanks Barbara Antunes, SteveAshford, Sabrina Bajwah, Elmien Brink,Barbara Gomes, Paula Lobo, Caty Pannell,Caroline Rumble, Mel Rumble, and Vicky

Robinson for their comments on an earlierversion of this guidance.The development of this guidance was ledby the German Association for PalliativeMedicine (Deutsche Gesellschaft fürPalliativmedizin e.V., DGP), incollaboration with members of theDepartment of Palliative Care, Policy andRehabilitation, King’s College London.Thecontribution of all members of workpackage 4 is also kindly acknowledged:Luc Deliens, Julia Downing, Pedro LopesFerreira, Dagny Faksvåg Haugen, SteinKaasa, Roeline Pasman, Franco Toscaniand Martine de Vlieger.Disclaimer: This guidance only reflectsthe authors' views.The EuropeanCommission is not liable for any use thatmay be made of the informationcontained herein.Contact address: Dr Claudia Bausewein,Cicely Saunders Institute, Department ofPalliative Care, Policy and Rehabilitation,King’s College London, London SE5 9PJ,e-mail: Claudia.Bausewein@kcl.ac.uk

PRISMAReflecting the Positive DiveRsities of EuropeanPrIorities for ReSearch and Measurement in

End-of-Life CAre

PRISMA is funded by the European Commission’s Seventh Framework Programme(contract number: Health-F2-2008-201655). Its overall aim is to co-ordinate high-qualityinternational research into end-of-life cancer care. PRISMA aims to provide evidence andguidance on best practice to ensure that research can measure and improve outcomes forpatients and families. PRISMA activities aim to reflect the preferences and culturaldiversities of citizens, and the clinical priorities of clinicians, and appropriately measuremultidimensional outcomes across settings where end-of-life care is delivered.

Principal Investigator: Richard HardingScientific Director: Irene J Higginson

www.prismafp7.eu

Under the Auspices of the EuropeanAssociation for Palliative Care,

EAPC Onluswww.eapcnet.eu

Department of Palliative Care,Policy and Rehabilitation

Deutsche Gesellschaft fürPalliativmedizin e.V.

WHO Collaborating Centre forPalliative Care and Older People

Foreword iiiIntroduction 1

CHAPTER 1 Outcome measurement andpalliative care 2What outcome measurement is 2Why it is important to measure outcomes 2How outcomes are measured 3How outcome measures are used inpalliative care 3Practical and ethical challenges of out-come measurement in palliative care 4Key points 4Further reading 4References 4

CHAPTER 4 Scores of outcome measures and their analysis 17Assessment and screening: individualpatient scores 17Monitoring change: patient scores over time 19Analysing data from several patients 20Data analysis: common descriptivestatistics 20Key points 23Further reading 23References 23

CHAPTER 2 Choosing and using outcomemeasures 5Choosing an outcome measure 5Types of PROMs in palliative care 6Generic versus specific 6Single items versus multiple items 6Domains and dimensions 7The patient 7Carers and family 9Quality of care 9Disease group and conditions 9Completing the outcome measure 9How often to use an outcome measure 10What to do with the data 10Steps to take before starting outcomemeasurement 10Key points 10Further reading 11References 11

CHAPTER 5 Quality improvement andorganisational change 25Change in organisations: types of change and enabling change 25Understanding perspectives withinorganisational contexts 28Determining what is needed and bywhom: agreeing a data collection plan 28Equipping organisations for outcomemeasurement success 30Making a real difference through audit and achieving successful outcomes 30Benchmarking 33Key points 34Further reading 34References 34

CHAPTER 3 What makes a good measure 12Validity 12Face and content validity 12Criterion and construct validity 12Reliability 13Inter-rater reliability 13Test-retest reliability 13Internal consistency 13Appropriateness and acceptability 13Responsiveness to change 14Interpretability 14Translation into other languages 14Key points 15Further reading 15References 15

CHAPTER 6 Where to find more information 36Websites 36Systematic reviews 37Books 39List of acronyms 40

Contents

Outcome measurement in palliativecare: how to assess what is in thepatient’s heart and mind

“I only want what is in your mind and inyour heart”, is what David Tasma, a patientdying from cancer, said to Dame CicelySaunders in 1949.This may also be used tosummarise what we need in order to treatpalliative care patients well.We needknowledge about symptom control, goodcommunication skills and empathy, and inexchange we need to know what thepatient is feeling (in their heart) andthinking (in their mind).

Surely this cannot be measured. One ofthe major differences between palliativecare and other areas of healthcare is theholistic approach it takes, includingpsychosocial and spiritual dimensions inaddition to the physical suffering.This alsopresents a major challenge: how to assessconcepts such as suffering, dignity andspirituality? These concepts are less welldefined and more difficult to measure than,for example, blood pressure or bodyweight. Measurement of physical symptomssuch as pain is well established, but palliativecare workers might challenge theseassessments with the argument thatfeedback from the patient on how they arefeeling today is more important than thescore on a symptom scale.This is reflectedby the lack of widely accepted standards foroutcome measurement, and the wide scopeof different instruments and scales that havebeen summarised in several recent reviews.

However, if we want palliative care tobecome part of the regular healthcaresystem so that it can be easily accessed byevery patient that needs it, we have tocomply with the rules.We have to provethe quality of the care that we deliver,

account for the resources that areallocated and verify that patients arereceiving the best possible care in relationto these resources.

This means that palliative care serviceshave to measure the outcome of theirinterventions, and be prepared to comparethem to other models of care or to othertypes of service delivery. Outcomemeasurement of palliative care will becomeeven more important as new players enterthe game. Specialists such as geriatriciansor neurologists are treating other patientgroups, and a public health approach forpalliative care has been recommended inaddition to specialist palliative care,exposing, for example, general practitionersto palliative care issues. Recently, aworkgroup from Boston has demonstratedthe benefit of early access to palliativecare, which improved not only the qualityof life and reduced the costs of treatment,but also increased survival times.This hasreinforced the need to start palliative careearly, and not restrict it to end-of-life care.

All this adds to the need to provideexpert guidance on symptom assessmentand measurement in palliative care.Healthcare professionals starting to workin palliative care have to be trained onwhat measurement instruments to use,how to use them and what to do withthe results.

This booklet provides the practicalguidance that is needed for patient-reported outcome measurement inpalliative care. It is based on theexperiences of the PRISMA project, fundedby the European Commission’s SeventhFramework Programme.The booklet doesnot focus on specific instruments, butrather explains the background and themeaning of outcome measurement, and

then proceeds with step-by-step trainingon how to do it.The booklet is suitablefor training specialist and non-specialiststaff in palliative care outcomemeasurement. It can be used as basictraining material for a researchcollaborative to agree on a commonunderstanding, as well as for new modelservices introducing palliative care inunusual settings or for new patient groups.

The authors are to be highlycommended for their contribution to theimprovement of palliative care. Followingthe guidance provided by this booklet, thenext step should be the development of aframework for the selection of outcomemeasurement instruments according to thesetting and dimensions that have to becovered. For this purpose, a Task Force ofthe European Association for Palliative Care(EAPC) has recently been established, ledby the authors of this booklet. I hope thatthe basis provided here, and the results thatcan be expected from the EAPC TaskForce, will lead to the continuousdevelopment and improvement of palliativecare outcome measurement and thereby,finally, to an on-going improvement ofpatient care across Europe.

Lukas RadbruchPresident of the European Association for Palliative Care (EAPC)

3iii4

Foreword

314

More and more people are livingwith a chronic disease near the endof their life, in Europe as well as therest of the world.

Palliative care needs are thereforeincreasing, and they are also becomingmore complex because of the range ofillnesses patients are suffering from.Furthermore, the integration of palliativecare within the healthcare system, andacross countries, varies greatly, which addsto the challenge of providing high-qualitypalliative care.

Outcome measurement has a majorrole to play in improving the quality,efficiency and availability of palliativecare. Measuring changes in a patient’shealth over time, and finding out thereasons for those changes, can helpservice providers focus on learning andimproving the quality of services.Outcome measures, specifically Patient-Reported Outcome Measures (PROMs),are tools that can effectively be used inpalliative care to assess and monitorcare, either for individual patients, oracross populations. PROMs put thepatient at the centre of care and focuson what matters to them.

There is an extensive range ofdifferent types of outcome measures for

use in clinical care, audit and research, andit can be difficult to know which tochoose and how to effectively use them.This guidance gives palliative careclinicians information about whatoutcome measurement is and how it canbe used to improve care plus guidance onchoosing and using outcome measures.This booklet will be useful for anyoneworking with those requiring palliativecare, including nurses, doctors,psychologists, social workers, thoseproviding spiritual care and therapists.

The PRISMA project, funded withinthe European Commission’s SeventhFramework Programme, aims to informbest practice and harmonise research andoutcome measurement in end-of-life careacross Europe.As part of PRISMA, anonline survey on the use and experiencesof professionals with outcomemeasurement in palliative care, and anexpert workshop about outcomemeasurement in palliative care, wereconducted. Both emphasised the need formore guidance and training on outcomemeasurement. In response to that, thisguidance was developed to informpalliative care clinicians involved in patientcare, audit or research, who want toknow more about the practice of outcomemeasurement.

Introduction

What outcome measurement isOutcome measurement is a way ofmeasuring changes in a patient’s healthover time.An outcome can be describedas “the change in a patient’s current andfuture health status that can be attributedto preceding healthcare”.1 Outcomemeasurement involves the use of ameasure to establish a patient’s baselinehealth status, and then evaluating changesover time against that baseline. Outcomemeasures help to record these changes.By using these measures it is also possibleto measure the structure, process andoutput of care. However, outcome is whatdirectly affects the patient and their family(Figure 1.1), and this is what we focus onin this guidance.

Why it is important to measureoutcomesOutcome measures are widely used inhealth research to describe patientpopulations or to assess the effectivenessof interventions, but they are not, as yet,always incorporated into routine clinicalpractice.3 However, with the increasingfocus on patient autonomy, equitableservice delivery and transparentinformation compelling service providers,healthcare commissioners and funders todemonstrate effectiveness and value formoney, outcome measurement isbecoming a more important procedure toconsider. In addition, funding fromgovernments or commissioners isbecoming more often a condition for on-

going provision of outcomes data.4 The useof outcome measurement is thereforebecoming increasingly important inhealthcare, both in general and in palliativecare in particular.

Accountability to patients, fundersand governments is another driving forcein terms of outcome measurementactivity in healthcare, with serviceproviders being required to demonstrateefficiency and high-quality care. Palliativecare services need to have a genuineinterest in using outcome measures inorder to enhance quality assurance,maintain on-going quality improvementand strengthen the learning capacity ofthe organisation. Service providers,commissioners and funders also have an

324

Chapter 1Outcome measurement and palliative care

INPUT/STRUCTURE PROCESS OUTPUT OUTCOME

Which resources are required or used?

Staff, equipment, consumables (i.e.,syringe drivers, drugs)

How are the resources used?

Prescription of drugs, use of syringe drivers, staff visits, staff meetings/clinics,information etc.

Productivity or throughput

Discharge rate, day hospice attendance, number of drugs/opioids, number of consultations, com-pleted care plans etc.

Change in health status or quality oflife attributable to health care

Change in pain levels, improved quality of life, decreased anxiety

Figure 1.1 A palliative care example regarding the sequence involved in outcome measurement (adapted from Higginson and Harding 20072)

334

Outcome is whatdirectly affectsthe patient andis what we focuson in thisguidance

ethical responsibility to ensure thatvulnerable patients and their families,such as those requiring palliative care,receive services that effectivelycontribute to their well-being and qualityof life, despite advanced disease.

How outcomes are measured As outcomes in healthcare are related topatients’ experiences, patients are themain source of information concerningchanges in their health status, quality of lifeor symptoms.These outcomes can bemeasured using a variety of tools, forexample Patient-Reported OutcomeMeasures (PROMs or PROs), which arequestionnaires or instruments used tocapture these changes.

How outcome measures are used inpalliative care Outcome measures are used for assessing,measuring, evaluating or monitoring arange of different aspects of healthcare.There are three main purposes behind theuse of outcome measures: clinical care,audit and research.The type of outcomemeasure used will be dictated by thepurpose.

In clinical care, outcome measures canbe used to:

4 establish patients’ baselines (forexample, baseline pain level,existential distress or spirituality);

4 assess patients’ symptoms, as well asfamilies’ and patients’needs/problems;

4 monitor changes in patients’ healthstatus or quality of life;

4 facilitate communication withpatients/families and the healthcareteam;

4 aid clinical decision making;4 evaluate the effect of interventions,

care or services.

For clinical purposes, outcome dataare usually recorded in the patient’s clinicalrecord and shared amongst team membersto promote cohesive, co-ordinatedpatient-centred care (see Chapter 4).Thistype of data may also feature in case studyreports and publications, or within reportsto funders in relation to funding care forindividual patients.

Audit is a systematic approach toevaluating quality or performance in

healthcare. It involves setting orestablishing standards, monitoring orobserving performance or practice, andthen evaluating what was done inrelation to the standard identified.Aclinical audit focuses specifically onclinical care, but audit projects can alsobe broader in their focus, for example,the practices within organisations ordepartments may be examined in anorganisational audit.

For the purposes of audit, outcomemeasures can be used to:

4 establish standards of practice inparticular departments (for example,in medicine, nursing, social work,physiotherapy or music therapydepartments), or within palliativecare teams or organisations;

4 assess the care given againstestablished standards, with the viewto improving standards;

4 determine uptake of service;4 benchmark or compare standards of

practice in one organisation withanother organisation.

The main aim of research is tounderstand the patients’ situations betterand improve their care.To demonstrateresults, measurement of outcomes iscrucial.Therefore, the use of outcomemeasures and, increasingly, patient-reported outcomes is an intrinsic part ofresearch and researchers are often muchmore familiar than clinicians with the useof outcome measures.

For the purposes of research,outcome measures can be used to:

4 screen whether patients meet inclusioncriteria for a study;

4 assess patients’ functional status;4 measure or describe patients’

symptoms, quality of life and quality ofcare;

4 monitor changes in patients’ healthstatus or quality of life;

4 evaluate the effect of interventions,care or services.

Outcome measurement data for researchare usually presented in a summed oraggregated manner. Patients andorganisations are not usually identified, andfindings may be published for widedissemination.

Practical and ethical challenges ofoutcome measurement in palliativecarePatients in palliative care have uniqueneeds.The illness trajectory in palliativecare, for example, poses a challenge tooutcome measurement as patients’ healthwill deteriorate and symptoms willprobably worsen.This deteriorationmakes the detection of health-relatedoutcomes challenging.At the very least,deterioration in physical health isexpected, and changes in cognitiveabilities are also likely to occur closer tothe time of death.These challengesinfluence what type of outcome measurecan be used, who they can be used withand when they can be used. For example,PROMs are impossible to use closer tothe time of death once the patientbecomes unconscious.

Ethical considerations also play animportant role in enabling outcomemeasurement in palliative care. Forexample, is outcome measurement datasufficient for determining what care isneeded for patients at the end of life?Should outcome measures be used forsymptoms that are difficult to report?How often should symptoms bemeasured when patients might ratherspend time with loved ones before theirdeath? If there is no valid outcomemeasure to use, should we just notmeasure at all?

To aid the development andimplementation of outcome measurementin palliative care, an international expertmeeting was convened in 2010 byPRISMA5 (see Introduction).The groupagreed on the following:

4 Standardisation and agreement of acore set of tools in palliative careneeds to be based on rigorousscientific criteria, rather than byconsensus only. Standardisationneeds to be balanced with diversityand flexibility.

4 Training, support and resources foroutcome measurement for clinicalcare, audit and research arerequired for palliative care cliniciansand researchers.These could beweb-based, printed materials orface-to-face training sessions.Thetraining needs to include how to

choose an outcome measure, alongwith information and practicalstrategies on implementing outcomemeasures in organisations, and howto analyse and interpret findings.

4 When translating outcomemeasures for use in other countries,it should not simply be a literaltranslation of the tool. Instead itshould encompass the process oflearning what things mean: theheuristic process of meaning.Themeasures need to have the samemeaning and impact in differentcultures and languages. Culturalcompetencies and local differencesmust be taken into account whentranslating outcome measures foruse in palliative care.

4 A multi-professional approach isrequired in outcome measurementin palliative care, and the differentcompetencies of the professionalgroups involved in palliative caremust be acknowledged. Nursesshould be involved in thedevelopment of outcome measuresin order to aid successfulimplementation of the measures inclinical care.

KEY POINTS

4 Outcome measurement is a way ofmeasuring changes in a patient’shealth (which can be attributed topreceding healthcare) over time.

4 It can be used to improve thequality of healthcare services.

4 Outcome measurement can be usedfor clinical care, audit and researchpurposes.

4 There is an increasing need forrobust outcome measurement inthe field of palliative care, but thisposes particular challenges andrequires special consideration withregard to patients’ situations at theend of life.

Further readingPalliative care4 Davies E, Higginson IJ, eds. Palliative care:

the solid facts. Copenhagen,WHORegional Office for Europe, 2004(www.euro.who.int/InformationSources/Publications/Catalogue/20050118_2,accessed 1 December 2010).

4 Davies E, Higginson IJ, eds. Better palliativecare for older people. Copenhagen,WHORegional Office for Europe, 2004(www.euro.who.int/InformationSources/Publications/Catalogue/20050118_1,accessed 1 December 2010)

4 Harding R, Dinat N, Sebuyira LM.Measuring and improving palliative care inSouth Africa: multiprofessional clinicalperspectives on development andapplication of appropriate outcome tools.Prog. Palliat. Care. 2007;15:55-9.

Use of outcome measures4 Dawson J, Doll H, Fitzpatrick R, Jenkinson

C, Carr A.The routine use of patientreported outcome measures in healthcaresettings. Br. Med. J. 2010;340:c186.

References1 Donabedian A. Explorations in quality

assessment and monitoring.Ann Arbor,Mich.: Health Administration Press; 1980.

2 Higginson IJ, Harding R. Outcomemeasurement. In:Addington-Hall JM,Bruera E, Higginson IJ, Payne S, editors.Research methods in palliative care.Oxford, New York: Oxford UniversityPress; 2007. p. 99-110.

3 Gruenewald DA, Higginson IJ,Vivat B,Edmonds P, Burman RE. Quality of lifemeasures for the palliative care of peopleseverely affected by multiple sclerosis: asystematic review. Mult. Scler.2004;10(6):690-725.

4 Department of Health. Equity andexcellence: liberating the NHS.TheStationery Office; 2010.

5 Harding R, Higginson IJ. PRISMA: A pan-European co-ordinating action to advancethe science in end-of-life cancer care. Eur.J. Cancer. 2010;46(9):1493-501.

344

Choosing an outcome measureA huge variety of outcome measures existin palliative care.They differ in the domainsand dimensions they measure, and in theirlength, accessibility and cost.This diversitymakes the selection of a measurechallenging. Ideally, original literature (forexample, publications about thedevelopment of the measure) about aspecific outcome measure and studies onthe validation of that measure should bereviewed to inform one’s choice. However,as this is often not practical for clinicians,review articles (which systematicallyidentify, appraise and synthesise theevidence) about outcome measures canhelp to get an overview of the necessaryinformation.

When deciding which measure to use,the aims and the reason for using anoutcome measure should be taken intoconsideration.This includes consideringthe context of the outcomemeasurement, that is, whether data will beroutinely collected in clinical care, foraudit purposes or within a research study(see Chapter 1). For example, for clinicalcare, short, widely accepted measuresmight be more practical; whereas inresearch, a series of measures may beneeded. Relevant questions to ask whenchoosing an outcome measure are shownin Figure 2.1.

354

Chapter 2 Choosing and using outcome measures

What is theaim of use?

What types of PROMs are

available?

What will happen to the data?

What are the domains and dimensions?

What is the disease group/

condition?

Who will fillin the outcome

measure?

How often willthe outcomemeasure be

used?

Figure 2.1 What to consider when choosing an outcome measure

Types of PROMs in palliative careGeneric versus specific

Generic or specific outcome measures canbe used within palliative care. Genericmeasures are multidimensional measuresthat include physical, psychological andsocial health components.They are able tobe used on a large range of health andquality of life concepts, and in varioushealth conditions, populations andinterventions. Specific measures arespecially designed for particular domains,health conditions, signs and symptoms,body parts or populations (see Table 2.1).

Single item versus multiple items

Outcome measures that use single-itemscales measure a concept of interest using asingle question. For example, the severity ofa symptom such as pain can be measuredby asking a patient to rate the severity on ascale from 0 (‘no pain’) to 10 (‘worst pain Ican imagine’).The endpoints, which are alsocalled ‘anchors’, need to be clearly definedbut can vary in the wording. If just twoendpoints are provided on a continuousline (normally 10 cm), this is called a visualanalogue scale (VAS).When using a VAS, thepatient marks a point between the twoendpoints where the patient rates the pain(see Example 2.1).The actual value is then

364

Table 2.1 Type of outcome measures

Generic measures Specific measures

Advantages 4applicable across a broad population for 4specifically developed to measure outcomes in comparing different conditions; palliative care;

4large range of domains; 4for use in specific conditions or domains (for 4can compare palliative treatment with example, symptoms, function, palliative care

other treatments. needs);4more responsive to clinically meaningful changes.

Disadvantages 4often lack responsiveness to change; 4do not allow direct comparison with data4not validated in palliative care; collected from another patient group.4do not allow specific problems to be identified.

Examples 4SF-361 4Palliative care Outcome Scale (POS)5

4EuroQol (EQ-5D)2 4Hospital Anxiety and Depression Scale (HADS)6

4General Health Questionnaire (GHQ)3 4Edmonton Symptom Assessment Scale (ESAS)7

4Sickness Impact Profile4

Example 2.1 - Visual analogue scale (VAS)

Example 2.2 Numerical rating scale (NRS)

No pain at all

0 10

Worst pain I can imagine

No pain at all

0 1 2 3 4 5 6 7 8 9 10

Worst pain I can imagine

measured using a ruler. If the line hasnumbers in between (for example, from 0to 10, as shown in Example 2.2), this iscalled a numerical rating scale (NRS).

There can sometimes be problemsusing a VAS with palliative care patientswho are very ill, as they need to be able tosee the scale. Patients may need to putglasses on and sit up to fill in the VAS.Using numerical scales can mean morepatients can take part as they do not needto see the scale in order to answer thequestion.

Outcome measures that use multiple-item scales combine various questions on aspecific area.They are more complex todevelop, and can be more burdensome forpatients but they are often more effectivein describing a multidimensionalphenomenon.

Domains and dimensionsMost outcome measures cover variousdomains and dimensions. Domainsdescribe the scope of an area of interest;dimensions relate to measurablequantities or particular aspects of aproblem. Outcome measures in palliativecare can cover several domains, forexample, the patient, family and carers, orquality of care, as well as physical,

psychological, social or spiritualdimensions (see Figure 2.2).The choice ofa measure depends on which outcomeneeds to be measured, for example,individual symptoms, palliative care needsor quality of life.

The patientA patient’s experience can be related tophysical (for example, symptoms andfunctional status), psychological (forexample, cognition and emotions), socialand cultural (for example, family andfriends, organisational and financial), andspiritual (for example, beliefs, meaning andreligion) domains, which are allinterlinked.As palliative care aims toprovide holistic care for patients andfamilies, an outcome measure shouldideally cover several of these domains, aswell as aspects of care. Some examplesfor this are given in Table 2.2.

A large number of outcome measureshave been developed to measure specificphysical dimensions, for example,symptoms such as pain, breathlessness orfatigue.These measures give a more in-depth view of the problem and, as they arevery specific, it is likely that they are moreoften used in research rather than clinicalcare. Psychological symptoms, such as

374

Outcomemeasures inpalliative carecan coverseveral domains

Example 2.3 Domains and dimensions of outcome measures in palliative care (adapted from Mularski et al 20078)

Patient

Carer/family

Qualityof care

PsychologicalSocial

Spiritual

Physical

Cultural

Carer well-being

Grief and bereavement

Carer burden

Advance care planning

Continuity of care

Satisfaction and quality of care

384

Table 2.2 Examples of multidimensional outcome measures in palliative care

Outcome measure Number of items Completion time Additional comments

Palliative care 10 items on physical symptoms, mean time 6.9 min scores from 0 (‘no effect’) to 4Outcome Scale emotional, psychological and (patients) and 5.7 min (‘overwhelming’): patient, staff (POS)5 spiritual needs, provision of (staff); and carer version; widely used

information and support repeated assessments of palliative care measure 1 open question on main patients and staff mean freely available after registrationproblems time < 4 min5

POS-S Symptom list 10 symptoms few minutes scores from 0 (‘no effect’) to 4 2 questions about the symptom (‘overwhelming’); additionalthat affected the patient the most symptom versions availableand that has improved the most for other conditions (POS-S MS,

POS-S renal); freely available after registration

Distress Thermometer9 overall distress score median length of time distress score 0-10;20 symptoms, 5 items on practical 5 min, with 75% taking other items yes/noproblems, 4 on family problems, no more than 10 min10

5 on emotional problems,2 on spiritual concerns

Edmonton Symptom 9 symptoms and 1 “other approximately 5 min11 each symptom with NRS 0-10Assessment Scale problem” developed to measure the most(ESAS)6 commonly experienced symptoms in

cancer patients; freely available

Memorial Symptom 28 physical and 4 psychological 20-60 min,13 measuring presence, frequency,Assessment Scale symptoms short form < 5 min severity and distress of symptoms;(MSAS)12 short form version available (MSAS-SF):

only presence and distress of symptoms;developed for cancer patients but also used in other conditions

Hospital Anxiety 14 items (7 depression, 2-6 min14 developed to assess depression and Depression 7 anxiety) and anxiety for people with physical Scale (HADS)7 illness; not freely available

EORTC 5 functional scales (physical, first assessment 12 min not freely available, widely usedQLQ-C3015 role, emotional, social, and (SD 7.5 min), second in cancer research; modular

cognitive), 3 symptom scales assesment 11 min supplement available for a range(fatigue, nausea/vomiting and (SD 6.5 min)15 of malignancies(lung, breast,pain), a global health status/ gastric, brain etc.) QoL scale and six single items (dyspnoea, insomnia, appetite loss, constipation, diarrhoea,and financial difficulties)

EORTC pain, physical function (3 items), < 20 min17 not freely available, shortenedQLQ-C15-PAL16 emotional function (2 items), version of the EORTC QLQ-C30

fatigue (2 items), QoL for palliative care patients(1 item), symptoms (6 items)

depression or anxiety, are either measuredusing separate measures or are included inthe symptom measures.

Besides symptoms, most patientsexperience gradual, abrupt or intermittentfunctional decline during the course oftheir disease and towards the end of life.Functional status refers to the patients’mobility and the ability to perform certainroutine tasks.These are also calledactivities of daily living (ADL), for examplebathing, dressing and preparing meals.Thewidely used Karnofsky Performance Status(KPS) describes 10 levels of function.18 Amodified version for palliative care is theAustralia-modified Karnofsky PerformanceStatus (AKPS).The AKPS is moreappropriate for clinical settings andincludes multiple care settings such aspalliative care.19 Another example is thePalliative Performance Scale (PPS) whichuses five observer-rated domains that arecorrelated to the KPS.20

Advanced disease also has an impacton a patient’s personal life. Social needs andcultural aspects should be assessed as theywill influence the experience of symptomsand can cause psychological distress.Practical needs, such as organisation ofcare or financial constraints, will add to apatient’s burden, as can family dynamics andcommunication problems. Only a fewoutcome measures cover practical andsocial needs, for example the POS has onequestion on practical matters, and theDistress Thermometer has five items onpractical problems.

The concept of spirituality is difficult todefine. Some authors refer to it asincluding meaning of life and death,transcendence and forgiveness, as well as apatient’s interpretation of their illness.21,22

The measurement of spirituality is multi-faceted. For example, some outcomemeasures focus entirely on spirituality,whereas other tools include spirituality-related items as part of assessing quality oflife and religiosity. Spirituality is often notconsidered when assessing a patient’ssituation, nor is it often covered in normalconversations, and it can therefore bemissed. Using an outcome measure thatincludes at least one or two items relatingto the spiritual dimension can help identifyareas for further investigation and support.For example, the POS includes a questionabout ‘feeling good about oneself’ or‘whether life is worthwhile’.The Qual-E

includes three (out of 31) items onspiritual aspects of quality of life,23 and theMcGill Quality of Life Questionnaire(MQOL) includes four (out of 17) items onthe meaning and purpose of life, life worth,feelings about oneself, and value of life.24

Carers and familyCarers and families often experienceburden and have their own personalneeds (for example, social, emotional andfinancial needs). Outcome measures forpalliative care often focus on the degreeof burden and strain experienced bycarers, especially their physical and mentalhealth, finances, and social life. Someoutcome measures specifically examinethe needs and experiences directly relatedto carer tasks, such as giving medication,providing physical care, or managing time.Other instruments are designed for carersof patients with specific diseases, such asdementia or stroke.The Zarit BurdenInventory (ZBI) was originally designed forcarers of dementia patients, but has nowalso been validated and used in relation toother conditions and in palliative care.9

Although originally developed with 22items, the 12-item and six-item versionsare widely used, and a short version withfour items can be used as a screeningtool.19 The POS includes one question onfamily anxiety,5 and the CAMPAS-R hastwo items on carer anxiety anddepression.20

Quality of carePatients’ and carers’ perceptions of, andpreferences for care (place; information)and place of death are important domainsin outcome measurement.This includestheir perceptions of the amount ofinformation provided, the level ofcommunication with themselves and theirfamilies, and their satisfaction with theoverall care.Assessing the quality of carecan highlight areas that need to beimproved. Ideally, a combination ofmeasures should be used to measurequality of care, including process measures(that measure the performance of aprocess), measures that look at thestructure that supports the care that isbeing delivered, and outcome measures.

Disease group and conditionsSome outcome measures have beendeveloped and validated for specific

conditions or disease groups, for example,cancer or respiratory disease. Strictlyspeaking, a measure should only be usedwith the patient group with which it hasbeen validated.As palliative care isprovided for people with a wide range ofconditions, outcome measures that havebeen validated across different conditionsare useful and important. Some measuressuch as the EORTC-QLQ C30 or FACIT(Functional Assessment of Chronic IllnessTherapy) have sub-sections or modulesthat are disease-specific, for example theyinclude specific symptom lists.27, 28

Networks such as interRAI(www.interrai.org), which draw togetherresearchers committed to improvinghealthcare for elderly, frail or disabledpeople, have agreed and tested a set oftools for use with specific patient ordisease groups (especially for cognitivelyimpaired patients or those with dementia).

Completing the outcome measureThe gold standard for reportingoutcomes is normally considered to beinformation collected directly from thepatient. PROMs can either be self-completed or facilitated by aninterviewer, personally, or over thetelephone. In a clinical setting, self-administration will be the most practicalway to get information from patients.However, this poses a challenge inpalliative care when patients aredeteriorating and may not be able toanswer questionnaires, especially towardsthe end of life.Although it is best forpatients to fill in the questionnaire ontheir own, in reality they are often helpedby professionals if they are too weak ortoo tired. Research has shown thatprofessionals tend to underestimaterather than overestimate patients’symptoms.29 Underestimation ofsymptoms has been particularly reportedfor drowsiness, shortness of breath, pain,fatigue and anorexia.29,30 It is thereforeimportant that professionals score onlywhat the patient reports, rather thanusing their own assumptions of what theythink the patient would score.

If patients are unable to answer thequestionnaire, proxies such as relatives orhealthcare providers can be used to fill inthe measures for the patients. However,there has been a debate about howaccurate these ratings are, and how much

394

they reflect patients’ views. Someoutcome measures include a special proxyor family version, for example the POS.Ideally, ratings from both patients andproxies should be collected andcompared.This allows for the adjusting ofproxy ratings if the patient is not able tocomplete the measure as their diseaseprogresses. Measures such as the POSallow for this.

How often to use an outcomemeasureThe frequency that an outcome measureshould be used depends on the aim of themeasurement, for example whether it is anassessment of change in symptoms, or anevaluation of an intervention, and what isbeing measured and how quickly it ischanging. Symptoms should be assessedfrequently, for example on a daily basis, toallow review of their management. Incontrast, quality of life, a more complexconstruct, might only change over a longerperiod of time, or in response to asignificant event (for example related todisease progression or a change in socialsupport network), and thereforemeasurement regularity should take thisinto account.

The time it takes to complete ameasure is also a consideration whendetermining how frequently to use anoutcome measure. If a measure is to beused regularly, it should be short and ableto be completed within minutes. Measuresthat take longer to complete, for example,half an hour or an hour, can be tiring forpatients and this may lead to patientburden and missing data. Longer measuresshould be used with greater intervalsbetween use. Longer measures are oftenused for research, rather than for routineclinical use.

It is important to establish an earlybaseline assessment with patients aschanges will occur earlier in their carerather than later31, and changes in patients’symptoms or palliative care needs will bemissed if there is no baseline assessment.Collecting data on an on-going basis can bechallenging as patients and staff may notrecognise why it is important. Repeateddata collection not only captures changes,but also helps to identify new problemsthat need timely interventions. Explaining tothe patient and staff members why thesame outcome measure needs to be filled

in repeatedly is therefore vital. Repeateddata collection will become morechallenging as palliative care patientsdeteriorate over time. Using the data in thepresence of the patient and referring to theanswers they gave previously will increasecompliance as patients will feel that filling inthe outcome measure is not a waste oftime, but an investment in their own care.

Outcome measures often pre-define aperiod called a ‘measurement window’which the questions relate to, for example,the last week or the last three days.This isoccasionally changed by users. Strictlyspeaking such a change affects the validityof the measure, but trials have shown thatthis is more relevant when a measureassesses the presence/absence of asymptom, rather than the severity of thesymptom. Nevertheless, the practice ofchanging the measurement window whenusing tools should be approached withcaution, and ideally the measure should beused as it was designed to be used.

What to do with the dataIf data are collected for clinical purposes,the questionnaires need to go into theclinical notes, be brought to the attentionof clinicians, shared with patients, and usedto influence care and clinical decisionmaking. It has been shown that feedback ofPROMs results to clinicians has greaterimpact on discussion and detection ofpatients’ problems, rather than onsubsequent management of theseproblems.32 Teams should develop aroutine for how patient reports are usedin ward rounds, team meetings, or otherconsultations. In some organisations,patients enter their answers usingcomputer touchscreens and results aredirectly presented to clinicians in theconsultation. If the data are used foradditional purposes, such as audit orresearch, they should go to a central placewhere they can be entered into thecomputer or used otherwise. How resultsare fed back to the clinical team to informtheir work should be considered.

Steps to take before startingoutcome measurement

4 Familiarise yourself with therequirements of the chosenoutcome measure. In particular,determine whether the dimensions

and the measurement window fityour needs.

4 Complete any available training andread the guidance regarding the useof the outcome measure.

4 Determine beforehand why you areusing the measure. Is it going to beused for assessment or screeningpurposes or both?

4 Determine when the measure willbe used. Will it be used at certainpoints to monitor treatment?

4 Pilot the measure with a fewpatients.This can help determinewhether or not the measure issuitable for the intended purpose.

4 Involve patients in the process:explain why PROMs are helpful, howthey are used and how they canmake a difference to their care.

4 Determine how the data will bestored and reported upon. Forexample, this might involve filing thecompleted outcome measure in thepatient’s records, and reporting thepatient’s aggregated score to themultidisciplinary team.

KEY POINTS

4 Many types of outcome measures areavailable for use in palliative careincluding PROMs, generic and specificmeasures, and single-item andmultidimensional measures.

4 PROMs can and should be used forclinical, audit and research purposes.They are helpful with assessment,monitoring and reporting.

4 The number of existing outcomemeasures in palliative care makes thechoice of the right measure difficult,but there are certain questions toask that can help you to select themost appropriate measure.

4 The measures that are available aredesigned to be used in certain ways;ideally clinicians should stick to therecommendations for use.Adaptations are possible, butadapting their use in practice shouldbe done cautiously, as this mayinvalidate their use.

4 Use of established outcomemeasures is preferable to developingnew measures.

3104

Further reading4 Cherny NI.The problem of suffering and

the principles of assessment in palliativemedicine. In: Hanks GWC, Cherny NI,Christakis N, Fallon M, Kaasa S, PortenoyR, editors. Oxford Textbook of PalliativeMedicine. 4th ed. Oxford; New York:Oxford University Press; 2009. p. 58-80.

4 Luckett T, King M. Choosing patientreported outcome measures for cancerclinical research – Practical principles andan algorithm to assist non-specialistresearchers. Eur. J. Cancer.2010;46(18):3149-57.

References1 Ware Jr J, Sherbourne C.The MOS 36-item

short-form health survey (SF-36): I.Conceptual framework and item selection.Med. Care. 1992;30(6):473-83.

2 Group E. EuroQol - a new facility for themeasurement of health-related quality oflife. Health Policy. 1990;16:199-208.

3 Goldberg D, Hillier V. A scaled version ofthe General Health Questionnaire. Psychol.Med. 1979;9(01):139-45.

4 Bergner M, Bobbitt R, Carter W, Gilson B.The Sickness Impact Profile: developmentand final revision of a health status measure.Med. Care. 1981;19(8):787-805.

5 Hearn J, Higginson IJ. Development andvalidation of a core outcome measure forpalliative care: the palliative care outcomescale. Palliative Care Core Audit ProjectAdvisory Group. Qual. Health Care.1999;8(4):219-27.

6 Bruera E, Kuehn N, Miller M, Selmser P,Macmillan K.The Edmonton SymptomAssessment System (ESAS): a simplemethod for the assessment of palliative carepatients. J. Palliat. Care. 1991;7(2):6-9.

7 Zigmond AS, Snaith RP. The HospitalAnxiety and Depression Scale.ActaPsychiatr. Scand. 1983;67(6):361-70.

8 Mularski RA, Dy SM, Shugarman LR,Wilkinson AM, Lynn J, Shekelle PG, et al.ASystematic Review of Measures of End-of-Life Care and Its Outcomes. Health Serv.Res. 2007;42(5):1848-70.

9 Roth A, Kornblith A, Batel Copel L, PeabodyE, Scher H, Holland J. Rapid screening forpsychologic distress in men with prostatecarcinoma. Cancer. 1998;82(10):1904-8.

10 Gessler S, Low J, Daniells E,Williams R,Brough V, Tookman A, et al. Screening fordistress in cancer patients: is the distressthermometer a valid measure in the UKand does it measure change over time? Aprospective validation study.Psychooncology. 2008;17(6):538-47.

11 Chang VT, Hwang SS, Feuerman M.Validationof the Edmonton Symptom Assessment

Scale. Cancer. 2000;88(9):2164-71.12 Portenoy RK,Thaler HT, Kornblith AB,

McCarthy Lepore J, Friedlander-Klar H,Kiyasu E, et al.The Memorial SymptomAssessment Scale: an instrument for theevaluation of symptom prevalence,characteristics and distress. Eur. J. Cancer.1994;30(9):1326-36.

13 Albers G, Echteld MA, de Vet HCW,Onwuteaka-Philipsen BD, van der LindenMHM, Deliens L. Evaluation of quality-of-lifemeasures for use in palliative care: asystematic review. Palliat. Med.2010;24(1):17-37.

14 Herrmann C. International experiences withthe Hospital Anxiety and Depression Scale -a review of validation data and clinicalresults. J. Psychosom. Res. 1997;42(1):17-41.

15 Aaronson N,Ahmedzai S, Bergman B,Bullinger M, Cull A, Duez N, et al.TheEuropean Organization for Research andTreatment of Cancer QLQ-C30: a quality-of-life instrument for use in internationalclinical trials in oncology. JNCI Journal ofthe National Cancer Institute.1993;85(5):365.

16 Groenvold M, Petersen MA,AaronsonNK,Arraras JI, Blazeby JM, Bottomley A, etal.The development of the EORTC QLQ-C15-PAL: a shortened questionnaire forcancer patients in palliative care. Eur. J.Cancer. 2006;42(1):55-64.

17 Suárez del Real Y,Allende Pérez S,AlférezMancera A, Rodríguez RB, Jiménez Toxtle S,Mohar A, et al.Validation of theMexican–Spanish version of the EORTCQLQ C15 PAL questionnaire for theevaluation of health related quality of life inpatients on palliative care. Psychooncology.2010 Jul 26 [Epub ahead of print].

18 Karnofsky DA, Abelmann WH, Craver LF,Burchenal JH.The use of the nitrogenmustards in the palliative treatment ofcarcinoma.With particular reference tobronchogenic carcinoma. Cancer.1948;1(4):634-56.

19 Abernethy A, Shelby-James T, Fazekas B,Woods D, Currow D.The Australia-modified Karnofsky Performance Status(AKPS) scale: a revised scale forcontemporary palliative care clinical practice[ISRCTN81117481]. BMC Palliat. Care.2005;4(1):7.

20 Anderson F, Downing G, Hill J, Casorso L,Lerch N. Palliative performance scale (PPS):a new tool. J. Palliat. Care. 1996;12(1):5-11.

21 Selman L, Harding R, Gysels M, Speck P,Higginson IJ.The Measurement ofSpirituality in Palliative Care and theContent of Tools Validated Cross-Culturally:A Systematic Review. J. Pain SymptomManage.; 2011 in Press.

22 Puchalski C, Ferrell B,Virani R, Otis-Green S, Baird P, Bull J, et al. Improvingthe quality of spiritual care as a dimensionof palliative care: the report of theconsensus conference. J. Palliat. Med.2009;12(10):885-904.

23 Steinhauser KE, Clipp EC, Bosworth HB,Mcneilly M, Christakis NA,Voils CI, et al.Measuring quality of life at the end of life:validation of the QUAL-E. Palliat. Support.Care. 2004;2(01):3-14.

24 Cohen S, Mount B, Strobel M, Bui F. TheMcGill Quality of Life Questionnaire: ameasure of quality of life appropriate forpeople with advanced disease.A preliminarystudy of validity and acceptability. Palliat.Med. 1995;9(3):207-19.

25 Higginson IJ, Gao W, Jackson D, Murray J,Harding R. Short-form Zarit CaregiverBurden Interviews were valid in advancedconditions. J. Clin. Epidemiol.2010;63(5):535-42.

26 Ewing G,Todd C, Rogers M, Barclay S,McCabe J, Martin A.Validation of a symptommeasure suitable for use among palliativecare patients in the community: CAMPAS-R.J. Pain Symptom Manage. 2004;27(4):287-99.

27 Aaronson N,Ahmedzai S, Bergman B,Bullinger M, Cull A, Duez N, et al.TheEuropean Organization for Research andTreatment of Cancer QLQ-C30: a quality-of-life instrument for use in internationalclinical trials in oncology. J. Natl. Cancer Inst.1993;85(5):365-76.

28 Cella D, Nowinski CJ. Measuring quality oflife in chronic illness:The functionalassessment of chronic illness therapymeasurement system. Arch. Phys. Med.Rehabil. 2002;83(12, Supplement 1):s10-s7.

29 Laugsand EA, Sprangers MAG, Bjordal K,Skorpen F, Kaasa S, Klepstad P. Health careproviders underestimate symptomintensities of cancer patients:A multicenterEuropean study. Health Qual. LifeOutcomes. 2010;8(1):104.

30 Nekolaichuk CL, Bruera E, Spachynski K,MacEachern T, Hanson J, Maguire TO.Acomparison of patient and proxy symptomassessments in advanced cancer patients.Palliat. Med. 1999 Jul;13(4):311-23.

31 Lambert MJ, Hawkins EJ. Measuringoutcome in professional practice:Considerations in selecting and using briefoutcome instruments. Prof. Psychol-Res. Pr.2004 Oct;35(5):492-9.

32 Greenhalgh J.The applications of PROs inclinical practice: what are they, do theywork, and why? Qual. Life Res.2009;18(1):115-23.

3114

How useful an outcome measure is can bedetermined by its psychometricproperties, and how well these relate toits aim.A measure is ‘good’ if it can beshown to have validity, reliability,appropriateness and acceptability, theability to be translated into differentlanguages, responsiveness to change, andinterpretability of results.

ValidityValidity is one of the most important aspectsof an outcome measure.1 It refers to what atool is measuring and whether it ismeasuring what it should be measuring.2 Themost important types of validity are face,content, criterion and construct validity.

Face and content validityFace and content validity are closely linkedconcepts that describe whether a measureis assessing the relevant aspects for thepurpose, and whether the domains coveredare appropriate, important and sufficient.1

The quality criteria for these two areas arenot standardised and assessment is basedon the subjective views of experts. Forpalliative care, the experts are patients,families and healthcare professionals.Therefore, their views on the content of ameasure are paramount.

Face and content validity are ofcentral importance to the choice ofPROMs for routine clinical practice, asthey highlight the extent to which themeasure captures the views of patientsand other key stakeholders.3

Using an outcome measure with high facevalidity can (adapted from Nevo 1985)4:4 increase the co-operation and

motivation of the patient, family orcarer during and after the measureadministration;

4 reduce dissatisfaction amongpatients with low scores;

4 increase the co-operation ofemployees, administrators,commissioners and policy makers toimplement or use the outcomemeasure.

Criterion and construct validityCriterion validity refers to whether themeasure correlates with anotherinstrument that measures similar aspects.Preferably, the other instrument is the‘gold standard’, meaning it has beenvalidated, and is widely used and acceptedin the field. For a new measure, thecorrelation with the gold standard isexpected to be between 0.4–0.8 for it tohave an acceptable criterion validity.2

If no other measure or gold standardexists for comparison, the measure mustbe linked to a theory or hypothesis inorder to show construct validity.Construct validity is the extent to which ameasurement corresponds to thetheoretical concepts or constructs that itwas designed to measure. If therelationship between the measure andtheory that it is testing cannot be shown,the problem can be either with themeasure or with the theory that was used.

3124

Chapter 3What makes a good measure

ReliabilityThe reliability of an outcome measurerefers to whether the measure producesthe same or similar results whenadministered in unchanged conditions.Reliability is important as it can reducemeasurement error or errors that arerelated to the process of measurement.Providing clear definitions for the scoresfrom an outcome measure helps to makeit more reliable. Fewer points on the scalealso improves reliability.

Inter-rater reliabilityInter-rater reliability assesses whethersimilar results are reached when differentobservers are used to rate the samesituation or patient. Normally, inter-raterreliability is calculated with Cohen’s kappastatistical test, which takes into accountthe proportion of agreement between thetwo raters in relation to the proportion ofresponses that could be expected bychance.2 Cohen’s kappa can have a valuebetween 0 and 1, with levels of 0.21–0.4indicating fair agreement, 0.41–0.6moderate agreement, and 0.61–0.8substantial agreement.2

Test-retest reliabilityTest-retest reliability assesses whethersimilar results are reached over twodistinct periods of time in unchangedconditions.The time intervals chosendepend on the variability of the domainbeing measured and the potential forchange over time.5 Test-retest reliability is

assessed by Cohen’s kappa statistical test,which is controlling for chance agreement.1

Internal consistencyInternal consistency evaluates howindividual items of the outcome measurecorrelate with each other.The qualitycriteria to assess internal consistency isCronbach’s alpha, which reports theaverage of correlations between allpossible halves of the scale.1 A very highinternal consistency (>0.9) suggests thatmany items of the measure are capturingsimilar aspects.1 Internal consistency isimportant if an outcome measure is usedto monitor a single underlying conceptwith multiple items. However, if theunderlying clinical phenomenon iscomplex, internal consistency is not sorelevant.6

Appropriateness and acceptabilityMany PROMs have been primarilydeveloped for use in research, with theemphasis on psychometric properties.However, a psychometrically-soundmeasure may not always be very practicalfor clinical use.Therefore, appropriatenessand acceptability are also used to indicatewhether a measure is suitable for itsintended use.1 Barriers for use in clinicalcare include measures that are too longfor patients to answer, or that require alot of time for administration; complicatedscoring systems; costs related to the useof the measure; or poor accessibility (thatis, they may not be fully published, fully

3134

A measure is ‘good’ if it canbe shown tohave validity,reliability,responsivenessto change,appropriateness& acceptability

available or access may be restricted).These aspects are particularly important inthe context of palliative care, wherepatients are cared for in different settings,such as at home, in hospital or in ahospice; patients’ time is limited; and theircondition, which may involve cognitiveimpairment and frailty, poses a challenge tothe use of outcome measurement.Therefore, there needs to be a balancebetween sound psychometrics and thefeasibility of a measure for clinical use.

Responsiveness to changeResponsiveness to change refers towhether the measure can detect clinicallyimportant changes over time that arerelated to the course of the disease or toan intervention, such as symptommanagement.This is particularlyimportant in outcome measurement as,by definition, outcomes are related tochange (see Chapter 1), whereasassessment of health status is related to aparticular point in time.3

The quality criteria to assessresponsiveness to change are multiple.Important questions to answer whenassessing the responsiveness of anoutcome measure are:1

4 Does the change detected by theoutcome measure correlate withthe change measured by a goldstandard?

4 Does the detected change comparewith what the patient or clinicianhas identified as an importantchange?

4 Is the detected change associatedwith changes in treatment or care?

When an outcome measure has proven tobe responsive to change, the minimallyclinically important difference (MCID)needs to be determined.The MCID isdefined as the smallest change ordifference in an outcome measure that isperceived as beneficial.10 This change caneither be retrieved by asking patientsabout differences or by calculating it usingmathematical criteria.11 MCID areavailable for many measures but, ingeneral, a difference of about 0.5 standarddeviations of the endpoint being assessedis a useful estimate.11 Having too fewpoints on a scale (for example, 0, 1 and 2)often makes the measure less responsive

to change. Floor and ceiling effects occurwhen scores from an outcome measureare not discriminated below or above acertain level (meaning that they will notdetect change).

InterpretabilityThe interpretability of an outcomemeasure refers to whether the results(which are often a number) can betranslated into something more meaningfulto the patient, the family or clinician.Aninterpretable tool should enable aresponse to these questions: What issevere? What is the cut-off point when theoutcome measure is used for diagnosis?How many points correlate with asymptom change?1

Terwee et al highlighted a range of scoresthat are helpful in assessing theinterpretability of a tool:6

4 means and standard deviations(SD*) of scores of (subgroups of) areference population (norm values);

4 means and SD of scores of relevantsubgroups of patients whose scoresare expected to differ (for example,groups with different clinicaldiagnoses, age groups, gendergroups or primary versus secondarycare setting);

4 means and SD of scores of patientsbefore and after treatment(s) ofknown efficacy;

4 means and SD of scores ofsubgroups of patients based onpatients' global ratings of change.

*standard deviation is a measure of the spreadof values of a variable around a populationmean value (see Chapter 4).

Translation into other languagesTo be able to use outcome measures withdifferent groups to compare resultsbetween countries, outcome measuresneed to be translated into other languagesby following a formal process and thesame rigorous validation process alsoapplies as for the original measure. Eventhough this is lengthy and costly,7 it is animportant procedure to ensure accuratescores when outcome measures are usedand compared. Much research has beenconducted on translating certain tools (forexample the POS), and there are alsoguidelines regarding how questionnaires

Example 3.1 Validation and translation of thePOS in Argentina adapted from Eisenchlas et al9

In order to be able to use the POS inArgentina, Eisenchlas et al9 carried outa cross-cultural adaptation andpsychometric analysis of a Spanish(Argentine) translation of the POS. Inthis case study we will focus on thecross-cultural adaptation, which hasbeen divided into three steps:

Appraisal of conceptualequivalence was performed toensure that the content of the POScovers the needs of Argentinianpalliative care patients and is relevantto them.The authors carried out aliterature review of health-relatedquality of life issues in palliative carepatients in Argentina; interviewed localpalliative care professionals; and askedpatients in two focus groups aboutissues they expected to be addressed.This process assured the conceptualequivalence of the POS in theArgentinian culture.

Intermediate forward andbackward translation, followed byan expert committee review: Twoforward and two backwardtranslations were undertaken by twoindependent translators at each phase.Several versions of the POS in Spanishwere produced, and the expertcommittee agreed on a single version.

Qualitative pre-testing: The newlycreated Argentinean POS was testedwith a sample of 65 palliative carepatients and 20 palliative careprofessionals. Following thecompletion of the POS, both groupsparticipated in individual, semi-structured interviews. Questionsaround the relevance, length,comprehension and interpretability ofthe POS were asked.After pre-testing,seven POS questions were changed.

The POS version developed after thisprocess of cross-cultural adaptationwas then psychometrically tested andvalidated.

3144

should be translated.8

Translation involves the considerationof semantic and conceptual meaning andprocedures to ensure equivalence betweencultures. Both forward (translation intothe new language) and backwardtranslation (translation of the newlanguage version back into the originallanguage) are important elements of thisprocess. In addition to the meaning ofterms used in outcome measures, accuracyin the translation requires translation inrelation to the sense of those terms.Also,the same language might have differentmeanings in different cultures.9

Informal translation without validationin the new language might lead to loss ofmeasurement properties.Therefore,considering whether the outcomemeasure has been translated adequately isan important step in assessing the qualityof the outcome measure, and this mayinfluence whether or not the tool can beused with certain groups of patients or incertain countries.

KEY POINTS

4 Validity and reliability are vitalelements of a high-quality outcomemeasure.

4 When using an outcome measure,you need to be sure that themeasure is valid, reliable, acceptableand responsive to change.

4 Some outcome measures lack keyinformation on reliability, sensitivityto change and interpretability.

4 Outcome measures need to beappropriate and acceptable topatients, and to the circumstancesthey are used in.

4 Translation of outcome measuresinto different languages must followthe same rigorous process as usedfor the development of a newmeasure.

4 An outcome measure has to showresponsiveness to change if changeover time is to be evaluated.

Further reading4 Higginson IJ, Harding R. Outcome

measurement. In:Addington-Hall JM,Bruera E, Higginson IJ, Payne S, editors.Research methods in palliative care.Oxford; New York: Oxford UniversityPress; 2007. p. 99-110.

4 Streiner DL, Norman GR. Healthmeasurement scales: a practical guide totheir development and use. 4th ed.Oxford: Oxford University Press; 2008.

4 Terwee CB, Bot SDM, de Boer MR, vander Windt DAWM, Knol DL, Dekker J, etal. Quality criteria were proposed formeasurement properties of health statusquestionnaires. J. Clin. Epidemiol.2007;60(1):34-42.

References1 Higginson IJ, Harding R. Outcome

measurement. In:Addington-Hall JM,Bruera E, Higginson IJ, Payne S, editors.Research methods in palliative care.Oxford, New York: Oxford UniversityPress; 2007. p. 99-110.

2 Streiner DL, Norman GR. Healthmeasurement scales: a practical guide totheir development and use. 4th edition.Oxford: Oxford University Press; 2008.

3 Greenhalgh J, Long AF, Brettle AJ, GrantMJ. Reviewing and selecting outcomemeasures for use in routine practice. J.Eval. Clin. Pract. 1998;4(4):339-50.

4 Nevo B. Face Validity Revisited. J. Educ.Meas. 1985;22(4):287-93.

5 US Department of Health and HumanServices, Food and Drug Administration.Guidance for industry. Patient-reportedoutcome measures: use in medicalproduct development to support labelingclaims. Silver Spring: US Food and DrugAdministration; 2009.

6 Terwee CB, Bot SDM, de Boer MR, vander Windt DAWM, Knol DL, Dekker J, etal. Quality criteria were proposed formeasurement properties of health statusquestionnaires. J. Clin. Epidemiol. 2007;60(1):34-42.

7 Dawson J, Doll H, Fitzpatrick R, JenkinsonC, Carr A.The routine use of patientreported outcome measures in healthcaresettings. Br. Med. J. 2010;340:c186.

8 Cull A, Sprangers M, Bjordal K,AaronsonN,West K, Bottomley A, et al. EORTCquality of life group translation procedure.2nd edition: EORTC Brussels; 2002.

9 Eisenchlas JH, Harding R, Daud ML, PérezM, De Simone GG, Higginson IJ. Use ofthe Palliative Outcome Scale in Argentina:A Cross-Cultural Adaptation andValidation Study. J. Pain Symptom Manage.2008;35(2):188-202.

10 Jaeschke R, Singer J, Guyatt GH.Measurement of health status:Ascertaining the minimal clinicallyimportant difference. Control. Clin.Trials.1989;10(4):407-15.

11 Sloan JA.Assessing the minimally clinicallysignificant difference: scientificconsiderations, challenges and solutions.COPD. 2005;2(1):57-62.

3154

Outcome measures produce differenttypes of scores or numbers which can beattributed to individual questions orsummarise all the questions.These scorescan be helpful and can be used in differentways either for an individual patient or fora group of patients.

Assessment and screening: individualpatient scoresOutcome measures can be used to assessand screen individual patients, andmultidimensional and uni-dimensionalmeasures can assist with this. For example,the Karnofsky Performance Status (KPS),which involves the assessment of threedimensions of health status (activity, workand self-care), can be administered by anyhealthcare professional to quickly assesspatients’ level of functioning.With a KPSscore of 60 or higher, a patient may beeligible for attending day hospice services.Alternatively, those scoring 60–40 may beeligible for attending the same service, butwith carer support.Those scoring 40 orless may be too ill to attend the dayhospice but, if they need symptom control,either a home care service, a hospice or aninpatient unit might be more appropriatefor them. In addition to assessment,outcome measures can also be used toscreen patients. For example, a single itemon mood in an outcome measure mighthighlight that a patient is depressed.Toassess this further, a more specificquestionnaire on depression (for examplethe HADS or the Beck DepressionInventory (BDI)) or a psychiatricassessment might be necessary.

When using an outcome measure,each patient’s answer usually has a

numerical value attached to it, or anumerical value which can be assigned tothe answer option if answers arecategorised. For example, single-itemmeasures such as a VAS or an NRS, whichare often used to measure the severity ofa symptom, result in one overall score. Nocalculations are then required for anindividual patient.

Unlike single-item measures, multi-itemoutcome measures sum scores ofindividual questions to an overallaggregated score. Sometimes these scoresneed to be divided by the number ofquestions, or more complicatedadjustments of the scores are necessary.The relevant information needs to befound from the user guide of the

3174

Chapter 4 Scores of outcome measures and their analysis

Example 4.1 Use of a single-item instrument for scoring symptoms

No shortnessof breath

0 1 2 3 4 5 7 8 9 10

The patient circled 6 on the NRS, indicating moderate to severebreathlessness for this patient.

Shortness of breath is as bad as it can be

A patient is attending a breathlessness clinic. The severity of breathlessness is assessed with the following NRS.

‘On a scale from 0 to 10, indicate how much shortness of breath you had on average over the last 24 hours’

6

outcome measure. For clinical purposes,outcome measures that do not requirefurther and more complicatedcalculations are more practical.

Some outcome measures giveimportant information both fromindividual items and overall scores which

can be used to inform clinical decisions, asshown in Example 4.2. In this case, thetotal POS score is useful in getting thebroad picture, whereas individual scoressuch as pain and depression giveimportant information on key aspects ofthe patient’s situation.

3184

Example 4.2 Using POS individual and overall scores, and the related clinical meaning

Patient admitted to palliative care unit with severe pain

POS Score/description Clinical meaning Team members input required

Pain 4 Overwhelmingly Not responding well to opioids Palliative care physician and nurse

Other symptoms 1 Slightly Nausea, constipation Palliative care physician and nurse

Anxiety 2 Sometimes Occasionally anxious about Palliative care physician, nurse, therapist,pain getting worse and chaplain

Family anxiety 4 Always Family anxious about deterioration Nurse, social workerpreoccupied and patient not getting better

Information 1 Hard to Did not understand why opioids Explanation by palliative care physicianunderstand are not working

Support 0 As much as I Feels supported No action currently necessary,wanted continue with volunteer input

Depressed 3 Occasionally Risk of clinically relevant depression Physician

Self-worth 3 Occasionally Risk of clinically relevant depression Further assessment and potential referral to psychologist

Wasted time 0 None at all Just admitted No action currently necessary

Personal affairs 2 Practical Financial issues, advance care Social workerproblems in planningthe process of being addressed

Total score (0-40) 20 Overall moderate palliative care needs

Monitoring change: patient scoresover timeUsing outcome measures more than oncewith the same patient enables changesand treatment effects to be monitored,and can provide useful informationregarding patient trajectories over time(improvement or worsening). For manyoutcome measures, minimal changes inscores (or MCID) are described,indicating how much difference in a scorethere needs to be in order for it to beclinically meaningful. For example, for thePOS, a variation of one point in individualitems is linked to clinical meaningfulchange.1, 2 If information about changescores does not exist, clinical judgement

should be used to determine therelevance of the change in relation to theindividual patient.

There are different ways of makingpatients’ scores that are collected overtime more useful to clinicians and patients.This can be either in the form of tables (asin Example 4.3), or by visually plotting ormapping the scores. An Excel spreadsheetcan provide both a table and a graphicalversion of the scores. Presenting theinformation in this way can aid themonitoring of treatment and theidentification of any patterns in relation toresponses to treatment, the timing ofinterventions or other salient factors (seeExample 4.4 overleaf).

3194

POS question 1st May 8th May 15th May Clinical meaning

Pain 4 3 2 Improvement

Other symptoms 1 1 1 Stable and adequate

Anxiety 2 1 3 Fluctuating, possibly increasing

Family anxiety 4 3 3 Slight improvement

Information 1 0 0 Improvement

Support 0 0 0 Stable

Depressed 3 2 2 Slight improvement, requires monitoring

Self-worth 3 1 0 Improvement

Wasted time 0 3 3 Deterioration, requires follow-up

Personal affairs 2 2 0 Slight improvement

Total score 20 16 14 Overall improvement, however some areas need follow-up

Example 4.3 Comparing POS individual and overall scores over time, and their clinical meaning

This type of visual mapping can becompleted in relation to single items (asshown in relation to pain in Example 4.4), orin relation to a number of items (as shownin Example 4.5). Displaying information inthis way can aid understanding of therelationship between different symptoms,and also complement discussions withpatients on determining which symptomsrequire the most attention.

Analysing data from several patients If a team is interested in the characteristicsof a whole patient group, rather than oneindividual patient, data can be analysed in amore sophisticated way. For largeramounts of data, the use of computersoftware is inevitable. For this, data need tobe entered into a computer, which is timeconsuming and can be liable to mistakes asthe data need to be transferred from ahard copy to an electronic format.Alternatively, some organisations useelectronic formats for data collectionwhere patients enter their answers directlyinto wireless tablet computers.This savestime and reduces the risk of mistakes.

Occasionally, when working with largersets of patient scores, data might bemissing: questions might be skipped bymistake; the patient may not have wantedto reply, may not have understood thequestion, or may not have been well enoughto answer the question; or staff may havelacked time to use the questionnaire.Although it is difficult to avoid missing datawhen working with patients with advanceddisease who are frail or close to the end oflife3, the best way to avoid missing data is tohave quality control procedures in place(for example, double checking of data,availability of questionnaires in a large fontsize, the provision of training on how to usethe outcome measure, and raising the issueof missing data with staff).

Data analysis: common descriptivestatisticsFor clinical purposes, the use of statistics isnot always required when outcomemeasures are used. However, commondescriptive statistics may be useful in certaincircumstances, for example, when comparingtwo different patient groups on two differentwards. Calculating the mean, median or SDmay be useful at times.

The mean, also known as average, isthe sum of all the scores divided by the

3204

Example 4.4 Pain scores over time using the POS patient version (question 1)

4

3

2

1

0

Nov 1 Nov 4 Nov 7 Nov 10 Nov 13 Nov 16

Q1 - Over the past 3 days, have you been affected by pain?

0 Not at all, no effect1 Slightly – but not bothered to be rid of it2 Moderately – pain limits some activity3 Severely – activities or concentration markedly affected4 Overwhelmingly – unable to think of anything else

Example 4.5 Pain and anxiety scores using two items of the POS (question 1 and 3)

4

3

2

1

0

Nov 1 Nov 4 Nov 7

PainAnxiety

Nov 10 Nov 13 Nov 16

3224

number of scores.The mean is a measureof the centre of a distribution if the valuesbeing summarised have a symmetricaldistribution.4 This is not the case if most ofthe values are distributed towards oneend of the scale or if there are manyextreme values. In the context of outcomemeasurement, it is a common and simplecalculation to do.

The median is another measure of thecentre of distribution.The median is morerobust as it is not affected by extremevalues.4 Data are arranged in increasingorder.The median is the middle value whichdivides the data in half; 50% of observationsare lower and 50% of observations arehigher than the median. If there is an evennumber of values, the median is the averagebetween the two middle values.

Median = value of order in anordered sequence

Mean, χ = =Σχn

sum of valuesnumber of values

Example 4.6 Calculating a group of patients’mean breathlessness scores using the NRS

Example 4.7 Calculating the median distressscore of COPD and lung cancer patients (fromExample 4.6)

Description: Two groups of patients, onewith COPD and one with lung cancer,have the following NRS scores for distressdue to breathlessness.

How much distress did you have dueto your breathlessness (0 = no distressat all; 10 = maximum distress I canimagine)?

COPD Lung cancer

Patient 1 10 -Patient 2 - 3Patient 3 6 -Patient 4 9 -Patient 5 - 8Patient 6 8 -Patient 7 - 7Patient 8 - 1Patient 9 7 -Patient 10 - 2

Mean 8 4.2

The average distress score for the COPDgroup is (10+6+9+8+7)/5 = 8; for the lungcancer group it is (3+8+7+1+2)/5 = 4.2.By calculating the mean, we can see thatthese two patient groups differ in theiraverage distress levels, and that COPDpatients have, on average, severe distresslevels (NRS=8) and lung cancer patientshave moderate distress levels (NRS=4.2).Nevertheless, there are two lung cancerpatients experiencing severe distress levels.

Distress due to breathlessness inCOPD and lung cancer patients,measured using an NRS

COPD Lung cancer

Patient 1 10 -Patient 2 - 3Patient 3 6 -Patient 4 9 -Patient 5 - 6Patient 6 8 -Patient 7 - 8Patient 8 - 1Patient 9 7 -Patient 10 - 2

Median 8 3

To calculate the median, the differentvalues need to be ordered:

COPD group 6 7 8 9 10

Lung cancer 1 2 3 7 8

The median in each group is the middlevalue.

Median = (n+1)/2=(5 values+1)/2= 6/2=3rd value

The median for COPD is 8 and for lungcancer 3.

(n+1)2

Median 33.5

3234

Σ(χ−χ)2

(n-1)

Example 4.8 Calculating SD using the POS total score

A group of patients using the POS have the following sum scores at admission:

POS sum score at admission

Patient 1 36Patient 2 31Patient 3 15Patient 4 36

Mean (SD) 29.5 (9.95)

Calculating the SD of the POS sum scores of four patients at admission shows that thepatient total scores are relatively widely spread (SD 9.9).The mean of all scores for thetime of admission is 29.5.

Σ(χ−χ)2

(n-1)SD =

(36-29.5)2 + (31-29.5)2 + (15-29.5)2 + (36-29.5)2

(4-1)SD =

(6.5)2 + (1.5)2 + (-14.5)2 + (6.5)2

3SD =

2973

SD =

SD = 9.95

Example 4.6 and 4.7 also highlight theimportance of working with both individualand group scores. If only groups scoreswere used, patients 5 and 7 in the lungcancer group would have been missed.

In addition to mean and median, thespread of values or variation of data can bedescribed by the SD or the interquartilerange. The SD is the average distance fromthe mean of every value. It can be used ifthe data are symmetrically distributedaround the mean.The SD will help you tosee how homogenous (same) orheterogeneous (different) your data are.Thesmaller the SD, the less spread are the data(see Example 4.8).

SD, s=

If there are too many extreme values,and therefore SD is not appropriate, theinterquartile range is an alternativemeasure to reflect the distribution ofdata. It is used in combination with themedian and describes the intervalbetween the 25th and 75th percentile;meaning that 25% of all the scores arebelow that score, and 75% of all thescores are above that score (seeExample 4.9).

Example 4.9 The 25th and 75th percentileusing the POS scores shown in Example 4.8

Ordered PercentilePOS scores

15 25th = 233136 75th = 3636

Key points

4 Single-item scales for individualpatients require no furthercalculations.

4 For multi-item scales, sum scorescan usually be calculated, butindividual scores might also giverelevant information and areimportant to consider.

4 Scores over time from the samepatient enable changes to bemonitored and provide informationon individual patients’ trajectories.

4 To compare groups of scores, simpledescriptive statistics are useful, suchas mean, median, SD and quartiles.

Further reading4 Kirkwood BR, Sterne JAC. Essential

Medical Statistics. 2nd ed. Malden, Mass.:Blackwell Science; 2003.

References1. Hearn J, Higginson IJ. Development and

validation of a core outcome measure forpalliative care: the palliative care outcomescale. Palliative Care Core Audit ProjectAdvisory Group. Qual. Health Care.1999;8(4):219-27.

2. Aspinal F, Hughes R, Higginson IJ, ChidgeyJ, Drescher U,Thompson MA.A user'sguide to the Palliative Outcome Scale.London: Department of Palliative Care,Policy and Rehabilitation, King's; 2002.

3. Bernhard J, Cella DF, Coates AS,Fallowfield L, Ganz PA, Moinpour CM, etal. Missing quality of life data in cancerclinical trials: serious problems andchallenges. Stat. Med. 1998;17(5-7):517-32.

4. Pereira-Maxwell F. A-Z of MedicalStatistics: a companion for criticalappraisal. London:Arnold; 1998.

Even though research has shown thatclinicians and researchers are generallywilling to use outcome measures withinhealthcare,1 and specifically within palliativecare2, barriers within organisations continueto prevent or hinder their routine use.2

Implementation of outcomemeasurement requires facilitation, changeand communication.This chapter isdesigned to equip clinicians, researchersand managers with ideas, tools andstrategies to enable the regular use ofoutcome measures.A widely used qualityimprovement cycle (audit) is explained, andresources that can aid outcomemeasurement implementation arehighlighted. Change management andorganisational theory are covered here toensure that implementation dimensionsare considered, pitfalls are avoided andsuccessful implementation of outcomemeasurement occurs, regardless of theorganisational context.

Change in organisations: types ofchange and enabling change Change management has the potential toaid quality improvement within healthcaregenerally.3 Change is an essentialcomponent of implementing outcomemeasurement, and can vary in size, scopeand type (for example, staggered/gradual/sharp/immediate change) (see Example 5.1).

Understanding the factors, processesand forces that drive change withinorganisations can aid the management andfacilitation of outcome measurementimplementation. “Facilitation is a techniqueby which one person makes things easierfor others”.4 Facilitation involves providingsupport to help change attitudes, habits,skills, and ways of thinking and working.Good facilitation aids change by helpingclinicians, researchers and managersunderstand what needs to change, howthis change can occur, and what theoutcome and impact of the change will

3254

Chapter 5 Quality improvement and organisational change

Example 5.1 Change in size and scope

A clinician’s choice to change from the patient POS version

to the carer POS version with one

patient

Implementation of routine use of

PROMs in a palliative care unit, hospice and home care

service

Implementation of an end-of-life care strategy

within a national healthcaresystem that relies on PROMs

to determine healthcare provider payments

Change is anessentialcomponent ofimplementingoutcomemeasurement

be.There are three core dimensions inrelation to the role of a facilitator:characteristics (for example, respect andempathy), role (for example, access tofacilitator and negotiation), and style (forexample, flexibility and presence).4 Whengood facilitation is evident, alongside goodevidence and the right environment,successful implementation of evidence-based practice can occur.4

An easy-to-use and quick method tohelp analyse external and internal forcesinvolved in change is the PESTLE methodof analysis. In essence, the PESTLE methodinvolves identifying the political (P),economic (E), social (S), technological (T),legal (L) and environmental (E) factors thatdrive and inhibit change.This method helpsidentify forces that can prompt change;helps determine whether a change isrequired and how urgent it is; and helpsidentify the resources that might berequired for the change to occur (seeExample 5.2, opposite).

Although the history of thedevelopment of this method is difficult toestablish, it is a method that can be usedvery simply, or in a very complex way, tohelp one think practically about the needfor change. A PESTLE analysis can becompleted from many differentperspectives, for example from the

perspective of an individual clinician, amulti- or inter-disciplinary team, adepartment or service, an organisation, aregion, or from a national perspective.

Acceptance or reluctance to useoutcome measures is influenced by manyfactors. For example, medical doctors andphysiotherapists may be more inclined toaccept and use outcome measures, ascomponents of their work are directlyrelated to dimensions that are easilymeasurable (for example, range ofmovement, intensity of pain). However,other professionals, who focus on moreemotional, psychological or spiritualdimensions, may find it more challenging toembrace the use of outcome measures inpractice.There are arguments for bothusing and not using outcome measuresclinically in palliative care, as illustrated in Table 5.1.

Considering the level of evidence thatsupports the change is also important.Successful implementation of evidence-based practice requires consideration ofthree core elements: the level and natureof the evidence to support the change; thecontext or environment into which thechange is to be put in place; and themethod or way in which the changeprocess is facilitated. Successfulimplementation is more likely to occur

3264

Table 5.1 Arguments for and against the use of outcome measures in palliative care

Arguments for outcome measures Arguments against outcome measures

Helps assess and screen patients and symptoms. Restricts creativity in practice.

Supports the evidence-based medicine movement Does not fit with the ethos of palliative care as it in order to ensure service quality, effectiveness detracts from patient-centred care.and accountability.

Enables identification of what works and what does Detracts from the benefits of interventions.not work in terms of clinical interventions.

3274

Description of a fictitiousserviceSt Barnaby’s is a long-establishedhospice in the UK comprising 16beds for palliative care patients.The majority of its funding comesfrom charitable sources, however, athird of the funding comes fromthe National Health Service (NHS)in England. St Barnaby’s offers abroad range of services to patientsand their families using amultidisciplinary approach to care,that is provided by palliative carespecialist doctors, nurses (locumsand permanently-employednurses), an occupational therapist,a music therapist, a chaplain andseveral volunteers.

Description of change contextRecently, the quality of UK hospiceservices has received a lot ofattention in the media; prompted bya recent high-profile case highlightingsymptom management issues with acancer patient. In response to this,numerous letters to a nationalnewspaper’s editor were published.An overwhelming feeling of theimportance of the need foradequate symptom control of alldying patients was clear from thosewho wrote in.This high-profile casehappened to correspond with therelease of the new Government’sWhite Paper which also featured inthe media.This paper highlightedthat funds for NHS-funded serviceproviders will soon be reliant ontheir performance, and that theirperformance will, in part, bemeasured by provider reportsdetailing PROMs.These eventsprompted the manager of thehospice to complete a PESTLEmethod of analysis about whetheror not to formally introduce theuse of outcome measures withinthe hospice.

PESTLE analysisForces to support formalintroduction of outcomemeasurement

4 White Paper focusing on the needfor outcome measures (Political).

4 Payment to service providers willsoon be driven by PROMs findings(Economic).

4 Pressure from patients and theirfamilies for measurement ofsymptoms to ensure adequatesymptom control (Social).

4 A new computerised patient recordis about to be used in the hospiceand this could be developed toinclude outcome measurement data(Technological).

4 A recently introduced Carers EqualOpportunities Act requires carerassessments by healthcare providers(Legal).

4 Shared rooms in the hospice meansthat patients and families are awareof symptom issues and that it isimportant to ensure all patients’symptoms are adequately controlled(Environmental).

Forces against the formalintroduction of outcomemeasurement

4 Introduction of outcomemeasurement will require trainingand additional resources such astime, and cover for locum staffinvolved in providing and receivingthe training (Economic).

4 Some staff are resistant to usingoutcome measures with patients asit is not always viewed as beingconducive to good patientexperience of palliative care (Social).

4 Outcome measurement may createadditional paperwork for staff andthis may reduce patient contacthours (Economic).

4 The nurses’ station is small andoutcome measurement paperworkwill mean that staff need morespace to complete paperwork(Environmental).

Example 5.2 Decision to formally introduce outcome measures into a UK-based hospice

when: (a) there is high-quality evidence tounderpin the change; (b) the context ororganisation is able to cope with change;and (c) effective facilitation is in place.4

Facilitation methods to helpimplement outcome measurement are:

4 A cascade management approachthat involves the promotion ofautonomy in staff regarding theiruse of outcome measures;

4 Staff involvement in decision making;4 The use of measures that can be

analysed within existing resources;4 Advice regarding data analysis;4 The use of measures that are

relevant to clinical care;4 The use of staff training to

complement implementation;4 Encouraging staff to view outcome

measures as an integral componentof clinical care.5

Understanding perspectives withinorganisational contextsUnderstanding the culture of anorganisation, or an organisation’s valuesystem, including the dominant values,visions, perspectives, standards andbehaviours6, 7 is important whenimplementing outcome measurement.Thisinvolves understanding the relationshipswithin the organisation (usuallyunderstood through leadership roles) andthe organisation’s approach tomeasurement (for example, monitoringsystems).4 A useful framework forunderstanding the culture of organisationswas highlighted by Charles Handy, whoproposed four types of organisationalculture (see Example 5.4, opposite).8

Consideration of these different culturesin attempting to implement outcomemeasurement may aid the development ofstrategies to support the implementationof measures within an organisation.

As organisations are made up ofindividuals, understanding the individualwithin their organisational context canalso help understand individual responsesto change that involves outcomemeasurement.This understanding can thenbe used to help enable change in individualstaff that are resistant to change (seeExample 5.3).

Determining what is needed and bywhom: agreeing a data collectionplanDuring the process of implementingoutcome measures, discussions about whatdata to collect, for what reasons, and bywhom, need to occur. Systems mappingcan help with this task.This consists ofdeveloping a pictorial representation ofthe organisation (or local area) and linkswithin and between identified components,as illustrated in Example 5.5. Systemsmapping can also be used to identify linksbetween those components within andthose outside of an organisation. Systemsmaps are helpful in determining what typeof information is required and by whom.

Example 5.3 Tips for a data collection plan

Questions to consider, and todiscuss with others, whenagreeing a data collection plan:

4What do we want to collect? 4What do we need to collect? 4How can we improve the return

rate of outcome measurementdata?

4What do people want from thedata?

4What do people need from thedata?

4How can we present findings in ameaningful way?

4What systems can we put in placeto make the data collection,inputting and reporting processeasy, and to minimise the impactof these tasks on other duties?

3284

3294

Example 5.4 Types of organisational culture (adapted from Handy 19998)

Power culture

Role culture

Person culture

Task culture

4 Influence and control within an organisation is centred on a central figure or group.4 The relationships with people in that central group matter more than informal titles

or positions within organisations.4 Influencing the key opinion leaders and power holders is required for outcome

measurement implementation.4 Implementation might not succeed without their endorsement.

4 Delegation of authority is based on organisational structure or formal roles. Forexample, the role of the clinical nurse might be very important in terms ofdetermining the work of other nurses in the palliative care unit, or the role of thepalliative care consultant may be very important in terms of identifying key clinicalareas of specialty within a hospice.

4 It is important to recruit the most senior staff of all the clinical disciplines to theimplementation task as they ultimately hold responsibility for introducing outcomemeasures across the organisation, and to more junior staff. For example,implementation of outcome measures might automatically fall to the most seniornurse on the ward.

4 Involves an emphasis on the task at hand, rather than the roles of the peoplerequired to complete the task.

4 Teams are formed to solve particular problems within the organisation.4 Clinical staff that hold outcome measurement competencies and knowledge, and

service managers with change management expertise, might be enlisted to constructand deliver an outcome measurement implementation programme with in thesetypes of organisations.

4 An outcome measurement team involving various members from differentprofessions and rankings within an organisation might be established to introduceoutcome measures within the organisation.

4 Involves scenarios where individuals or groups of individuals exercise great controland influence within the organisation.

4 Less common or relevant to healthcare environments, and more relevant tobusiness/private sector organisations.

In a hospice, clinicians use PROMs in the care of patients.The results mightbe presented in the form of the actual measures completed, or in an Excelspreadsheet detailing patient scores over time.Families may want to know what they can do to help attend to areas ofneed.Verbal reports of the clinical decisions and interventions that wereinformed by the outcome measure data may be requested by families.Consequently, clinicians may need to be able to help educate families aboutwhat is required.Volunteers may require general information about outcome measures usedwithin the organisation.A brief summary sheet or information pack may helpvolunteers to understand outcome measurement and the PROMs used inthe hospice.Managers may require summed or aggregated reports regarding symptommanagement with all patients receiving a certain type of intervention/service within the organisation.These types of reports, for example, may begenerated by relational databases involving simple descriptive statisticalcalculations (see Chapter 4).

Equipping organisations for outcomemeasurement successOnce it has been agreed that outcomemeasures will be implemented and usedwithin an organisation, a changemanagement plan can be constructed toaid comprehensive implementation. Ideally,this type of plan should be constructedwith those involved in the change; bedeveloped with knowledge about theresources that are available; and informedby the outcome that is required.

Considering these perspectives isuseful as it helps determine how peoplemay perceive, respond, become involvedand embrace the change ahead.8 Changeperceived as being owned by oneself orone’s organisation (active change), incontrast to a change that is perceived asimposed (reactive change), may requirefewer resources and less time toimplement. Considering change in this waymay lead to the identification of the bestway to enable outcome measurementimplementation and, importantly, it willhelp identify correspondingcommunication strategies that may helpwith the change. Example 5.6 sets outsome principles for understandingorganisational change and Example 5.7lists some rules for managing change.

Developing a good communicationstrategy will involve creatingopportunities for stakeholders to haveownership of the change process and toexpress their thoughts about what isrequired and how the change can beimplemented. Keeping in mind the thingsthat drive people (for example, clinicalcare for clinicians, governance formanagers, and accountability and qualityfor commissioners) can help shape thecontent of communication with them. Forexample, when highlighting the benefits ofoutcome measurement in discussionswith clinicians, the potential of themeasures to improve assessment andcare might be emphasised.While formanagers the notion of how outcomemeasurement can help streamline andimprove services is important.

Making a real difference throughaudit and achieving successfuloutcomesAn audit can be completed once thechange has been implemented andoutcome measurement is being used

3304

Example 5.5 An example of a hospice systems map in relation to use of PROMs

Patientusing a PROM

Families

Volunteers who work with

patients and families and hear about outcome measurement

Clinicians who use PROMs with

palliative care patients and

families

Hospice managers who

rely on outcome measurement

data collected by clinicians to

report to funders

within the organisation.Audits focus eitheron individual patient care (case audit), aservice (for example at a departmentlevel) or an organisation.Audits can helpidentify major risks, reinforceimplementation of evidence-basedpractice, influence improvements andensure governance (or the accountabilityof services).Audits can also aid qualityassurance.The audit cycle involves settingstandards and goals; monitoring andobserving practice; and then using thefeedback or findings to improve quality(see Figure 5.2).

3314

Example 5.6 Understanding organisationalchange9

Principles for understandingorganisational change (quotedfrom Pugh 19939)

1 Organisations are organisms.2 Organisations are occupational

and political systems.3 All members of an organisation

operate simultaneously in threesystems – the rational, theoccupational, and the political.

4 Change in people may occurdifferently:

4Those who are successful yet arealso experiencing tension incertain parts of their work mayhave confidence in their abilityand the motivation to change;

4Those who are successful mayhave the confidence to change butmight also need to be motivatedto change;

4Those who are not experiencingsuccess may resist change as theymay feel they need to protectthemselves and that staying thesame might help them do this.

Example 5.7 Rules for managing change9

Six rules for managing change(quoted from Pugh 19939)

1 Work hard at establishing theneed for change.

2 Do not think out the change,think through it.

3 Initiate change through informaldiscussion to get feedback andparticipation.

4 Positively encourage thoseconcerned to articulate theirobjectives.

5 Be prepared to change yourself.6 Monitor the change and reinforce it.

Figure 5.2 The audit cycle10

Preparingfor audit

Makingimprovements

Measuringperformance

level

Sustainingimprovements

Selectingcriteria for

audit review

Importantly, where standards of care arenot established, pre-audit activity can becompleted to help establish standards.Thistype of audit activity is useful for newservices that are being introduced and fornew interventions. Pre-audit activity issimilar to the usual audit cycle, however,instead of measuring performance inrelation to already established standards,the first step is identifying what standardsare currently being achieved, or ones thatmight be possible.Audit can be beneficialto patients, staff and organisations (seeExample 5.8).

Similarly to change management, theprocess of audit implementation canbe aided by engaging people withthe audit process to ensure andenable their ownership of theprocess. Helping people tounderstand the relevance of theaudit outcomes to their own workmay aid data collection and result inthe sustained use of outcomemeasures. Some useful tips forsuccessful audit are given inExample 5.9.

Example 5.9 Tips for successful audit inoutcome measurement

4 Each outcome measurement auditshould form part of a structuredaudit programme.

4Various stakeholders (for examplemultidisciplinary team members)need to be involved in the auditand the actions that result fromthe audit.This includes serviceusers and managers.

4Outcome measure audits shouldalso include measures related toprocess (see Figure 1.1, page 8).

4The published evidence aboutoutcome measurement should beconsidered in relation to whatstandard is being measured orestablished.This ensures thatgood standards are set andimproved upon.

4 Staff involvement in audit is key topromoting ownership.

4 Establishing a culture of criticalenquiry can be aided throughencouraging staff to exerciseautonomy in audit activities.11

3324

Example 5.8 The benefits of audit to...

Patients

4 Identifying and addressing practice-based problems.4 Identifying resource requirements for services and interventions so

that patients get what they require.4 Identifying service use and service needs.

Palliative care staff and clinical departments

4 Enabling the monitoring and review of the quality of care provided.

4 Identifying systematic ways of addressing clinical problems orchallenges.

4 Identifying areas for improvement, and ensuring that difficulties areconsidered in more detail.

Organisations

4 Providing data to measure the organisation’s performance againstpalliative care standards.

4 Identifying areas for service delivery improvement.4 Bringing together important information for reports to funders.4 Enabling comprehensive and summary reports for those working

within the organisation to aid self-monitoring and qualityimprovement.

BenchmarkingBenchmarking is the process of comparing,sharing and developing practice in order toachieve and sustain best practice.12

Benchmarking is useful in establishing howsimilar organisations compare.As aconsequence, best practice can be identified;beacon (leading) sites can become knownand used to aid wider development; andnational standards can be established.Similarly to the audit cycle, benchmarkinginvolves a cyclical series of steps to identifyand improve standards (see Figure 5.3).12

An excellent example of a national andvoluntary programme where benchmarkingof standards involves outcomemeasurement is the Palliative CareOutcome Collaboration (PCOC) inAustralia. PCOC uses established standardsof palliative care to develop and support anational benchmarking system that willcontribute to improved outcomes.Thisnational initiative allows for the collection,analysis and reporting of large sets ofoutcome measurement data. Informationabout PCOC and examples of theirreports can be accessed via their websiteat http://chsd.uow.edu.au/pcoc/

As with all types of aggregated (orsummed) outcome measurement data, thefindings should always be interpreted withthe following factors considered:

4 What measures were used? 4 What is the context of the data that

were collected? 4 Do the presented datasets compare,

or do they require adjusting beforecomparison can take place?

4 What is the response rate? 4 Was there any bias in the responses

that were provided?

Opinion leaders and facilitators arerequired for benchmarking programmes tobe successful. Opinion leaders are similar tofacilitators in that they can influence change,habits, practice and therefore outcomes.However, they can also be different fromfacilitators in that they may draw more upontheir status and technical skills to influencechange, rather than drawing upon theinterpersonal and group skills that arerequired for successful facilitation. Socialnetworking may also be more important inrelation to the role of an opinion leader.Opinion leaders may operate as facilitators asthere is overlap between these two roles.4

Agreebest practice

Review achievement towards best

practice

Produce and implement action

plan aimed at achieving

best practice

Disseminate improvements and/or review

action plan

Assess clinical area against best practice

3334

Figure 5.3 Cyclical approach to benchmarking12

Key points

4 Improving quality and implementingoutcome measurement inevitablyinvolves change.

4 Change is aided by understandingthe type of change required; theforces inhibiting or encouraging thechange; the resources required forthe change; and the meaning of thechange to the various stakeholdersand participants.

4 Promoting outcome measures,establishing the benefits andrelevance of outcome measurement,and good communication are key tothe implementation of outcomemeasurement in palliative care.

4 Audit and benchmarking areimportant in identifying standards ofoutcome measurement practice,areas of good measurement practiceand areas to improve.

4 Opinion leaders and facilitators arecentral to these processes.

Further reading4 Dunckley M,Aspinal F, Addington-Hall J,

Hughes R, Higginson IJ.A research studyto identify facilitators and barriers tooutcome measure implementation. Int. J.Palliat. Nurs. 2005;11(5):218-25.

4 Higginson IJ. Clinical audit in palliativecare. Oxford: Radcliffe Medical Press;1995.

4 Mullins LJ. Management and organisationalbehaviour. 6th ed. Harlow: PearsonEducation; 2010.

4 Education resources, Clinical governanceScotland. www.clinicalgovernance.scot.nhs.uk/section2/audit.asp

4 Cooper J, Pettifer A. Promoting ownershipin palliative care audit. Int. J. Palliat. Nurs.2004;10(3):119-22.

4 The Palliative Care OutcomeCollaborative (PCOC) websitehttp://chsd.uow.edu.au/pcoc/about_pcoc.html

References1 Dawson J, Doll H, Fitzpatrick R, Jenkinson

C, Carr A.The routine use of patientreported outcome measures in healthcaresettings. Br. Med. J. 2010;340:c186.

2 Bausewein C, Simon ST, Benalia H,Daveson B, Desliens L, Downing J, et al.PRISMA WP4 final report: Internationalsurvey on outcome measures. London:DGP and King's College London; 2010.

3 Grol R, Baker R, Moss F. Qualityimprovement research: understanding thescience of change in health care. Qual. Saf.Health Care. 2002;11(2):110-1.

4 Kitson A, Harvey G, McCormack B.Enabling the implementation of evidencebased practice: a conceptual framework.Br. Med. J. 1998;7(3):149-58.

5 Dunckley M,Aspinal F, Addington-Hall J,Hughes R, Higginson IJ. A research studyto identify facilitators and barriers tooutcome measure implementation. Int. J.Palliat. Nurs. 2005;11(5):218-25.

6 Cole G. Organisational behaviour: theoryand practice: DP Publications; 1995.

7 Mullins LJ. Management and organisationalbehaviour. 6th ed. Harlow: PearsonEducation; 2010.

8 Handy C. Understanding organizations.4th ed: Penguin Business Management;1999.

9 Pugh D. Understanding and managingorganisational change In: Mabey C, Mayon-White B, editors. Managing change. 2ndedition: Paul Chapman; 1993.

10 Benjamin A. Practice:The Competentnovice: audit - how to do it in practice. Br.Med. J. 2008;336(7655):1241-5.

11 Cooper J, Pettifer A. Promoting ownershipin palliative care audit. Int. J. Palliat. Nurs.2004;10(3):119-22.

12 Department of Health. Essence of Care:Patient-focused benchmarks for clinicalgovernance. London: Department ofHealth; 2001.

3344

In this chapter we provide a wide range ofresources for the use of outcomemeasures in palliative care.The list is notconclusive, but is aimed at helping thosewho want to find out more aboutoutcome measurement in general, orabout specific outcome measures.

WebsitesThere are a variety of websites thatprovide information on outcomemeasurement in palliative care, either aspart of a general website or sites thatare specifically designed for palliativecare. No single website contains all therelevant information on all existingoutcome measures; however, there arewebsites that contain all the relevantinformation about specific individualoutcome measures.

www.csi.kcl.ac.uk/tools.html Irene Higginson, Department of PalliativeCare, Policy and Rehabilitation, King’sCollege London, UK – regular updatesDetailed information on the Palliative careOutcome Scale (POS) and the Support TeamAssessment Schedule (STAS)

www.palliative.org Robert Fainsinger, Edmonton, Canada – up-to-dateWebsite of the Palliative Care Program inEdmonton; primarily ESAS

www.facit.org FACIT measurement systemFACIT with specific measures for differenttumour entities, different symptoms andnon-cancer specific scales (includingpalliative care); variety of language versionsavailable

www.dyingwell.com/MVQOLI.htm Ira Byrock, USA – regular updatesMissoula-VITAS Quality of Life Index(MVQOLI) – with Guide to use the MVQOLI

www.iqola.org/instruments.aspxwww.sf-36.org USA – regular updatesInternational quality of life assessment projectto validate and translate the SF-8/12/36;Short form Health Survey SF-8/12/36

www.proqolid.org PRO and QoL database = PROQOLIDMapi-Research-Trust, France – regularupdatesSearch in: alphabetic, generic, population(including. ‘terminal patients’), dimension,disease, author’s name, language, type ofinstrument, mode of application; 690instruments, >1100 translations, 82 databasesare included, with short descriptions and links15 outcome measures under ‘terminalpatients’

www.caresearch.com.au/caresearch/ClinicalPractice/ServiceIssues/Audit/POS/tabid/247/Default.aspx CareSearch: palliative care knowledgenetwork (Flinders University,Australia) –regular updatesDifferent sections (for example, clinicalpractice, finding evidence) with a presentationof tools; a variety of tools for differentpurposes (clinical practice, research, audit);Detailed information on CAMPAS-R, ESAS;LCP, POS, PCOC, STAS

www.chcr.brown.edu/pcoc/toolkit.htm TIME – Toolkit for instruments to measureend of life care; Joan Teno, USA – lastupdate 2004Range of outcome measures are presented andreviewed, with advice on how to choose anduse them (audit, research); 188 instruments

3364

Chapter 6 Where to find more information

including: advance care planning, carer well-being, continuity of care, emotional symptoms,functional status, grief and bereavement,physical symptoms, quality of life, spirituality

http://palliative.info/pages/Tools.htm Mike Harlos,Winnipeg, Manitoba, Canada –regular updatesOffers an organised, up-to-date collection oflinks to web-based palliative care resources, aswell as locally-developed palliative carematerial; section with ‘Assessment andEvaluation Tools’ (links and PDFs); someinstruments are presented/linked: EdmontonFunctional Assessment Tool (EFAT), KPS,Palliative Performance Scale (PPS) - Version 2,MVQOLI, the Multidimensional Quality of LifeQuestionnaire for persons with HIV/AIDS(MQOL-HIV), Australian GovernmentDepartment of Veterans' Affairs Pain andSymptom Control Measurement Tool,Edmonton Staging System (ESS), ESAS

www.promotingexcellence.org Originally Robert Wood Foundation, nowhosted by Growth House, Inc. – lastupdate 2009Sections: clinical care, educational, evaluation,organisational.Within evaluation: clinical quality,community assessment, cost/utilisation,education, organisational, programmatic, providerassessment, satisfaction/perception of care; keyclinical assessment and research tools; 31instruments are described and links provided

www.hospicecare.com/resources/pain-research.htm International Association for Hospice andPalliative Care (IAHPC) official website,USA – regular updatesOfficial website of IAHPC; section with

‘Assessment and Research Tools’; ~40instruments are presented with links/PDFs

Systematic reviewsOver recent years, several systematicreviews have been published on variousaspects of outcome measurement inpalliative care (by year of publication)

Albers, G. et al. (2010) Evaluation ofquality-of-life measures for use inpalliative care: a systematic review,Palliat. Med., 24(1), 17-37.To make an inventory of all currentlyavailable QoL measurement tools suitablefor use in the palliative care population andto assess the content and clinimetric qualityof the instrument; PubMed, Embase, Cinahl,PsychInfo (engl.+dutch) (1990-2008);Inclusion: development/validation of the tool,QoL in non-curative treatment patientsIdentified: 2,015 references; Included: 33 tools(36 studies); e.g. ESAS, FACIT-PAL, MSAS, POS Ranking: 1. MQOL, 2. QUAL-E, 3. QODD;Conclusion: Many instruments were identified,but most were not yet evaluated. Evaluationof existing instruments with good contentvalidity should have priority over developmentof new ones.

Vivat, B. (2008) Measures of spiritualissues for palliative care patients: aliterature review, Palliat. Med., 22(7),859-868.To discuss data from a literature reviewabout measures of spiritual issues forpalliative care patients (as part of adevelopment process of an instrument);Medline, Cinahl, ClinPsyc (1996-2001/2001-2007) (terms: cancer AND spiritul*)Identified: 1,066 references; Included: 29

measures; ‘functional’ (spiritual health orspiritual well-being, function): e.g. FACT-Sp-Ex,MiLS, SpIRIT; ‘substantive’ (spiritualbeliefs/experiences – content): e.g. INSPIRIT,SAS, SpS; Conclusion: none are entirely suitablefor use with palliative care patients in theUnited Kingdom or continental Europe.

Jordhoy, M.S. et al (2007) Assessingphysical functioning: a systematicreview of quality of life measuresdeveloped for use in palliative care,Palliat. Med., 21(8), 673-682.How, and to what extent, physicalfunctioning assessments have been includedand performed in QoL instrumentsdeveloped for palliative care in particular.Although they focussed on ‘physicalfunctioning’, they did a general systematicreview of QoL tools in palliative care.Medline (2005); Inclusion: life threateningillness/palliative care specificIdentified: 1,326 references; Include: 224instruments (general) – of these 39 aspalliative care specific; uni-/bi-dimensional: e.g.ESAS,VAS; Multidimensional: e.g. QUAL-E,STAS, POS; Conclusion: Despite itsimportance, physical functioning assessmentseems to be a minor part of palliative careQoL instruments.

Mularski, R.A., et al (2007) ASystematic Review of Measures ofEnd-of-Life Care and Its Outcomes,Health Serv. Res., 42(5), 1848-1870.To identify psychometrically soundmeasures of outcomes in end-of-life careand to characterise their use inintervention studies; update of thesystematic review conducted by Lorenz(see overleaf): 09/2004-11/2005

3374

Identified: 24,423 articles; Included: (see below)35+48+16 new = 99 tools; Conclusion: Ingeneral, most measures have not undergonerigorous development and testing. Measuredevelopment in end-of-life care should focus onareas with identified gaps, and testing should bedone to facilitate comparability across the caresettings, populations and clinical conditions.Intervention research should use robustmeasures that adhere to these standards.

Pearson, E. J.,Todd, J. G. and Futcher,J. M. (2007) How can occupationaltherapists measure outcomes inpalliative care?, Palliat. Med., 21(6),477-485.To identify an outcome measure foroccupational therapy interventions withpalliative clients, in particular homeassessments; Medline, Cinahl, PsychInfo,CareSearch, PROQoLID, DARE, PallCareIndex (1980-2006); Inclusion: QoL,palliative care, occupational therapyIdentified: 45 tools; Included: 24 tools, e.g.EORTC-QOL-C30, FACT-G, PACA, POS;Conclusion:The research found that it may befeasible for occupational therapists to use aQoL tool as a routine part of assessing eachpalliative care patient, with the objective offocusing interventions on priority areasidentified by the patient.

Richardson,A., Medina, J., Brown,V.and Sitzia, J. (2007) Patients’ needsassessment in cancer care: a reviewof assessment tools, Support. CareCancer, 15(10), 1125-1144.To discuss the importance of systematicassessment of needs in routine care and thecontribution tools can make to this process;Medline, Embase, BNI, ERIC, Cinahl (1984-2004); Inclusion: clinical purpose; Exclusion:research purpose, single domain, satisfaction(comprehensive list of in/exclusion)Identified: 1,803 papers – 36 tools; Included:15 tools

Kirkova, J., et al (2006) CancerSymptom Assessment Instruments:ASystematic Review, J. Clin. Oncol.,24(9), 1459-1473.To evaluated currently available symptomassessment instruments for adult cancerpatients.A secondary objective was tocompare instruments by psychometriccriteria; Medline, Embase, Cinahl, CochraneReviews, BIOSIS (1980/90-2004); Inclusion:symptom assessment; Exclusion: QoL

Identified: 76 articles + 2 conference papers;Included: 21 instruments; symptom-targeted(<5 symptoms): e.g. INV, HADS, NS; Multi-symptom (>5 symptoms): e.g. ESAS, MSAS +modified instruments; Conclusion:Theinstruments vary in symptom content andextent of psychometric validation. Bothcomprehensive and shorter instruments havebeen developed, and some instruments areintended for specific symptom assessment orsymptoms related to treatment.There is noideal instrument, and the wide variety ofinstruments reflects the different settings forsymptom assessment.

Lorenz, K., et al. (2004) End-of-lifecare and outcomes. Summary,Evidence Report/TechnologyAssessment No. 110. (Prepared bythe Southern California Evidence-based Practice Center), Rockville,MD:AHRQ Publication.Lorenz, K. (2008) Progress inMeasuring and Improving Palliativeand End-of-Life Quality, J. Palliat.Med., 11(5), 682-684.Lorenz, K.A., et al (2008) Evidencefor Improving Palliative Care at theEnd of Life:A Systematic Review,Ann. Intern. Med., 148(2), 147-159.Lorenz, K.A. et al (2006) QualityMeasures for Symptoms andAdvance Care Planning in Cancer:ASystematic Review, J. Clin. Oncol.,24(30), 4933-4938.Focusing on the outcomes: patient andfamily satisfaction; pain, dyspnoea,depression and anxiety, and behaviouralproblems in dementia; continuity;caregiving burden other than bereavement;and advance care planning.A systematicreview was conducted to evaluate:1 The scope of the end-of-life population.2 Outcome variables that are valid

indicators of the quality of the end-of-life experience for the dying person andsurviving loved ones.

3 Patient, family and healthcare systemassociated with better or worseoutcomes at end-of-life.

4 Processes and interventions associatedwith improved or worsened outcomes.

5 Future research directions for improvingend-of-life care.

Medline, DARE, NICE, NLM (1990-2004)Inclusion: published, English, humans Identified: 21,745 articles; Include: 35 tools (fromTIME - Toolkit of Instruments to Measure End of

Life Care; up to the year 2000) + 48 newinstruments (200-2004) = 83 tools in total,e.g. ESAS, EORTC-QOL-C30, FACT, MSAS, POS,QODD, STAS; Conclusion:With regard tomeasures, the review identified one high-quality, widely-recognised resource. Measuredevelopment is most advanced for cancerpopulations or mixed populations that consistlargely of cancer patients.The largest numberof measures evaluated quality of life, quality ofcare, and symptoms.The literature documentsmany measurement challenges including proxyrespondents, timing of interviews, andcognitive thresholds.

Bruley, D. K. (1999) Beyond Reliabilityand Validity:Analysis of SelectedQuality-of-Life Instruments for Usein Palliative Care, J. Palliat. Med.,2(3), 299-309.To review quality of life instruments fortheir potential usefulness in the palliativecare setting. Conceptualisations of qualityof life throughout history, andcontemporary conceptualisations ofquality of life are briefly discussed. Medline(1992-1998), Cinahl (1982-1997)Inclusion: multidimensional, self-reported,appropriateness in palliative careIdentified: 20 instruments; Included: 6instruments; SF-36, EORTC-QOL-C30, QLI,HQLI, MQOL, MVQOLI; Conclusion:Theresearcher or clinician should consider all ofthese factors when choosing the quality of lifeinstrument that best fits the purpose.

Hearn, J. and Higginson, I. J. (1997)Outcome measures in palliative carefor advanced cancer patients: a review,J. Public Health, 19(2), 193-199. Toidentify and examine outcome measuresthat have been used, or proposed for use,in the clinical audit of palliative care ofpatients with advanced cancer, and tosystematically assess these using well-defined criteria; Medline, Cancerlit,Healthplan, Oncolink (1985/ 1991-1995);Inclusion: cancer, measure contained morethan one domainIdentified: 41 measures (list of excluded isgiven); Included: 12 measures (5-56 items)e.g. ESAS, EORTC QLQ-C30, PACA, STAS;Conclusion: Each measure meets some, butnot all, of the objectives of measurements inpalliative care, and fulfills some, but not all, ofthe criteria for validity, reliability,responsiveness and appropriateness.

3384

Books Most major textbooks on palliative careinclude chapters or sections on outcomemeasures.

Addington-Hall J., et al. ResearchMethods in Palliative Care. Oxford:Oxford University Press; 2007 Chapter 7: Outcome Measurement (IreneHigginson, Richard Harding)7 measures in more detail (POS, STAS, ESAS,Zarit Burden etc.)

Bruera, E., Higginson, I.J., Robb, S.D.,von Gunten, C.F.. Textbook ofPalliative Medicine. 1 ed. London:Hodder Arnold; 2006 Chapter 40:Tools for pain and symptomassessment in palliative care; Chapter 41:QoL assessment in palliative care Several symptom assessment tools aredescribed, only a few QoL measures arementioned

Hanks, G., Cherny, N., Christakis,N.A., Fallon, M., Kaasa, S., Portenoy,R.. Oxford Textbook of PalliativeMedicine. 4 ed. Oxford: OxfordUniversity Press; 20106.3 QoL in palliative care - principles andpractice; 7.7 The measurement of pain andother symptoms; 7.10 Clinical andorganisational audit and qualityimprovement in palliative medicine;6.3: several measures are described; (e.g.EORTC QOL-C30, FACT-G, SEIQOL, MQOL)

Walsh, D., et al. Palliative Medicine.Ist edition. Philadelphia: SaundersElsevier; 2009Chapter 63: Clinical symptom assessment;Chapter 64: Qualitative and quantitativesymptom assessment; Chapter 65:Measuring QoLSeveral instruments of symptom assessmentand QoL are described and linked

3394

ADL Activities of Daily LivingAKPS Australia-modified Karnofsky Performance Status

BDI Beck Depression InventoryCAMPAS-R Cambridge Palliative Audit Schedule

EAPC European Association for Palliative CareEFAT Edmonton Functional Assessment Tool

EORTC-QLQ C30 European Organisation for Research and Treatment ofCancer Core Questionnaire – Quality of Life

ESAS Edmonton Symptom Assessment ScaleESS Edmonton Staging System

FACIT Functional Assessment of Chronic Illness TherapyFACT-G Functional Assessment Cancer Therapy–General

GHQ General Health QuestionnaireHADS Hospital Anxiety and Depression ScaleIAHPC International Association for Hospice and Palliative Care

KPS Karnofsky Performance StatusLCP Liverpool Care Pathway

MCID Minimally clinically important differenceMQOL McGill Quality of Life Questionnaire

MQOL-HIV Multidimensional Quality of Life Questionnaire forpersons with HIV/AIDS

MS Multiple sclerosisMSAS Memorial Symptom Assessment Scale

MSAS-SF Memorial Symptom Assessment Scale - Short FormMVQOLI Missoula-Vitas Quality Of Life Index

NHS National Health Service (UK)NRS Numerical rating scale

PACA Palliative Care Assessment toolPCOC Palliative Care Outcome Collaboration

POS Palliative care Outcome ScalePPS Palliative Performance Scale

PRISMA Reflecting the Positive DiveRsities of European PrIoritiesfor ReSearch and Measurement in End-of-Life CAre

PROMs/PROs Patient-Reported Outcome MeasuresQODD Quality of Death and Dying (measure)

QoL Quality of lifeQUAL-E Quality of Life at End of Life (measure)

SD Standard deviationSEIQoL-DW Schedule for the Evaluation of Individual Quality of Life-

Direct WeightingSTAS Support Team Assessment ScheduleVAS Visual analogue scaleZBI Zarit Burden Inventory

List of acronyms

PRISMA partners

CENTRO DE ESTUDOS E INVESTIGAÇAO

EM SAUDE DA UNIVERSIDADE DE COIMBRA

Deutsche Gesellschaft fürPalliativmedizin e.V.

www.prismafp7.eu