Associations between online patient ratings and traditional quality measures in the English NHS

7/30/2019 Associations between online patient ratings and traditional quality measures in the English NHS

1/28

Associations between Internet-based patient ratingsand conventional quality measures in the English NHS

November 6 th 2012

Felix Greaves Department of Primary Care and Public Health Imperial College London [email protected] @felixgreaves
mailto:[email protected]:[email protected]


2/28

We love rating things on the internet


3/28

We now do this in health too


4/28

The number of ratings is increasing

USA UK

Gao et al., JMIR, 2012 Greaves et al., JMIR, 2012

0

5000

10000

15000

20000

25000

A u g - 0

8

O c

t - 0 8

D e c -

0 8

F e

b - 0

9

A p r -

0 9

J u n - 0

9

A u g - 0

9

O c

t - 0 9

D e c -

0 9

F e

b - 1

0

A p r -

1 0

J u n - 1

0

A u g - 1

0

O c

t - 1 0

D e c -

1 0

F e

b - 1

1

A p r -

1 1

J u n - 1

1

A u g - 1

1

O c

t - 1 1

D e c -

1 1

C u m u l a

t i v e n u m

b e r o

f h o s p

i t a l r a

t i n g s


5/28

What some clinicians think


6/28

What people say about online rating

Why it s bad1. Potential straining of

doctor-patient relationships

2. Reviews may be maliciousor fake

3. Selection bias by thoseleaving reviews

4. Lack of meaningful data ontechnical quality of healthcare

Why it s good1.Doctors can often be poor

judges of their patients

satisfaction and experience2.Feedback changes doctors

performance3.People will inevitably use

the Internet to voiceopinions, so why notcapture this information in auseful form


7/28

Where we are

Physician and hospital ratings websites arebecoming more popular

They remain controversial The relationship between ratings and hard

measures of quality is not known Can we compare ratings to other healthcare

quality measures?


8/28

The opportunity for a natural experiment


9/28

Getting the data


10/28

We thought we could compare ratings with sometraditional measures of quality

Vs.


11/28

Average scores

Score Mean rating

(out of 5) Range

The environment where I

was treated was

3.6 2.6-5.0

I was treated with dignityand respect by the hospitalstaff

4.0 2.7-5.0

I was involved withdecisions about my care

3.8 2.4-5.0

The hospital staff workedwell together

4.1 2.9-5.0

Greaves et al. BMJ Quality and Safety, 2012


12/28

Average scores

Score Mean rating

(out of 5) Range

The environment where I

was treated was

3.6 2.6-5.0

I was treated with dignityand respect by the hospitalstaff

4.0 2.7-5.0

I was involved withdecisions about my care

3.8 2.4-5.0

The hospital staff workedwell together

4.1 2.9-5.0

Greaves et al. BMJ Quality and Safety, 2012

The majority of ratings are positive 67% would recommend to a friend


13/28

NHS ChoicesMeasure

Survey question:Spearman

Rho p value

Proportion of patientsrecommending

Overall, how would you rate thequality of care you received

0.40


14/28

NHS ChoicesMeasure

Survey question:Spearman

Rho p value


Overall, how would you rate thequality of care you received

0.40


15/28

Ratings compared to outcomes

NHS Choices Measure Other variable Spearman

Rho p value


Hospital Standardised MortalityRatio

-0.20 0.01


Standardised morality rate for highrisk conditions

-0.22 0.01


Standardised morality rate amongsurgical inpatients with serioustreatable complications

-0.00 0.99


Standardised mortality rate fromlow mortality conditions

0.03 0.70


Emergency readmission rate within28 days

-0.31


16/28

Ratings compared to outcomes


Rho p value


Hospital Standardised MortalityRatio

-0.20 0.01


Standardised morality rate for highrisk conditions

-0.22 0.01


Standardised morality rate amongsurgical inpatients with serioustreatable complications

-0.00 0.99


Standardised mortality rate fromlow mortality conditions

0.03 0.70


Emergency readmission rate within28 days

-0.31


17/28

Comparing cleanliness ratings with infection rates

Greaves et al, 2012 Arch Int Med


Rho p value

Patient

perception of cleanliness

Rate of MRSA bacteraemia

(per 1,000 bed days)

-0.30


18/28

Comparing cleanliness ratings with infection rates

Greaves et al, 2012 Arch Int Med


Rho p value

Patient

perception of cleanliness

Rate of MRSA bacteraemia

(per 1,000 bed days)

-0.30


19/28

A wisdom in the crowd of patients?


20/28

Comparison of the NHS Inpatient Survey and ratings on the NHSChoices website

NHS Inpatient Survey NHS Choices ratings

Mechanism Paper-based survey Ratings left on a website

Number ofresponses 69,000 per year

5,000 per year

Selection Random; patients receive asurvey requestingcompletion after leavinghospital

Self-selecting; patients arenot solicited

Proportionpositive

79% rated their overall careas excellent or very good

67% would recommend to afriend

Cost Likely more expensive Likely less expensive


21/28

Comparison of the NHS Inpatient Survey and ratings on the NHSChoices website

NHS Inpatient Survey NHS Choices ratings

Mechanism Paper-based survey Ratings left on a website

Number ofresponses 69,000 per year

5,000 per year

Selection Random; patients receive asurvey requestingcompletion after leavinghospital

Self-selecting; patients arenot solicited

Proportionpositive

79% rated their overall careas excellent or very good

67% would recommend to afriend

Cost Likely more expensive Likely less expensive

There were 10,000 hospital ratings in theUK over 2 years

Over the same time period there were29,118,009 hospital admissions

0.04% of hospital admissions are rated


22/28

The stereotypical reviewer?


23/28

Where are peoplerating their care?


24/28

Associations between whether a practice is ratedwith population and practice characteristics

Independentvariable

Z statistic p value

Practicepopulation size

15.38


25/28

The cloud of patient experience


26/28


27/28

Christopher Millett

Ara Darzi Dominic King Henry Lee Utz Pape Azeem Majeed Robert Wachter Daniel Cano-Ramirez

Thanks to my co-authors:


28/28

Thank you

[email protected]

Associations between online patient ratings and traditional quality measures in the English NHS

Documents

Transcript of Associations between online patient ratings and traditional quality measures in the English NHS