PRECOG: Developing a practical, PRECOG: Developing a practical, evidence-based approach to evidence-based approach to

assessing cataract surgical outcomesassessing cataract surgical outcomes

Nathan Congdon, MD, MPHNathan Congdon, MD, MPH

Zhongshan Ophthalmic Zhongshan Ophthalmic Center, Preventive Center, Preventive

Ophthalmology Unit, Ophthalmology Unit, Guangzhou, ChinaGuangzhou, China

ORBIS InternationalORBIS International

Financial interest

No financial interest

The problem of un-operated The problem of un-operated cataractcataract

The key to solving this The key to solving this problem, still the problem, still the world’s leading cause world’s leading cause of blindness, is training of blindness, is training additional surgeonsadditional surgeons

The critical issue is The critical issue is outcome quality, for outcome quality, for which the WHO has which the WHO has set standards:set standards:– Presenting acuity >= Presenting acuity >=

6/18 in 80% of post-6/18 in 80% of post-operative patientsoperative patients

Barriers to assessment of cataract Barriers to assessment of cataract outcomesoutcomes

The proportion of The proportion of patients returning after patients returning after surgery is often very surgery is often very small in many parts of small in many parts of the developing world.the developing world.

It is un-known whether It is un-known whether vision outcomes among vision outcomes among patients who do present patients who do present for follow-up for follow-up spontaneously are spontaneously are representative of all representative of all persons undergoing persons undergoing operations.operations.

A new approach to outcomes A new approach to outcomes assessment?assessment?

Wide adoption of small-Wide adoption of small-incision, sutureless incision, sutureless surgery mean more rapid surgery mean more rapid recovery of vision post-recovery of vision post-operativelyoperatively

Many surgical facilities, Many surgical facilities, especially in rural areas, especially in rural areas, admit patients for 1-3 admit patients for 1-3 days after surgerydays after surgery

Can the principal Can the principal assessment of post-assessment of post-operative vision be operative vision be carried out at time of carried out at time of hospital discharge?hospital discharge?

Advantages of early outcomes Advantages of early outcomes assessmentassessment

Collect data on all Collect data on all patients readilypatients readily

Avoid bias in data Avoid bias in data collectioncollection

Reduce costs to patients Reduce costs to patients and hospitals for follow-and hospitals for follow-upup

PRECOG: PRECOG: PProspective rospective RReview of eview of EEarly arly CCataract ataract OOutcomes and utcomes and GGradingrading

Objectives:Objectives:

Early assessment:Early assessment: Assess validity of visual acuity measured at hospital Assess validity of visual acuity measured at hospital

discharge after cataract surgery as a predictors of discharge after cataract surgery as a predictors of medium-term (>= 50 days) vision (“Study hypothesis”) medium-term (>= 50 days) vision (“Study hypothesis”)

Better use of existing dataBetter use of existing data Assess extent to which vision of persons spontaneously Assess extent to which vision of persons spontaneously

returning for follow-up care >= 50 days after cataract returning for follow-up care >= 50 days after cataract surgery are predictive of VA for entire operated cohort surgery are predictive of VA for entire operated cohort (“Traditional approach”)(“Traditional approach”)

PRECOG: SettingPRECOG: Setting Urban and rural facilities providing cataract Urban and rural facilities providing cataract

surgery (n = 41):surgery (n = 41):– East Asia:East Asia:

• China (18)China (18)• Vietnam (4)Vietnam (4)• Indonesia (2)Indonesia (2)

– India:India:• All Aravind centers (5)All Aravind centers (5)

– Latin America:Latin America:• Peru (2), Ecuador (1), Paraguay (1), Guatemala (1), Mexico (2)Peru (2), Ecuador (1), Paraguay (1), Guatemala (1), Mexico (2)

– Africa:Africa:• Eritrea (2)Eritrea (2)• Ethiopia (3)Ethiopia (3)

PRECOG: Participants and Sample SizePRECOG: Participants and Sample Size

50-100 consecutive 50-100 consecutive persons aged > 30 years persons aged > 30 years and under-going surgery and under-going surgery for age-related cataract for age-related cataract at each participating at each participating facilityfacility

Exclusion criteria:Exclusion criteria:– Traumatic cataractTraumatic cataract– Ocular co-morbidities Ocular co-morbidities

including glaucoma, including glaucoma, retinal disease, corneal retinal disease, corneal abnormalities or uveitis.abnormalities or uveitis.

PRECOG: Follow-upPRECOG: Follow-up

Target of > 90% follow-up at >= 50 days post Target of > 90% follow-up at >= 50 days post op, either through:op, either through:

– Spontaneous return to clinicSpontaneous return to clinic– Return to clinic potentiated by special intervention Return to clinic potentiated by special intervention

(phone call, offer of free transport, etc.)(phone call, offer of free transport, etc.)– Home visitHome visit

Type of follow-up recordedType of follow-up recorded, so that patients , so that patients returning spontaneously, under usual returning spontaneously, under usual conditions (WITHOUT phone call, home visit conditions (WITHOUT phone call, home visit etc.) can be studiedetc.) can be studied

PRECOG Results: ParticipantsPRECOG Results: Participants

Hospitals (n = 41)Hospitals (n = 41)– Annual surgical output: Range from < 500 Annual surgical output: Range from < 500

(several) to 91,759 (Aravind Madurai)(several) to 91,759 (Aravind Madurai)– Public: 31/41 (75.6%)Public: 31/41 (75.6%)– Rural: 24/44 (58.5%)Rural: 24/44 (58.5%)

CasesCases– A total of 3547, of which:A total of 3547, of which:

• 2246 (63%) SICS2246 (63%) SICS• 776 (22%) phaco776 (22%) phaco• Remainder ECCE (15%)Remainder ECCE (15%)

PRECOG Results: SurgeryPRECOG Results: Surgery

Pre-op VA <= 6/60 in operated eye: 84.6%Pre-op VA <= 6/60 in operated eye: 84.6%

Final (>= 50 days) uncorrected VA Final (>= 50 days) uncorrected VA • >= 6/18: 2089 (63.7%) >= 6/18: 2089 (63.7%) • <= 6/60: 338 (10.3%) <= 6/60: 338 (10.3%)

Complications:Complications:• Intra-op: 7.79% Intra-op: 7.79% • Post-op: 1.99%Post-op: 1.99%

PRECOG Results: Follow-upPRECOG Results: Follow-up

The proportion of subjects with follow-up vision The proportion of subjects with follow-up vision measured at >= 50 days after surgery was 3178/3547 measured at >= 50 days after surgery was 3178/3547 (92.5%)(92.5%)

By region, follow-up was:By region, follow-up was:– China 89.8%China 89.8%– India 93.6%India 93.6%– Vietnam/Indonesia 90.1%Vietnam/Indonesia 90.1%– Latin America 98.3%Latin America 98.3%– Africa 95.6%Africa 95.6%

Spontaneous follow-up at clinic: Spontaneous follow-up at clinic: 43%43% (Range from (Range from China 26% to Latin America 80%)China 26% to Latin America 80%)

Correlation of early vision with final Correlation of early vision with final visionvision

What we want to know: How do hospitals What we want to know: How do hospitals rank according to final VA outcome? rank according to final VA outcome? (proportion with VA >= 6/18)(proportion with VA >= 6/18)

We can compare two strategies to estimate We can compare two strategies to estimate this:this:

– Using discharge vision to rank hospitals (the goal Using discharge vision to rank hospitals (the goal of PRECOG)of PRECOG)

– Using the final vision Using the final vision among those patients who do among those patients who do return spontaneouslyreturn spontaneously (what we have traditionally (what we have traditionally done)done)

Discharge VA for all patients

Final VA for 40% of patients who DO return spontaneously to clinic

Final VA for ALL patients

The method we are testing in PRECOG

The method we have traditionally used

What we are trying to estimate

Correlation of early vision with final Correlation of early vision with final visionvision

Discharge vision and final vision are highly Discharge vision and final vision are highly correlated for all patients: Spearman r = 0.59correlated for all patients: Spearman r = 0.59

Hospital rankings using uncorrected Hospital rankings using uncorrected discharge vision appear discharge vision appear better-correlatedbetter-correlated with with rankings using final vision than are rankings rankings using final vision than are rankings using the 43% of patients who return using the 43% of patients who return spontaneously:spontaneously:

– Spearman r = 0.50 for discharge visionSpearman r = 0.50 for discharge vision– Spearman r = 0.28 for patients who return Spearman r = 0.28 for patients who return

spontaneouslyspontaneously

Can we do even better?Can we do even better?

Using best-corrected vision Using best-corrected vision does notdoes not improve improve the performance of discharge VA in the performance of discharge VA in predicting hospital rankings based on final predicting hospital rankings based on final VA (r = 0.45)VA (r = 0.45)

Dropping patients (15%) with ECCE has Dropping patients (15%) with ECCE has little little impactimpact on performance of discharge VA (r = on performance of discharge VA (r = 0.56)0.56)

Can we do even better?Can we do even better?

When we measure discharge vision as an index When we measure discharge vision as an index of outcome, there are inevitably some patients of outcome, there are inevitably some patients with temporary poor VA due to corneal edema with temporary poor VA due to corneal edema or other problemsor other problems

What if we could improve performance of poor What if we could improve performance of poor vision by dropping these patients?vision by dropping these patients?

When we drop the 20% of patients at each When we drop the 20% of patients at each hospital with the worst vision, discharge vision hospital with the worst vision, discharge vision is is better-correlatedbetter-correlated with final VA: with final VA: r = 0.67 r = 0.67

Concrete example Concrete example using PRECOG data*using PRECOG data*

As a program planner in MOH or NGO, you As a program planner in MOH or NGO, you want to separate hospitals into three want to separate hospitals into three categories:categories:

– Good (Top 25%: Can provide training to others)Good (Top 25%: Can provide training to others)– Medium (Middle 50%: No intervention needed)Medium (Middle 50%: No intervention needed)– Problem (Bottom 25%: Further training needed)Problem (Bottom 25%: Further training needed)

How well does early vision assessment work How well does early vision assessment work for this? for this?

*Omitting data from 3 hospitals in Ethiopia for *Omitting data from 3 hospitals in Ethiopia for whom data not yet cleanedwhom data not yet cleaned

Concrete example: Uncorrected VA, Concrete example: Uncorrected VA, drop worst 20% by visiondrop worst 20% by vision

Ranking on Uncorrected Discharge VA

Ranking Based on Uncorrected Final

VA Good

Medium

Problem

TOTAL

Good 6 3 0 9

Medium

3 14 3 20

Problem

0 3 6 9

TOTAL

9 20 9 38

•26/38 hospitals (68%) have the same ranking using discharge VA that they would have had using final VA

No hospitals went from Good to Poor or Poor to Good

Concrete exampleConcrete example

68% (26/38) of hospitals had the identical 68% (26/38) of hospitals had the identical ranking based on discharge and final visionranking based on discharge and final vision

If the vision of patients returning If the vision of patients returning spontaneously was used to rank hospitals, only spontaneously was used to rank hospitals, only 18/38 (47%) had the same ranking18/38 (47%) had the same ranking

Based on chance alone, two such ranking Based on chance alone, two such ranking systems would be expected to agree on 13/38 systems would be expected to agree on 13/38 (34%) of hospitals(34%) of hospitals

PRECOG Results: Standards for PRECOG Results: Standards for Early Vision AssessmentEarly Vision Assessment

If discharge vision will be used as an index for If discharge vision will be used as an index for surgical quality, the current WHO standard of surgical quality, the current WHO standard of 80% of patients with uncorrected VA >= 6/18 will 80% of patients with uncorrected VA >= 6/18 will likely need to changelikely need to change

In PRECOG, hospitals achieved the following In PRECOG, hospitals achieved the following standards for the % of patients with uncorrected standards for the % of patients with uncorrected discharge VA >= 6/18:discharge VA >= 6/18:

– 9090thth percentile: 71.8% percentile: 71.8% – 7575thth percentile: 60.6% percentile: 60.6%– 5050thth percentile: 45.3% percentile: 45.3%– 2525thth percentile: 31.1% percentile: 31.1%

CAVEATSCAVEATS

Though hospitals in PRECOG included rural Though hospitals in PRECOG included rural and urban, government and private facilities and urban, government and private facilities from many regions:from many regions:

– They were not chosen at randomThey were not chosen at random– We don’t know if they are truly representative of We don’t know if they are truly representative of

all facilitiesall facilities

Patients were chosen at random (consecutive Patients were chosen at random (consecutive surgeries), and follow-up was very good, but surgeries), and follow-up was very good, but not 100%not 100%

– Room for biasRoom for bias

PRECOG: SummaryPRECOG: Summary

If hospitals can measure discharge vision on 50-100 If hospitals can measure discharge vision on 50-100 consecutive patients, they can provide a robust index of consecutive patients, they can provide a robust index of cataract surgical outcome usable by themselves and cataract surgical outcome usable by themselves and program plannersprogram planners

No need to be able to refract (using BCVA does not No need to be able to refract (using BCVA does not improve accuracy of data)improve accuracy of data)

Works for hospitals performing ECCE as well as small Works for hospitals performing ECCE as well as small incision casesincision cases

Even small, rural hospitals throughout the world have Even small, rural hospitals throughout the world have now proven their ability to collect these datanow proven their ability to collect these data

PRECOG: Next stepsPRECOG: Next steps

Analyze other data we collected Analyze other data we collected to further guide optimal follow-to further guide optimal follow-up:up:

– Prevalence of refractive error Prevalence of refractive error and other conditions requiring and other conditions requiring treatment: treatment: how useful is how useful is follow-up?follow-up?

– Cost to patients and hospitals: Cost to patients and hospitals: how cost-effective is follow-up?how cost-effective is follow-up?

Look at simple adjustments to Look at simple adjustments to improve accuracy of discharge improve accuracy of discharge vision even furthervision even further

Work with WHO, IAPB, NGOs Work with WHO, IAPB, NGOs and governments to and governments to disseminate and begin using disseminate and begin using these results to evaluate these results to evaluate surgical quality in practicesurgical quality in practice