Overview of the Overview of the Pediatric Indicator ModulePediatric Indicator Module

Presenters: Kathryn McDonald and Sheryl Davies,Presenters: Kathryn McDonald and Sheryl Davies,Stanford UniversityStanford University

AHRQ QI User Meeting AHRQ QI User Meeting September 26-27, 2005September 26-27, 2005

AcknowledgementsAcknowledgements

Pediatric Module Development:• Kathryn McDonald, Stanford University• Patrick Romano, UC-Davis• Sheryl Davies, Stanford University• Amy Ku, Stanford University• Kavita Choudhry, Stanford University• Jeffrey Geppert, Battelle Health and Life Sciences• Corinna Haberland, Stanford University

Support for Quality Indicators II (Contract No. 290-04-0020):• Mamatha Pancholi, AHRQ Project Officer• Marybeth Farquhar, AHRQ • Mark Gritz and Jeffrey Geppert, Project Directors, Battelle

Health and Life Sciences

spinningwheelalpacas.com chkd.com/images/HospitalVisit.jpg

Children’s Hospitalizations, US 2000• 6.3 million • $46 billion• 36% of 1-17 yr olds in Children’s hospitals

Unique PopulationUnique Population

Dependent on adultsDependent on adults Constantly developingConstantly developing DemographicsDemographics Epidemiology Epidemiology Coding in pediatricsCoding in pediatrics

Simpson LA, al DDe. Measures of Children's Health Care Quality: Building towards Consensus. Manuscript in preparation: Simpson LA, al DDe. Measures of Children's Health Care Quality: Building towards Consensus. Manuscript in preparation: Background paper prepared for National Quality Forum; 2003 September 19.Background paper prepared for National Quality Forum; 2003 September 19.

Current Measurement StateCurrent Measurement State Simpson and colleagues searchSimpson and colleagues search

Simpson LA, et al. Measures of Children's Health Care Quality: Building towards Consensus. Manuscript in preparation: Background paper prepared for National Quality Forum; 2003 September 19.

Pediatric indicators

Inpatient

Small subset (~10) feasible with restricted data

Pediatric Applications of AHRQ Pediatric Applications of AHRQ QIsQIs

Miller et al., Sedman et al., NACHRI chart reviewsMiller et al., Sedman et al., NACHRI chart reviews

Lessons learnedLessons learned

Complications DO occur in childrenComplications DO occur in children Some complications clinically different Some complications clinically different Some indicators perform differently in kids or rare with current exclusionsSome indicators perform differently in kids or rare with current exclusions Death related PSIs seemed less useful as defined in kidsDeath related PSIs seemed less useful as defined in kids

Indicator Module Indicator Module DevelopmentDevelopment

Literature

Actual Use

Concept

SOURCES

Candidate Indicators

Evaluation

Selection

Framework for Assessing Framework for Assessing Pediatric Indicator ValidityPediatric Indicator Validity

Face validity/consensual validityFace validity/consensual validity– Does the indicator capture an aspect of quality that is important and subject to provider control?Does the indicator capture an aspect of quality that is important and subject to provider control?

PrecisionPrecision– Is there substantial “true” provider-level variation?Is there substantial “true” provider-level variation?

Minimum biasMinimum bias– Is it possible to account for differences in severity of illness that could potentially confound comparisons across Is it possible to account for differences in severity of illness that could potentially confound comparisons across

providers?providers?

Construct validityConstruct validity– Does the indicator identify quality of care problems that are flagged or suspected using other methods?Does the indicator identify quality of care problems that are flagged or suspected using other methods?

Fosters real quality improvementFosters real quality improvement– Is the indicator unlikely to be gamed or cause perverse incentives?Is the indicator unlikely to be gamed or cause perverse incentives?

Application/experienceApplication/experience– Is there reason to believe the indicator will be feasible and useful?Is there reason to believe the indicator will be feasible and useful?

Literature reviewLiterature review– To identify quality concepts and indicators To identify quality concepts and indicators – To determine previous work on indicator validityTo determine previous work on indicator validity

Hospital ICD-9-CM coding reviewHospital ICD-9-CM coding review– To ensure proper definition (correspondence between clinical concept and To ensure proper definition (correspondence between clinical concept and

coding practice)coding practice)

Clinical panel reviews Clinical panel reviews – To refine indicator definition and risk groupingsTo refine indicator definition and risk groupings– To establish face validity when minimal literature To establish face validity when minimal literature

Empirical analysesEmpirical analyses– To explore alternative definitionsTo explore alternative definitions– To assess nationwide rates, hospital variation, relationships among indicatorsTo assess nationwide rates, hospital variation, relationships among indicators– To develop appropriate methods to account for differences in underlying riskTo develop appropriate methods to account for differences in underlying risk

Indicator DevelopmentIndicator Development

Phased EvaluationPhased Evaluation

Phase IPhase I– Current AHRQ QIsCurrent AHRQ QIs

Eliminate QIs covering adult only chronic illnesses or those with Eliminate QIs covering adult only chronic illnesses or those with questionable validity for kidsquestionable validity for kids

Phase IIPhase II– Novel indicatorsNovel indicators

Require development or updatingRequire development or updating

Example Indicator EvaluationExample Indicator Evaluation

PANEL EVALUATION

FURTHEREMPIRICAL ANALYSES

REFINED DEF.

FURTHER REVIEW?

FINAL DEFINITION

INITIAL EMPRICAL ANALYSES

AND DEFINITION

LITERATURE REVIEW

USER DATA

Decubitus ulcerDecubitus ulcer

Patients with secondary dx 707.0 per 1000 patientsPatients with secondary dx 707.0 per 1000 patients

Exclude high risk patients: Transfers from long term care Exclude high risk patients: Transfers from long term care facility, paralysisfacility, paralysis

EXCLUDE SPINA BIFIDA PATIENTS

Literature Review and User Data

.

Initial Empirical ResultsInitial Empirical Results

Rates by age group and high risk groupsRates by age group and high risk groups– Higher rate in higher age groupsHigher rate in higher age groups– Ulcers occur more frequently in high risk groups but Ulcers occur more frequently in high risk groups but

some occur in traditionally low risksome occur in traditionally low risk– Lower rate in premature neonatesLower rate in premature neonates

Rates are provided without commentary to Rates are provided without commentary to panelists prior to conferencepanelists prior to conference

.

Medical/Surgical Panel Medical/Surgical Panel CompositionComposition

SpecialtySpecialty LocationLocation Pediatric Emergency MedicinePediatric Emergency Medicine Dallas, TXDallas, TX Thoracic Surgery, Congenital Heart SurgeryThoracic Surgery, Congenital Heart Surgery Washington, DCWashington, DC NeonatologyNeonatology Seattle, WASeattle, WA Neonatal & Pediatric NursingNeonatal & Pediatric Nursing San Francisco, CASan Francisco, CA Pediatric Surgery, Surgical Critical CarePediatric Surgery, Surgical Critical Care New Haven, CTNew Haven, CT Pediatric Critical CarePediatric Critical Care Louisville, KYLouisville, KY Pediatric Infectious DiseasePediatric Infectious Disease Augusta, GAAugusta, GA Pediatric General SurgeryPediatric General Surgery Nashville, TNNashville, TN PediatricsPediatrics Valhalla, NYValhalla, NY Pediatric Radiology, Diagnostic RadiologyPediatric Radiology, Diagnostic Radiology Seattle, WASeattle, WA Pediatric OncologyPediatric Oncology New York, NYNew York, NY HospitalistHospitalist Philadelphia, PAPhiladelphia, PA

.

Panel EvaluationPanel Evaluation

Expand population to INCLUDE high risk Expand population to INCLUDE high risk populationspopulations

Prefer stratification schemePrefer stratification scheme Skin breakdown in neonatesSkin breakdown in neonates

.

Post-Panel InvestigationPost-Panel Investigation

Empirical analysesEmpirical analyses– Examine rates of decubitus ulcer in potentially high Examine rates of decubitus ulcer in potentially high

risk groups.risk groups.– Identify similar risk strataIdentify similar risk strata

Coding consultCoding consult– Understand coding guidelines for infants with “skin Understand coding guidelines for infants with “skin

breakdown” or decubitibreakdown” or decubiti

.

Example EvaluationExample EvaluationRevised Definition for Decubitus UlcerRevised Definition for Decubitus Ulcer

Patients with secondary dx of 707x per 1000 patientsPatients with secondary dx of 707x per 1000 patients

Exclude patients transferred from long term care facility Exclude patients transferred from long term care facility and and another acute care facilityanother acute care facility

Stratify by:Stratify by:- Low RiskLow Risk- High risk (paralysis, spina bifida, anoxic brain damage)High risk (paralysis, spina bifida, anoxic brain damage)

.

ResultsResultsOverarching ThemesOverarching Themes

High risk populations are important in childrenHigh risk populations are important in children Bias and risk groupsBias and risk groups Expanded dataExpanded data Application of indicators keyApplication of indicators key Feedback and validity testing keyFeedback and validity testing key

Types of Modifications Types of Modifications Made to QIsMade to QIs

Expand population at riskExpand population at risk– Decubitus ulcer, postoperative sepsisDecubitus ulcer, postoperative sepsis

Restricted age rangeRestricted age range– Transfusion reaction, Diabetes, Asthma, Perforated appendixTransfusion reaction, Diabetes, Asthma, Perforated appendix– Exclusion of normal newbornsExclusion of normal newborns

Stratification/splitStratification/split– Iatrogenic pneumothorax, Accidental puncture laceration, Post-op hemorrhage/hematomaIatrogenic pneumothorax, Accidental puncture laceration, Post-op hemorrhage/hematoma

Added exclusion criteriaAdded exclusion criteria– Post-op wound dehiscence, Post-op respiratory failure, UTIPost-op wound dehiscence, Post-op respiratory failure, UTI

Modified numeratorModified numerator– GastroenteritisGastroenteritis

Indicators Not RecommendedIndicators Not Recommended

Clinically different in childrenClinically different in children Likely to occur in complex cases in children/ preventability Likely to occur in complex cases in children/ preventability

questionablequestionable Coding concernsCoding concerns

– Bacterial pneumoniaBacterial pneumonia– PO physiologic and metabolic derangementPO physiologic and metabolic derangement

Combined with other indicator, remaining cases not usefulCombined with other indicator, remaining cases not useful– DehydrationDehydration

Rates per 1000Rates per 1000Procedure-related ComplicationsProcedure-related Complications

0

2

4

6

8

10

12

14

16

Foreign body Iatro. Pneumothorax Accident. Punct/lac

all patients

neonates

non-neonates

No therapeutic

Minor therapeutic

Maj ther/no diag

Maj ther/min diag

Maj ther/maj diag

2 major ther

3+ major ther

Rates per 1000Rates per 1000Complications in All PatientsComplications in All Patients

0

5

10

15

20

25

30

Decubitus ulcer Transfusion rxn Selected infection (line)

High risk

Intermediate risk

Low risk

All patients

Rates per 1000Rates per 1000Postoperative ComplicationsPostoperative Complications

0

5

10

15

20

25

PO hemorrhage PO wound dehiscence PO respiratory failure

High riskIntermediate riskLow riskAll patients

Rates per 100,000 populationRates per 100,000 populationPotentially Avoidable HospitalizationsPotentially Avoidable Hospitalizations

0

20

40

60

80

100

120

140

160

180

Diabetes Asthma Gastro. UTI

0

5

10

15

20

25

30

35

Perforatedappendix**

Rates (%) Rates (%) Mortality IndicatorsMortality Indicators

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

Heart surgery Craniotomy

Dealing with BiasDealing with Bias

StratificationStratification– Clinically transparent, actual numbersClinically transparent, actual numbers– Low numbers, overwhelming number of resultsLow numbers, overwhelming number of results

Risk adjustmentRisk adjustment– Allows for comparisonsAllows for comparisons– Full adjustment impossible, black boxFull adjustment impossible, black box

ExclusionsExclusions– Easy comparisons, complex cases avoidedEasy comparisons, complex cases avoided– Low numbers, leaves out cases important to preventLow numbers, leaves out cases important to prevent

Risk AdjustmentRisk Adjustment

Reason for admission/ type of procedureReason for admission/ type of procedure– DRGsDRGs

ComorbidityComorbidity– Must develop de novoMust develop de novo

SES risk adjustment SES risk adjustment – Not unique to kids, but may over-adjustNot unique to kids, but may over-adjust

Phase II: Novel IndicatorsPhase II: Novel Indicators

Literature reviewLiterature review Organization contactOrganization contact

– Federal agencies, professional organizations, Federal agencies, professional organizations, advocacy groups, provider organizationsadvocacy groups, provider organizations

– 100+ contacted100+ contacted– Most indicators submitted not feasible given data Most indicators submitted not feasible given data

constraintsconstraints

Indicators Under ConsiderationIndicators Under Consideration

Ambulatory CareAmbulatory Care

Cellulitis hospitalization rate Cellulitis hospitalization rate Hospital admissions for influenza-related Hospital admissions for influenza-related

conditions, age 6-23 months conditions, age 6-23 months Immunizable condition hospitalization rateImmunizable condition hospitalization rate

Indicators Under ConsiderationIndicators Under Consideration

NeonatalNeonatal

Intraventricular hemorrhageIntraventricular hemorrhage Respiratory distress syndromeRespiratory distress syndrome Chronic respiratory diseaseChronic respiratory disease Meconium aspiration syndrome rate Meconium aspiration syndrome rate Nectrotizing enterocolitisNectrotizing enterocolitis Neonatal mortality Neonatal mortality Nosocomial bacteremia Nosocomial bacteremia Proportion of VLBW infants born at Level III Proportion of VLBW infants born at Level III

centers centers Retinopathy of prematurity Retinopathy of prematurity

Indicators Under ConsiderationIndicators Under Consideration

Patient Safety and MortalityPatient Safety and Mortality

Aspiration pneumonia Aspiration pneumonia Postoperative pneumonia Postoperative pneumonia Catheter-associated venous thrombosisCatheter-associated venous thrombosis Other postoperative metabolic derangements Other postoperative metabolic derangements

(hyponatremia, hypernatremia) (hyponatremia, hypernatremia) Trauma mortality Trauma mortality

Phase II: Next StepsPhase II: Next Steps

Literature reviewsLiterature reviews Update existing definitionsUpdate existing definitions Develop and test definitions using administrative dataDevelop and test definitions using administrative data Panel reviewPanel review Reformulation of indicatorsReformulation of indicators Development and release of new softwareDevelopment and release of new software

TimelineTimeline

January 2006January 2006– PedQI software release with current AHRQ QIs adapted PedQI software release with current AHRQ QIs adapted

for pediatric casesfor pediatric cases

Fall/Winter 2005Fall/Winter 2005– PQI, IQI, PSI updates converted to adult population focusPQI, IQI, PSI updates converted to adult population focus

Early 2007Early 2007– PedQI update with new indicatorsPedQI update with new indicators

ImplicationsImplications

AHRQ PSIs, IQIs and PQIs AHRQ PSIs, IQIs and PQIs – No longer apply to children, though concepts retained in PedQINo longer apply to children, though concepts retained in PedQI

Children’s vs. community hospitalsChildren’s vs. community hospitals– Focus on strata for stratified indicatorsFocus on strata for stratified indicators– Compare results within peer groupsCompare results within peer groups

Request to usersRequest to users– Monitoring of coding practices essentialMonitoring of coding practices essential– Communication to AHRQ about early experiencesCommunication to AHRQ about early experiences

AcknowledgmentsAcknowledgments

Funded by AHRQFunded by AHRQ Support for Quality Indicators II (Contract No. 290-04-0020) Mamatha Pancholi, AHRQ Project Officer Marybeth Farquhar, AHRQ QI Senior Advisor Mark Gritz and Jeffrey Geppert, Project Directors, Battelle Health

and Life Sciences

Data used for analyses:Data used for analyses:Nationwide Inpatient Sample (NIS), 1995-2000. Healthcare Cost and Nationwide Inpatient Sample (NIS), 1995-2000. Healthcare Cost and

Utilization Project (HCUP), Agency for Healthcare Research and Utilization Project (HCUP), Agency for Healthcare Research and QualityQuality

State Inpatient Databases (SID), 1997-2002 (36 states). Healthcare State Inpatient Databases (SID), 1997-2002 (36 states). Healthcare Cost and Utilization Project (HCUP), Agency for Healthcare Cost and Utilization Project (HCUP), Agency for Healthcare Research and QualityResearch and Quality

AcknowledgementsAcknowledgements

We gratefully acknowledge the data organizations in participating states that contributed data to We gratefully acknowledge the data organizations in participating states that contributed data to HCUP and that we used in this study: the Arizona Department of Health Services; California Office HCUP and that we used in this study: the Arizona Department of Health Services; California Office of Statewide Health Planning & Development; Colorado Health & Hospital Association; Connecticut of Statewide Health Planning & Development; Colorado Health & Hospital Association; Connecticut - Chime, Inc.; Florida Agency for Health Care Administration; Georgia: An Association of Hospitals & - Chime, Inc.; Florida Agency for Health Care Administration; Georgia: An Association of Hospitals & Health Systems; Hawaii Health Information Corporation; Illinois Health Care Cost Containment Health Systems; Hawaii Health Information Corporation; Illinois Health Care Cost Containment Council; Iowa Hospital Association; Kansas Hospital Association; Kentucky Department for Public Council; Iowa Hospital Association; Kansas Hospital Association; Kentucky Department for Public Health; Maine Health Data Organization; Maryland Health Services Cost Review; Massachusetts Health; Maine Health Data Organization; Maryland Health Services Cost Review; Massachusetts Division of Health Care Finance and Policy; Michigan Health & Hospital Association; Minnesota Division of Health Care Finance and Policy; Michigan Health & Hospital Association; Minnesota Hospital Association; Missouri Hospital Industry Data Institute; Nebraska Hospital Association; Hospital Association; Missouri Hospital Industry Data Institute; Nebraska Hospital Association; Nevada Department of Human Resources; New Jersey Department of Health & Senior Services; Nevada Department of Human Resources; New Jersey Department of Health & Senior Services; New York State Department of Health; North Carolina Department of Health and Human Services; New York State Department of Health; North Carolina Department of Health and Human Services; Ohio Hospital Association; Oregon Association of Hospitals & Health Systems; Pennsylvania Health Ohio Hospital Association; Oregon Association of Hospitals & Health Systems; Pennsylvania Health Care Cost Containment Council; Rhode Island Department of Health; South Carolina State Budget Care Cost Containment Council; Rhode Island Department of Health; South Carolina State Budget & Control Board; South Dakota Association of Healthcare Organizations; Tennessee Hospital & Control Board; South Dakota Association of Healthcare Organizations; Tennessee Hospital Association; Texas Health Care Information Council; Utah Department of Health; Vermont Association; Texas Health Care Information Council; Utah Department of Health; Vermont Association of Hospitals and Health Systems; Virginia Health Information; Washington State Association of Hospitals and Health Systems; Virginia Health Information; Washington State Department of Health; West Virginia Health Care Authority; Wisconsin Department of Health & Department of Health; West Virginia Health Care Authority; Wisconsin Department of Health & Family Services. Family Services.

Questions?Questions?