Evaluating information quality in the detection of pediatric asthma encounters

ANDREW J KNIGHTON, PHD, CPAINSTITUTE FOR HEALTH CARE DELIVERY RESEARCHINTERMOUNTAIN HEALTHCARE

Problem

• Quality is an attribute of data• Meaningful variation in health data quality• Variation in type and scope• Eliminating errors entirely is not feasible• Impact of data quality on decision quality varies

Defining terms

Data:4.21 liters

Information:Mr. X has an FVC of

4.21 liters on 1/1/2015

Knowledge:Mr. X’s falling FVC levels indicate a

problem

Increasing interactions and interrelationships

Incr

easi

ng

com

ple

xit

y

Blum (1986); Nelson and Joos (1989)

The study

• A retrospective cohort design was used to study patterns of asthma readmission at a large urban pediatric hospital using administrative data (Knighton, 2013)

• Variation in defining asthma encounters in the literature and in practice

• Given the pairing of asthma with certain respiratory conditions, we developed an asthma encounter detection algorithm using two criteria (Knighton, 2014):

• Approach: We compared the results of the algorithm against a “gold standard” criteria for asthma encounter detection through medical record review (n=110)

What we found• Improved

detection accuracy from 0.55 [95% CI: 0.46-.064] using primary dx only to 0.83 [0.76-0.90]

• Increased asthma encounters 55%

• Increased bed days 64%

• Increased estimate of the economic impact of asthma 71% from $8.6M to $14.8M

Knighton AJ, et al, Population Health Management, 2014

Figure 1. Receiver operating characteristic (ROC) curve by asthma ICD-9 diagnostic code position (1-5) for a sample of select respiratory encounters (n=110), including 95% CI for both sensitivity and false positive rate.

Implications

Knighton AJ, et al, Population Health Management, 2014

Figure 1. Receiver operating characteristic (ROC) curve by asthma ICD-9 diagnostic code position (1-5) for a sample of select respiratory encounters (n=110), including 95% CI for both sensitivity and false positive rate.

A

Specificity Sensitivity

B C

Case Detection Methods

Particular use

Required evidence quality

Selecting a case detection method

Case detection method

Linking particular use with case detection

Particular Use ExampleNature of Decision

Required Evidence QualityCase

Detection MethodSensitivity

False Positive

RateAccuracy

Clinical trial research

Identify intervention cases/controls

Patient-level Low to High NoneLow to Medium

A

Observation/ evaluation studies

Retrospective health services research study

Population-level

Medium to High

Low to Medium

Medium to High

A,B

Quality improvement analyses

Improve health services process for asthma care

Process-levelMedium to High

Low to Medium

Medium to High

B,C

Cost-effectiveness analyses

Impact of community intervention

Population-level

HighLow to Medium

Medium to High

B,C

Conclusions

• Multiple documented methods exist today for detecting asthma encounters

• Using ROC analysis to define evidence quality, we can reasonably link particular use categories with appropriate case detection methods

• A terminology familiar to producers and consumers of health data• Categorizing particular use could allow for improved standardization of

measurement• Potential for broad application to other health conditions and particular

uses

References

Blum B. Clinical Information Systems – A Review. West J Med. 1986; 145(12): 791-797.

Knighton AJ, Flood A, Speedie SM, Harmon B, Smith P, Crosby C, et al. Does initial length of stay impact 30-day readmission risk in pediatric asthma patients? J Asthma. 2013; (6): 1–21.

Knighton AJ, Flood A, Harmon B, Smith P, Crosby C, Payne NR. A novel method for detecting inpatient pediatric encounters using administrative data. Population Health Management. August 2014, 17(4): 239-246.

Nelson R, Joos I. On language in nursing: from data to wisdom. PLN Vision. 1989, p6.

Wang R, Strong D. Beyond accuracy: what data quality means to data consumers. Journal of Management Information Systems. 1996; 12(4); 5-34.