© Copyright 2009 by the American Association for Clinical Chemistry

Glucose Meter Performance Criteria for Tight Glycemic Control Estimated by Simulation Modeling

B.S. Karon, J.C. Boyd, and G.G. Klee

July 2010

http://www.clinchem.org/cgi/reprint/56/7/1091

© Copyright 2010 by the American Association for Clinical Chemistry

Journal ClubJournal Club

© Copyright 2009 by the American Association for Clinical Chemistry

IntroductionIntroduction

Current Glucose Meter Performance Criteria

Developed to determine accuracy needs required to support subcutaneous insulin dosing

Most often displayed as error grid

Error Grid Translates Accuracy into Clinical Implications for Subcutaneous Dosing

© Copyright 2009 by the American Association for Clinical Chemistry

Introduction (cont)Introduction (cont)Error Grid Example

© Copyright 2009 by the American Association for Clinical Chemistry

Introduction (cont)Introduction (cont)

Advantage of Error Grid Analysis

Translates error into clinical impact

Visual display of current guidelines (ISO 15197)

± 15 mg/dL at glucose < 75 mg/dL

± 20% at glucose ≥ 75 mg/dL

© Copyright 2009 by the American Association for Clinical Chemistry

Introduction (cont)Introduction (cont)

Limitations of Error Grid Analysis

Only meaningful for subcutaneous insulin dosing

Every meter looks good (majority of results fall into A and B)

© Copyright 2009 by the American Association for Clinical Chemistry

QuestionQuestion How do you determine performance criteria

necessary for tight glycemic control (TGC)?

Scott et al.1 have reviewed the controversy in this area

Outcome studies ideal, but resource intensive

Several comparison studies done, conflicting results

Fundamental problem, no agreement on how to establish accuracy needs for TGC

Error simulation modeling has been used to establishperformance criteria for subcutaneous insulin dosing

1Clin Chem 2009;55:18–20.

© Copyright 2009 by the American Association for Clinical Chemistry

MethodsMethods

0100020003000400050006000700080009000

10000

< 80 80-90 91-110 111-130131-150151-175176-200201-250251-300301-350351-400 > 400

0 0 to 2 1 to 3 2 to 4 3 to 6 4 to 10 6 to 12 10 to 16 12 to 20 14 to 24 16 to 27 18 to 30

Fre

qu

en

cy

Glucose range (insulin dose)

Histogram of 29,920 glucose values for patients on intravenous insulinMedian value = 116 mg/dL (IQR 102-135)

Start with distribution of observed glucose values during TGC for 2 surgical ICU units during a 6-month period; 86% values ≤ 150 mg/dL, prevent hypoglycemia.

© Copyright 2009 by the American Association for Clinical Chemistry

Methods (cont)CategoryCategory Glucose value (mg/dL)Glucose value (mg/dL) Insulin dose (Units/hour)Insulin dose (Units/hour)

0 < 80 0

1 80-90 0-2

2 91-110 1-3

3 111-130 2-4

4 131-150 3-6

5 151-175 4-10

6 176-200 6-12

7 201-250 10-16

8 251-300 12-20

9 301-350 14-24

10 351-400 16-27

11 > 400 18-30

Glucose value determines insulin dose category. Example of 3 category dosing error is real value in category 0, simulated value in category 3.

© Copyright 2009 by the American Association for Clinical Chemistry

Methods (cont)Methods (cont) Sample this distribution. For each initial value

sampled, simulate 10,000 values with distribution of bias and imprecision:

Glucose (simulated) = Glucose initial + [n(0,1) x CV x glucose(initial)] + [Bias x glucose (initial)

n(0,1) random number drawn from gaussian distributioncentered on zero with SD = 1

Bias (expressed as a fraction of the glucose concentration) isvaried from -0.2 to +0.2 in 0.01 increments

For each Bias, CV (also expressed as a fraction of glucoseconcentration) is varied from 0.00 to 0.20 in 0.01 increments

© Copyright 2009 by the American Association for Clinical Chemistry

Methods (cont)Methods (cont)

Calculate percentage simulated values that fall in same insulin dosing category as initial

Calculate percentage 1, 2, or ≥ 3 category dosing errors based on Mayo TGC protocol

Express results as contour plots, showing percentage dosing errors as a function of bias and imprecision

© Copyright 2009 by the American Association for Clinical Chemistry

Methods (cont)Methods (cont)

Superimpose boundaries for 10%, 15%, and 20% total error (TEa) on contour plots

Acceptable performance defined as ≥ 3 category insulin dosing errors occurring < 0.2% of the timeBased upon impact of 3 category error on insulin dose given

distribution of glucose values and ability to predict percent errors

with reasonable confidence interval

© Copyright 2009 by the American Association for Clinical Chemistry

ResultsResults

© Copyright 2009 by the American Association for Clinical Chemistry

Results (cont)Results (cont)

Up to 60% one category dosing errors when 10% TEa is simulated

Up to 80% one category dosing errors when 15% TEa is simulated

Up to 90% one category dosing errors when 20% TEa is simulated

© Copyright 2009 by the American Association for Clinical Chemistry

ResultsResults

© Copyright 2009 by the American Association for Clinical Chemistry

Results (cont)Results (cont)

Only 0.2% two category dosing errors when 10% TEa is simulated

Up to 5% two category dosing errors when 15% TEa is simulated

Up to 20% two category dosing errors when 20% TEa is simulated

© Copyright 2009 by the American Association for Clinical Chemistry

ResultsResults

Only the 20% TEa Only the 20% TEa condition was condition was associated with any associated with any frequency of 3 or frequency of 3 or more category more category insulin dosing insulin dosing errorserrors

© Copyright 2009 by the American Association for Clinical Chemistry

Methods Second simulation model based on gaussian

distribution of simulated results around each of 29,920 initial values

80 85 90 95 100 105 110 115 120

Generate 1000 simulated values with distribution of X% error using SAS (Carey, NC)

Determine how many simulated values would change insulin dosing category relative to original value

© Copyright 2009 by the American Association for Clinical Chemistry

Methods (cont)Methods (cont)

Error conditionError condition 10 % error10 % error 15 % error15 % error 20 % error20 % error

No change dose 71.4 % 58.7 % 48.8 %

1 category dose 28.4 % 39.3 % 44.8 %

2 category dose 0.2 % 2.0 % 6.1 %

≥ 3 category dose 0.0 % 0.02 % 0.3 %

© Copyright 2009 by the American Association for Clinical Chemistry

Methods (cont)Methods (cont)

Decreasing acceptable error tolerance from 20% to 10% will decrease 2 category errorsAdditional studies necessary to understand

impact of 2 category dosing errors

Only 20% TEa criteria allowed 3 category dosing errors Most dangerous as could lead to hypoglycemia

© Copyright 2009 by the American Association for Clinical Chemistry

ConclusionsConclusions

Only 20% TEa condition allowed 3 category or critical errors in either modelImprecision drives 3 category dosing errors

So far models predict that meters that maintain 15% TEa and minimize imprecision may be safe and effective for TGC monitoring

© Copyright 2009 by the American Association for Clinical Chemistry

Conclusions (cont)Conclusions (cont)

Models do not account for random patient interferences (such as effects of variation due to pO2, hematocrit, or sampling), which would increase the total error of glucose measurements, leading to the need for more stringent performance criteria

Models assume that a single dosing error leads to patient harmMore complex models needed to understand cumulative

dosing errors