Original Research—General Otolaryngology

National Operative Case Log GrowthCharts in Otolaryngology–Head and NeckSurgery Training

Otolaryngology–Head and Neck Surgery2015, Vol. 152(1) 73–79� American Academy ofOtolaryngology—Head and NeckSurgery Foundation 2014Reprints and permission:sagepub.com/journalsPermissions.navDOI: 10.1177/0194599814552400http://otojournal.org

Noel Jabbour, MD1, and Terance Tsue, MD2

Sponsorships or competing interests that may be relevant to content are dis-

closed at the end of this article.

Abstract

Objective. To report national standard case log growthcurves for operative procedures in otolaryngology andto describe a method by which program directors canchart surgical case numbers over resident training tolongitudinally assess sufficiency of cases and paritybetween residents.

Study Design. Data visualization and analysis.

Setting. American Council for Graduate Medical Education(ACGME) national case log data for otolaryngologyresidency.

Subjects. National data set; no individual subjects.

Methods. National statistical case log reports for otolaryngologywere obtained from the ACGME for each postgraduate year(PGY) level in 2009, 2010, and 2011. Estimated means and stan-dard deviations were calculated. The mean and increments ofstandard deviation were graphed against time to create case loggrowth charts, similar to pediatric growth charts.

Results. Case log growth charts were made for eachACGME Otolaryngology Residency Review Committee keyindicator procedure. Progress of an individual resident or ofa cohort of residents may be graphed against this growthchart background over their training time.

Conclusions. National operative case log growth charts allowresidents and program directors to graphically assess prog-ress in obtaining a sufficient variety and number of operativeprocedures over time throughout training. This can provideearly identification when residents begin to fall below thegrowth curve during training.

Keywords

operative log, case log, surgical education, surgical compe-tency, developmental growth charts

Received November 1, 2012; revised August 28, 2014; accepted

August 28, 2014.

There are many parallels between the development of

a surgical trainee and the growth of a child. So, too,

are there similarities between the methods used to

assess the progression of resident development and of child

development.

The program director’s semiannual review is the residency

equivalent to a well-child check. At a routine well-child visit,

a child’s neurocognitive, communication, and psychomotor

development is compared with that of his or her age-matched

peers. The counterpart to this in resident education is clinical

competency assessments, faculty evaluations of clinical per-

formance, and now, at a national level, the development of

the Accreditation Council for Graduate Medical Education

(ACGME) Milestones.1

In addition, at a well-child visit, a child’s anthropometric

measurements—height, weight, and head circumference—

are compared with those of his or her peers. This is made

possible by normative growth charts, which were developed

in the 1940s.2

Plotting a child’s growth against these standards has become

a routine method to identify deviations from normal growth so

that they may be addressed at the earliest possible time.

In surgical training, the operative case log is the best cur-

rent anthropometric measurement. However, there is no ana-

logue to the pediatric growth chart. Currently, operative

cases are most often viewed in isolation or at best with com-

parison to graduate resident median and mean numbers.

This method provides little insight into the adequacy of

training at various time points in the process and can lead to

late identification of training inadequacy.

Our goal is to develop and report national standard case

log growth curves for operative procedures in otolaryngol-

ogy and to describe a method by which program directors

1Children’s Hospital of Pittsburgh of UPMC, Department of Otolaryngo-

logy, University of Pittsburgh, Pittsburgh, Pennsylvania, USA2Department of Otolaryngology, University of Kansas Medical Center,

Kansas City, Kansas, USA

This article was presented at the 2012 AAO-HNSF Annual Meeting & OTO

EXPO; September 9-12, 2012; Washington, DC.

Corresponding Author:

Noel Jabbour, MD, Children’s Hospital of Pittsburgh of UPMC, Department

of Otolaryngology, University of Pittsburgh, 4401 Penn Avenue, Faculty

Pavilion, 7th Floor, Pittsburgh, PA 15224, USA.

Email: JabbourN@chp.edu

may use these curves to assess resident progress throughout

training.

Methods

Final mean operative case log numbers for ‘‘resident sur-

geon’’ classification, including standard deviation for 2011

otolaryngology graduates, were obtained from the ACGME.

In addition, median case numbers for each procedure cate-

gory and each key indicator procedure in otolaryngology

were obtained at each postgraduate year (PGY) level for the

academic years 2008-2009, 2009-2010, and 2010-2011.

Pooled national data sets were used, and no individual

data sets were accessed or obtained. As such, no institu-

tional review board approval was required for data visuali-

zation and analysis.

The median numbers for ‘‘resident surgeon’’ classification

at each PGY year level were averaged across the 3 years of

data to obtain average national median case numbers for each

procedure at each PGY level.

Estimation of the means and standard deviations (SD)

from the mean for each procedure at each PGY level was

made using 2 assumptions: (1) the ratio between the median

and the mean is relatively consistent throughout training,

and (2) the ratio between the mean and 1 standard deviation

above and below the mean is relatively consistent through-

out training.

Thus, for each procedure, estimated means at each PGY

level were calculated from the median numbers by using the

following formula:

MeanPGY�X ~5MedianPGY�X � MeanGrad=MedianGradð Þ:

Estimated standard deviations from the mean were calcu-

lated in a similar fashion at each PGY level by using the

following formula, where 1 SD equals 1 SD above the

mean:

1SDPGY�X ~5MeanPGY�X � 1SDGrad=MeanGradð Þ:

For each key indicator procedure, these values were

plotted on the y-axis against time in training on the x-axis.

Increments of 1 SD were similarly calculated and plotted. A

curve of best fit was drawn for each estimated mean and

increments of both 1 and 2 standard deviations above and

below the mean.

The shape of each growth curve was described based on

which portion of a sinusoidal curve it most resembled: the

growth phase of the curve, the transition phase of the curve,

or the plateau phase of the curve (Figure 1). Thus, the

shape of the curve at the conclusion of residency was

termed the end-point geometry; each curve was labeled as

fitting 1 of 3 categories: (1) growth phase (if the rate of

accumulation of cases was progressively increasing during

the final years of training), (2) transitional phase (if the rate

of accumulation of cases was just beginning to decrease in

the final years of training), or (3) plateau phase (if the

number of cases was relatively flat in the final year of train-

ing with peak growth in number of cases occurring earlier

in training).

Key indicator growth charts were created with collapsed

data for each of the 4 key indicator domains. The standard

deviation used for the combined data was estimated from

the square root of the sum of the variances of each individ-

ual procedure.

Results

Using the above methods, national case log growth charts

were developed for each of the 14 ACGME Otolaryngology

Residency Review Committee key indicator procedures.

For the Head & Neck Surgery key indicator domain, each

of the 4 key indicator procedures—glossectomy, parotidect-

omy, neck dissection, and thyroid/parathyroid surgery—had a

growth-phase end-point geometry (Figure 2). The same was

true for all 3 key indicator procedures in the Otologic

Surgery domain—mastoidectomy, tympanoplasty, and stape-

dectomy/ossiculoplasty (Figure 3). In the Facial Plastic

Surgery domain, mandible/midface fracture had transitional-

phase geometry, while both rhinoplasty and flaps had

growth-phase end-point geometry (Figure 4).

In the General/Pediatric key indicator domain, ethmoi-

dectomy had a growth-phase end-point geometry. Both the

Airway–Pediatric and Adult key indicator and Congenital

Neck Mass key indicator had a transitional-phase end-point

geometry, while Bronchoscopy had a plateau-phase geome-

try (Figure 5).

Figure 1. End-point geometry of growth curves. Examples of 3 categories: (A) growth phase, (B) transitional phase, and (C) plateau phase.

74 Otolaryngology–Head and Neck Surgery 152(1)

When viewing combined data across each domain, in the key

indicator domains of Head & Neck Surgery, Otologic Surgery,

and Facial Plastic Surgery (Figure 6), growth-phase end-point

geometry was found. The General/Pediatric key indicator domain

demonstrated transitional-phase end-point geometry.

Discussion

Program directors are charged by the ACGME to evaluate

the sufficiency of case exposure for each resident as well as

the parity of case exposure between residents.3 This should

be evaluated on a semiannual basis. Until recently, there has

been no described method for making these comparisons gra-

phically. In addition, there was no ability to compare cases

with national averages except for graduate case numbers.

Program-specific growth charts have been developed in

the Department of Otolaryngology at the University of

Minnesota that compare current resident cases with the

growth charts of historical averages of recent graduates of

the program. These have been termed Growth in Operative

Procedures against Historical Expected Results (GOPHER)

graphs. However, to our knowledge, national operative log

growth charts have not been described in any field.

In developing national case log growth charts, we sought to

obtain mean numbers of cases for every resident year in train-

ing. These data are theoretically available in the ACGME case

log system but are not easily extracted to obtain national

means except for graduate case numbers. However, median

numbers for each year as well as final mean numbers may be

more easily extracted. Thus, as described in the Methods, the

mean numbers presented are estimated means based on the

assumption that the ratio between the median and the mean

remains consistent throughout training.

There is often a significant discrepancy between mean

and median case numbers due to extreme highs in many

cases that inflate the mean, as has been demonstrated by

Rosenberg and Franzese.4 Creating a curve based on the

median numbers would be possible without this calculated

adjustment to estimate the mean but would display a trajec-

tory of a curve that would significantly undershoot the pub-

lished mean numbers. In addition, the standard deviation

must be displayed relative to the mean rather than the

median. However, it should be noted that a curve based on

the mean provides a high bar that most residents will not

attain.

Figure 2. Head & Neck Surgery key indicator domain: resident surgeon numbers vs postgraduate year (PGY) level of training.

Jabbour and Tsue 75

With an understanding of these limitations, program direc-

tors may use these growth curves as part of the semiannual

review for residents at every level of training. Each resident’s

case numbers may be plotted against the background of these

growth curves. This would be most helpful in the third through

fifth years of training, as key indicator cases tend to be

procedures that are performed in the latter years of training.

This can be demonstrated by the end-point geometry of the

key indicator growth curves.

Most procedures performed early in training, such as

tympanostomy tube placement and tonsillectomy, follow a

sigmoidal-shaped growth curve. That is, there is a growth

Figure 3. Otologic Surgery key indicator domain: resident surgeonnumbers vs postgraduate year (PGY) level of training.

Figure 4. Facial Plastic Surgery key indicator domain: resident sur-geon numbers vs postgraduate year (PGY) level of training.

76 Otolaryngology–Head and Neck Surgery 152(1)

phase that increases exponentially and then linearly, a tran-

sitional phase in which there is beginning to be a decreased

slope to the growth curve, and then a plateau phase in

which the curve nears an asymptote—when it is no longer

performed by senior residents with significant frequency.

Many of the key indicator cases have curves that resemble a

truncated version of this graph; we have thus categorized

them based on which portion of this graph the curve most

resembles.

Ten of the 14 key indicators had a growth-phase end-

point geometry, where the rate of accumulation of these

cases was still increasing in the final years of training. Four

of the key indicators had transitional-phase or plateau-phase

end-point geometries—mandible/midface fractures, airway–

pediatric and adult, congenital neck mass, and broncho-

scopy. For these cases, the peak time period for exposure to

these cases is before the final year of training. These key

indicators may be better suited for evaluation of middle-

level residents.

In addition, the program director may use these growth

charts as a background for graphically displaying overall

changes in residency case numbers compared with national

averages in a nearly real-time approach. This may enable a

more accurate assessment of the effect of programmatic

changes, such as changes to rotation schedules, affiliated

faculty, participating sites, or complement increases or

decreases.

In making these comparisons, it should be remembered

that programs vary in the timing of procedures based on

their rotation schedules; however, we hope that the ability

of an individual resident to plot his or her case log progress

against the background of a national case log growth chart

would enhance each resident’s ability to reflect on the

strength and weaknesses of his or her training. This would

provide further information to senior residents, who may be

charged with making case assignments in some institutions,

to identify the categories of cases in which they may still be

deficient.

At a national level, the ability to plot a resident case log

against a national average case log could provide valuable

information to the ACGME Residency Review Committee.

In addition to reaccreditation decisions, the residency

review committee is charged with the difficult task of

approving initial accreditation and approvals of complement

Figure 5. General/Pediatric key indicator domain: resident surgeon numbers vs postgraduate year (PGY) level of training.

Jabbour and Tsue 77

increases. In the current system, information about the addi-

tion of trainees in each of these situations—a new program

or a change in the number of residents in an existing

program—is not available to the ACGME Residency

Review Committee until 6 years after the decision has been

made, as only graduate resident case numbers are available

for review. This time period is often beyond the tenure of

those making the initial decision, and if a deficiency is ulti-

mately noted, there has already been a missed opportunity

for remediation for several years of resident graduates. It

would be possible to monitor such changes in nearly real

time to assess the progress of trainees in programs that have

undergone these significant changes.

Prior to their use for high-stakes decision making, signif-

icant improvements could be made for future iterations of

these growth charts. Continued efforts should be made to

obtain the true median numbers at each PGY level and stan-

dard deviations at each PGY level to avoid the use of esti-

mated means. Also, it would be possible to extract data at

time intervals shorter than 1 year, such as quarterly or

monthly. This would allow for improved accuracy of the

curves when used for comparison in a semiannual or real-

time fashion. In the future, it may be possible to develop

growth curves that are integrated into a case log report

system, such that they may be automatically generated.

Many have argued that there is too much emphasis

placed on case numbers and that it is operative performance

that should receive increased attention. We would argue that

both are crucial during surgical training. Revisiting the

well-child visit analogy, there is value to knowing both the

quantitative measures of physical growth and the more qua-

litative assessment of the child’s developmental milestones.

Similarly, the case logs serve as a numeric marker of the

case exposure that is a necessary prerequisite for meeting

the functional goals of operative performance.

Conclusion

We have developed national case log growth curves for

operative procedures in otolaryngology. These growth

curves may be used as a standard against which to assess

resident case exposure progress longitudinally throughout

training.

Figure 6. Collapsed key indicator procedure domains: resident surgeon numbers vs postgraduate year (PGY) level of training.

78 Otolaryngology–Head and Neck Surgery 152(1)

Author Contributions

Noel Jabbour, design, acquisition of data, analysis, drafting of

article, final review and approval; Terance Tsue, design, analysis,

critical revisions, final review and approval.

Disclosures

Competing interests: Noel Jabbour was a member of the ACGME

Otolaryngology RRC from 2010 to 2012 and has been a member

of the ACGME Otolaryngology Milestones Working Group from

2011 to present. Terance Tsue has been a member of the ACGME

Otolaryngology RRC from 2010 to present and a chairperson of

the ACGME Milestones Working Group from 2011 to present.

Views expressed in the article are those of the authors and not

those of the ACGME.

Sponsorships: None.

Funding source: None.

References

1. Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME

accreditation system—rational and benefits. N Engl J Med.

2012;366:1051-1056.

2. Kuczmarski RJ, Ogden CL, Guo SS, et al. 2000 CDC growth

charts for the United States: methods and development.

National Center for Health Statistics. Vital Health Stat. 2002;

11(246):1-190.

3. ACGME program requirements for graduate medical education

in otolaryngology: effective July 1, 2014. http://www.acgme.org/

acgmeweb/Portals/0/PFAssets/ProgramRequirements/280_ otolar-

yngology_07012014.pdf. Accessed July 23, 2014.

4. Rosenberg TL, Franzese CB. Extremes in otolaryngology resi-

dent surgical case numbers. Otolaryngol Head Neck Surg. 2012;

147:261-270.

Jabbour and Tsue 79