Effect of the New Standards for Case Logging on Resident Operative Volume: Doing Better Cases or Better Numbers?

R. Murthy, MD, A. Shepard, MD, A. Swartz, BS, A. Woodward, MD, C. Reickert, MD, H.M. Horst, MD and I. Rubinfeld, MD, MBA

Department of Surgery, Henry Ford Hospital, Detroit, MI

Center for Health Systems Research, Henry Ford Health System

Background

• The operating room experience is arguably the most important aspect of surgery residency training (Patient Care).

• Operative case volume is an important component of surgical training – the American Board of Surgery and ACGME minimum case requirement has increased from 500 to 750.

• In 2009, the ACGME modified the designation of major (index) operative cases to include many new types including some previously considered “minor.”

Changes in Case Classification • Rationale not completely clear, but ostensibly

to update what a surgical trainee’s operative experience should be in the ‘new era’ of surgical training.

• Some changes make sense – e.g. roux-en-y gastric bypass, inguinal hernia repair, open and lap appendectomy.

• Others do not – e.g. breast biopsy, toe amputation, I&D perirectal abscess.

• Impact of these changes unknown.

ACGME Op Log System• Web-based case logging system introduced in

2002, uses CPT codes to categorize cases.• Mandated method to document operative

experience• Limitations of Op Log data:– Dependent on self-reporting– Subjective / value judgments– Unclear number of re-codes– Not audited– Logging stopped at “perceived” threshold of “enough” – Does not reflect the “universe of case”

National Surgical Quality Improvement Project (NSQIP)• The Bad:• Not designed for resident education or to analyze an

individual surgeon• Sampling methodology• No resident input.• No focus on education.

• The Good:• Large national quality database.• Reliable, validated, audited.• Describes the universe of available cases rather than

just those logged.

Study Purpose

To assess the potential effect of the recent changes in what constitutes a major (index) case on the educational value of the resident operative experience.

Methods• We analyzed all general and vascular surgery cases

in the NSQIP (National Surgical Quality Improvement Project) database

• NSQIP public use files (PUF) from 2005 to 2008 were reviewed.

• Primary CPT case coding was mapped to the ACGME major (index) case category using both the old and new classification schemes.

• We also ranked by volume, looked at the top 20 procedure codes and summarized those by category

Methods

• Cases with and without resident coverage were analyzed.

• Non-specialty data (e.g. Urology, Cardiac Surgery, Gynecology) were analyzed exclusively to avoid bias.

• Categorical variables were analyzed with chi-square.

• Data analysis was performed with SPSS software (SPSS Inc. Chicago, IL. Version 19).

Results

There was a progressive increase in hospitals enrolled in NSQIP from 2005 to 2008.

Year Hospital #s

2005 37

2006 69

2007 173

2008 203

Case Volume and Distribution: General and Vascular Surgery

Year Gen Surg (%) Vasc Surg (%) Total

2005 29,550 (87.1) 4380 (12.9) 33,930

2006 100,332 (87.7) 14,111 (12.3) 114,443

2007 169,503 (86.2) 27,232 (13.8) 196,735

2008 197,197 (85.4) 33,714 (14.6) 230,911

Total 496,582 (86.2) 79,437 (13.8) 576,019

Case Volume and Distribution: Vascular and General Surgery

Case Volume and Distribution: Vascular and General Surgery

0

50000

100000

150000

200000

250000

2005 2006 2007 2008

Year

general surgery

Vascular

Total

Trend Over Four Years

Major case designation by old and new classification schemes

0

50000

100000

150000

200000

250000

2005 2006 2007 2008

Year

old criteria

new criteria

Comparison of major case designation: old and new criteria

Old Non-Major Old Major Total

New Non-Major 30,587 7089 37,676

New Major 173,977

(30.2%)

364,366 538,343

Total 204,564 371,455 576,019

Top 20 Procedures Summarized by Category: New Major Cases, Not Previously Designated as Major

Count Percentage Cumulative %

Breast 46,652 26.8 26.8

Bariatric 27,731 15.9 42.8

Ventral Hernia 23,199 13.3 56.1

Appendectomy 10,190 5.9 61.9

Amputation 6041 3.5 65.4

Ex Lap 5916 3.4 68.8

Lap Colectomy 3320 1.9 70.7

Peri-rectal abscess 2857 1.6 72.4

Inguinal hernia 2596 1.5 73.9

Varicose veins 2519 1.4 75.3

Category-based Discussion

• Bariatric surgery and lap colectomy:– Likely previously counted as something else– Highly technical– Increasingly fellow (not resident) cases

• Appendectomy, inguinal hernia repair, AK and BK amputations a reasonable call.

• Breast biopsy and peri-rectal abscess: Are these really major cases?

Conclusions

Some cases newly classified as major are technically advanced procedures (e.g. roux-en-y gastric bypass).

Other cases newly classified as major, are clearly not (e.g. breast lesion excision).

There is potential for the major case category to be diluted by less demanding case types.

Implications for Surgical Training• Are we preserving case volumes at the

expense of case quality and complexity?• Can we rely on the learners to maintain the

data?• Is it enough of a perspective of the broader

view of surgery?• Was this decision transparent and made with

appropriate due diligence?