ABA/ASA JOINT COUNCIL ON IN-TRAINING EXAMINATIONS 2004 REPORT TO SAAC/AAPD Glenn P. Gravlee, MD...
-
Upload
brett-gaines -
Category
Documents
-
view
213 -
download
0
Transcript of ABA/ASA JOINT COUNCIL ON IN-TRAINING EXAMINATIONS 2004 REPORT TO SAAC/AAPD Glenn P. Gravlee, MD...
ABA/ASA JOINT COUNCIL
ON IN-TRAINING
EXAMINATIONS
2004 REPORT TO SAAC/AAPDGlenn P. Gravlee, MD
Chair, Joint Council
The Ohio State University Medical Center
Department of Anesthesiology
JOINT COUNCIL MEMBERS
• ABA– David Chestnut
– Douglas Coursin
– Glenn Gravlee
– Patricia Kapur
– Mark Rockoff (Jim DiNardo)
– Raymond Roy
– Kenneth Tuman
• ASA– Arnold Berry
– John Cooper (John Rowlingson)
– Jeffrey Gross (V.C.)
– Philip Lebowitz
– Charles Otto
– Patricia Petrozza
– Mark Rosen
50 Question Writers
Title: Junior Editors
IN-TRAINING EXAMINATION EDITORS
Steven Allen, Audree Bendo, James Di Nardo, Carter Dodge, Sylvia Dolinski, John Ebert, John Emhardt, Robert Gaiser, Eric Kitain, Bruce Kleinman, Elliott Krane, Larry Krenis, Larry Kushins, Cynthia Lien, Vinod Malhotra, Donald Martin, Roger Mecca, John Moyers, Julia Pollock, Lindo Jo Rice, Robert Sladen, Richard Stypula, Richard Teplick, Helen Westman, Thomas Wolfe
IN-TRAINING EXAMINATION
• “3 exams in 1”
– In-Training Exam
– Requalifying Exam
– ABA Written Exam• Subset of 300 questions from ITE
– Plus: Recertification Exam• Well-performing A-types from ITE form the
question pool
Demographics of In-Training Examinations
4000
5000
6000
7000
8000
2000 2001 2002 2003 2004Tot
al N
um
ber
of
Exa
min
ees
Registrants
DEMOGRAPHICS OF IN-TRAINING EXAMINATIONS - AMG + IMG
0
200
400
600
800
1000
1200
1400
1600
1999 2001 2002 2003 2004
CA-1 CA-2 CA-3
Exam Content340 Questions ~ 100 are Links
0 50 100 150 200
Disease States
Anes Procedures
Physics/Math
Anat/Biochem
Pharmacology
Physiology
Number of Questions
15%
16%
4%
8%
47%
9%
Disease States Breakdown163 Questions
0
5
10
15
20
25
30
Pain
Respi
rato
ry
Cardi
ovas
cula
rCNS
GI/L
iver
/End
o/Ren
al
Hem
atol
Peds
OB
Oth
er
Nu
mb
er o
f Q
ues
tion
s
5.7%4.6%
7.2%
4.6% 4.6% 4.3%
5.7% 4.9%5.1%
Item Difficulty (Mean Logit ± SE)
-0.8
-0.6
-0.4
-0.2
0
0.2
1999 2000 2001 2002 2003 2004
Log
it S
cale
In-training ExamAverage Scaled Score
05
10152025303540
2000 2001 2002 2003 2004
All CA3 Residents
Gaps in Knowledge – CA3s
• 8 Surprising Gaps Identified
• Report being sent by ASA office
Examination Registration/Behavior
• Government Issue Picture ID
• NO Cell Phones or Pagers
• Calculators are NOT OK
• Consequences of Cheating
• Proctors are in charge
Joint Council Finances
• Joint Oversight by ASA/ABA
• Largely sustained by Exam fees
• In the Red ($50-100K) when N was low
Joint Council Finances
• Still losing $20-50K/year with strong N: increased proctor compensation, NBME fees mainly
• Exam cost @ $85 since 1996
• Joint Council approved increase in exam fee to $100 effective for 2005 exam
Content Outline Revision
• 1996 version used for 2004 Exam
• 2003 version used for 2005 Exam
• New Format: “Organocentric”
• Both versions on ASA website
– 2004 version disappears 12/31/04– www.asahq.org/publicationsandservices/contentoutlinerev2003
FAQs/Future Issues
• Computerized Exam?
• Phase out K-types?
• Introduce R-types and question
clusters (G types) in 2005
Q. Which of the following foods grows on trees?
1. Bananas
2. Tomatoes
3. Apples
4. Freedom (French) Fries
A:1,2,3 B:1,3 C:2,4 D:4 E:1,2,3,4
Answer: B
K-type evolution
• Always on exam: approx 40% in 1970s and 1980s
• Gradual decrease to current 20%
• Elimination being considered: primary rationale for evaluating R-types
• NBME dislikes, controversial among Joint Council members
Disadvantages to K-types
• No longer on USMLE
• Take longer to answer (approx 60 vs 50 sec)
• Slightly more difficult than A-types
• Examinees dislike them
Advantages to K types
• Question performance has been good – Discriminates stronger vs weaker examinees
pretty well, ideal K-type requires greater depth of understanding
• Anesthesiology doesn’t always lend itself to single best answer
• May take 3-4 A-types to test the same information as a single well-written K-type
K-type (interim) conclusions
• See how R-types perform
• Hold the line at 15-20% for now, reassess over next two years
• Let’s base K-type fate more upon the need for knowledgeable anesthesiologists than upon question popularity among examinees– Remember that they have 3-4 years to practice
Extended Matching (R-type)
A. Acute DIC
B. Hemophilia A
C. Platelet function defect
D. Idiopathic thrombocytopenia purpura
E. von Willebrand’s disease
F. Antithrombin III deficiency
1. 70 year-old man undergoing CABG/AVR is
oozing after CPB
Answer: C
1. A 70 year-old man undergoing CABG/AVR develops a coagulopathy after CPB.
Extended Matching (R-type)
A. Acute DIC
B. Hemophilia A
C. Platelet function defect
D. Idiopathic thrombocytopenia purpura
E. von Willebrand’s disease
F. Antithrombin III deficiency
Answer: B[Continue with additional questions]
An active 30 year-old man with a history of hemarthrosesand excessive bleeding after small cuts presents for
appendectomy.
R-types and G-types
Examples soon will appear on ASA Web site
Nonstandard Examinations
• Extended time, quiet room, reader, etc.• Application processing takes more time (4
months before exam)• Process described in ABA Booklet of
Information, Section 7.01• Same process for ITE as ABA written exam• Once approved by ITE, ABA usually
rubber-stamps it (same committee)
Nonstandard Exam: Concerns
• Numbers gradually increasing• Candidates often wait for ABA exam to request
accommodation• Candidates/examinees often allow insufficient
time• Candidates/examinees presume that a previous
accommodation automatically merits an ABA/ASA or ABA accommodation
Nonstandard Exam Concerns
• Recognize that A.D.D. does not automatically qualify for extended test time
• Recognize that psychological testing must be <5 years old
• Comments from candidates and from our expert consultants at times scare the Nonstandard Exams Committee
“When a resident fails to measure up to the academic standards and/or professional behavior expected, it is incumbent upon the program director to document these failures during the Residency Review Committee-mandated semiannual review.”
Miller SH, Plastic Reconst Surg 1990