- Klokkevold AAP Workshop 2014 · The Student Performance Committee also recommends students for...
41
Dr. Perry Klokkevold Program Director Postgraduate Periodontics Residency UCLA School of Dentistry Selecting Our Residents in the Pass/Fail Environment Selecting Our Residents in the Pass/Fail Environment 1 Dr. Perry Klokkevold Program Director Postgraduate Periodontics Residency UCLA School of Dentistry Dr. Perry Klokkevold Program Director Postgraduate Periodontics Residency UCLA School of Dentistry Dr. Paulo Camargo Associate Dean, Clinical Dental Sciences Chair, Section of Periodontics UCLA School of Dentistry Dr. Paulo Camargo Associate Dean, Clinical Dental Sciences Chair, Section of Periodontics UCLA School of Dentistry
Transcript of - Klokkevold AAP Workshop 2014 · The Student Performance Committee also recommends students for...
Dr. Pe r r y K lokkevo ld Program Director Postgraduate Periodontics Residency UCLA School of Dentistry
Selecting Our Residents in the Pass/Fail EnvironmentSelecting Our Residents in the Pass/Fail Environment
1
Dr. Pe r r y K lokkevo ld Program Director Postgraduate Periodontics Residency UCLA School of Dentistry
D r. Pe r r y K lokkevo ld Program Director Postgraduate Periodontics Residency UCLA School of Dentistry
D r. Pau lo Camargo Associate Dean, Clinical Dental Sciences Chair, Section of Periodontics UCLA School of Dentistry
D r. Pau lo Camargo Associate Dean, Clinical Dental Sciences Chair, Section of Periodontics UCLA School of Dentistry
UCLA School of Dentistry -‐ -‐ History of Pass/No Pass Grading System
Pass/No Pass at UCLA School of Dentistry started in Fall 1969
• only 1st and 2nd year; then all years by 1971
Occasional faculty discussions about returning to grades
• Pass/No Pass system has always prevailed
Merits of Pass/No Pass System
• Creates a different study/work environment
• Enhances collaboration and professionalism
• No sabotaging other’s work
2
presenter notes
Most recent discussion to consider returning to grades when National Boards went P/NP.
Discussions always included student input. NEVER came close to changing back to letter grades!
Pass/No Pass creates a different environment for learning (healthy, collegial, professional)
Dr. Stu White (emeritus professor; member of 1st class) “We’re all in this together!”
Letter grades are NOT good when it comes to Clinical Competency
3
Teaching Experience
Gathering Information about Applicants
• PASS Application vs Other?
• Demographics
• Education and Training
• Personal Statement
• Curriculum Vitae
• Transcripts
• GPA, Class Ranking
• Recommendation Letters
• PPIs
• Traditional Letters
• Institution Evaluation
!
• Additional Sources ?
4
Academic Achievement
Honors & Awards
Leadership Activities
Research Experience
Extracurricular Activities
Demonstrated Interest in
Periodontics
GPA, Class Rank
48%
44%
44%
GPA, Class Rank
5%
16%
20%
GPA, Class Rank
33%
42%
34%
GPA, Class Rank
GPA, Class Rank -‐ -‐ What do they mean?
__?
UCLA School of Dentistry -‐ Entering Student Statistics
10
insert white coat ceremony photo
Class of 2016 Class of 2015 Class of 2014
Females / Males 46 / 42 37 / 51 43 / 45
# Students 88 88 88
Overall GPA 3.69 3.67 3.70
Science GPA 3.66 3.66 3.67
DAT Acad Avg 22 22 21
DAT Perceptual 22 21 22
DAT Science 22 22 22
Student Evaluation
at UCLA SOD*
11
Honors / Pass / No PassH Honors
P Pass
NP No Pass
NP/P Remediated No Pass
NG No Grade
IP In Progress
I Incomplete
Awarded for outstanding performance (max 20% -‐Chair discretion)
Satisfactory completion of course requirements
Unsatisfactory completion of course requirements
Highest grade given after an NP is made up
Course requirements not completed for reasons beyond student’s control (e.g. student not assigned to block)
Progressing towards completion of clinical and pre-‐clinical course
Course requirements not completed due to illness, personal emergency or other excused absence (documentation required)
presenter notes
* Effective with Class of 2014 (previously did not include “H”)
* Faculty tend to NOT give “Honors” in Clinical courses. They are looking for competence.
* There are many dimensions of clinical work. Does the work demonstrate clinical competence? Does it PASS or NOT PASS?
* It is difficult to “grade” clinical procedures. Is it clinically acceptable or are there critical errors.
12
Student Evaluation
at UCLA SOD*
13
Student Evaluation
at UCLA SOD*
14
Student Evaluation
at UCLA SOD*
15
Performance Reports (PRs, formerly EPRs)
Narrative document of excellent (E report) or unsatisfactory (U report) performance.
• If a student is required to make-‐up an NP, IP or I in a course, the accompanying PR will specify the terms for remediation, make-‐up and expected timeline.
• If a student receives an H in the course, the accompanying PR will acknowledge the outstanding performance in the course.
16
Internal
Use Only
Performance Reports (PRs, formerly EPRs)
The following criteria may be used in PR:
1. Knowledge of Subject
2. Quality of Performance
3. Professional Behavior
4. Quantity
5. Patient Management17
Internal
Use Only
Dean’s Letter (by recommendation from the Student Performance Committee)
Prestigious letter of recognition earned annually by the Top 5-‐10% of each Class
•Determined by assessment of “H” grades
• Internal Performance Reports
• Those with a grade of NP or NP/P are NOT eligible
18
presenter notes
The Student Performance Committee also recommends students for graduation with honors (e.g. Magna cum laude, Summa cum laude, cum laude) based on an overall assessment of performance of the entire four years in dental school.
19
Recommendation Letters -‐ QUALITY of LETTER and WRITER !
PPIs
Traditional Written Letters
Institution Evaluation
20
Who is writing evaluation?
part-‐time? specialty? group director? clinic supervisor?
How well do they know the candidate?
Letter writers tend to be “selected” to write “good” evaluation.
What is overall mix of letter writers?
Are evaluations consistent? Is the sample siz
e large eno
ugh?
• PPIs
• Traditional
Written Letters
• Institution
Evaluation
Recommendation Letters
21
• PPIs
• Traditional
Written Letters
• Institution
Evaluation
Recommendation Letters
22
• PPIs
• Traditional
Written Letters
• Institution
Evaluation
Recommendation Letters
23
• PPIs
• Traditional
Written Letters
• Institution
Evaluation
Recommendation Letters
24
• PPIs
• Traditional
Written Letters
• Institution
Evaluation
Recommendation Letters
25
• PPIs
• Traditional
Written Letters
• Institution
Evaluation
Recommendation Letters
26
• PPIs
• Traditional
Written Letters
• Institution
Evaluation
Recommendation Letters
27
• PPIs
• Traditional
Written Letters
• Institution
Evaluation
Recommendation Letters
28
• PPIs
• Traditional
Written Letters
• Institution
Evaluation
Recommendation Letters
29
• PPIs
• Traditional
Written Letters
• Institution
Evaluation
Recommendation Letters
30
• PPIs
• Traditional
Written Letters
• Institution
Evaluation
Recommendation Letters
31
• PPIs
• Traditional
Written Letters
• Institution
Evaluation
Recommendation Letters
32
Recommendation Letters -‐ Institution Evaluation
33
States if student currently in
“Good Academic Standing”
Summarizes
Honors, Dean’s Letters, Awards
Describes
Leadership, Research, Extracurricular
!
Indicates Level of Recommendation
• Highest Recommendation
• Highly Recommend
• Recommend
!
• Do Not Recommend??
presenter notes
“It’s often the information that is NOT stated in the letter that is IMPORTANT.” Do Not Recommend -is NOT likely to be used… hence, the full range of possible recommendations NOT used.
34
Recommendations for Assessing Students from Pass/No Pass Schools
• Read EVERYTHING
• Evaluate Letter Writers as well as content
• Dissect Transcripts and Institution Letter
of Evaluation
• Information from application used to
determine which candidates to “Interview”
• Must INTERVIEW applicants!
• Personality / Attitude / Motivation
• Interaction with others
!
• Role of residents
35
presenter notes
The Interview is essential in the assessment of applicants:
“Good on Paper . . . but would you rent a room in your home to someone without meeting them?”
“System works. We do a good job. You cannot differentiate between schools.”
“Admissions Process: None of this is Science.”
“Cannot select residents from a Spreadsheet of Numbers.”
36
STANDARDIZED EXAMINATION BACKGROUNDOral and Maxillofacial Surgery
Options ✓ Creation of a new exam by the American Board of OMFS ✓ Adoption of an existing exam
Group consensus ✓ The most relevant area for resident selection was basic
medical knowledge ✓ OMFS residents often go through medical school
Group Decision ✓ Adoption of Comprehensive Basic Science Examination
(CBSE) – based on part 1 of the United States Medical Licensing Examination (USMLE)
✓ Exam is offered by the National Board of Medical Examiners ✓ Exam is routinely given to medical students (it is offered
twice per year) !
Examination ✓ Systems based
o Identify candidate’s areas of strength and weaknesses o Can be taken as many times as desirable; only report
highest score o In medical education: non-‐binding; used as preparation
for USMLE 37
• Concerns were similar to Periodontics; lack of a quantitative instrument to evaluate residency candidates • Exacerbation of the problem occurred when National Boards became P/NP
• In search of a replacement for National Board scores
Options ✓ Creation of a new exam by the American Board of
Periodontology ✓ Creation of a new exam by ADEA/ADA/AAP ✓ Adoption of an existing exam (unlikely, cannot use NDBs)
Group consensus ✓ What is the most relevant area for resident selection in
Periodontics? • Basic sciences, dentistry, periodontics, etc.
✓ Could the exam be general so that other specialty programs could use share its development/administration?
Group Decision !Create exam ✓ Role of ADEA, ADA, AAP, program directors
!Exam results can be used for admission as well as for prediction of resident performance
✓ Issue with CBSE • Students are often absent to study for the exam • Limitations of any other written examination
STANDARDIZED EXAMINATION Periodontics
38
• Exacerbation of the problem occurred when National Boards became P/NP
• In search of a replacement for National Board scores
PORTFOLIO DEVELOPMENTLicensure Mechanism in California
Includes various disciplines/areas ✓ Oral diagnosis and treatment planning ✓ Periodontics ✓ Direct restorations ✓ Indirect restorations ✓ Endodontics ✓ Removable prosthodontics
!
Quantitative requirements ✓ Minimum number of clinical experiences
Qualitative requirements ✓ Competency exams ✓ Possible remediation
Evaluates minimal competency ✓ Still a pass/no pass system for licensure
39
• New, alternative licensure mechanism in California – will be available in 2015 • Available to graduates of California dental schools
PORTFOLIO DEVELOPMENTPeriodontics
40
Modification of licensure portfolio modelIncludes various disciplines/areas ✓ Oral diagnosis / medicine ✓ Non-‐surgical periodontal therapy ✓ Periodontal therapy ✓ Periodontal restorative dentistry ✓ Maintenance ✓ Dental Implants
!Quantitative requirements ✓ Case documentation similar to old ABP style ✓ Case requirements
!Qualitative requirements ✓ Case documentation • Charting, Radiographs, Photographs
✓ Case description / conclusions !
• Submission as a component of the application documentation
• Case presentation during interview? • Still subjective • Logistically complicated
Dr. Pe r r y K lokkevo ld Program Director Postgraduate Periodontics Residency UCLA School of Dentistry
Selecting Our Residents in the Pass/Fail EnvironmentSelecting Our Residents in the Pass/Fail Environment
41
Dr. Pe r r y K lokkevo ld Program Director Postgraduate Periodontics Residency UCLA School of Dentistry
D r. Pe r r y K lokkevo ld Program Director Postgraduate Periodontics Residency UCLA School of Dentistry
D r. Pau lo Camargo Associate Dean, Clinical Dental Sciences Chair, Section of Periodontics UCLA School of Dentistry
D r. Pau lo Camargo Associate Dean, Clinical Dental Sciences Chair, Section of Periodontics UCLA School of Dentistry
No PassNo Pass
