April 14, 2010  

Nick Busing Secretary, Committee on Accreditation of Canadian Medical Schools President and CEO, AFMC The Association of Faculties of Medicine of Canada 265 Carling Avenue, Suite 800 Ottawa, ON   K1S 2E1 

& 

Dan Hunt LCME Secretary and Vice President,  Section for LCME and Accreditation Services  Assn. of American Medical Colleges 2450 N Street, NW Washington, DC 20037 

RE: Follow‐up Report ‐ Outstanding “Area of Transition”, Faculty of Medicine, University of Toronto 

Dear Drs Busing and Hunt: 

In response to the July 3, 2008 consolidated letter from the Committee on Accreditation of Canadian Medical Schools (CACMS) and the Liaison Committee on Medical Education (LCME), the Faculty of Medicine, University of Toronto is pleased to provide you with a follow‐up report, informing you of the progress made in fulfilling the outstanding “area of transition item”‐   “relating evaluation strategies to outcome objectives”.  This report includes, as requested, a table that illustrates how and where in the curriculum the educational program objectives will be evaluated (see Appendix 1.A & B) as well as a timeline for review of the educational program objectives (see Appendix 2). 

We hope that this follow‐up report has addressed all of the outstanding concerns.   We look forward to hearing from you.  If you have any questions, please feel free to contact the undersigned. 

Sincerely, 

 

Jay Rosenfield, MD, MEd, FRCPC. Vice Dean, Undergraduate Medical Education  Faculty of Medicine, University of Toronto  Cc:  Catharine Whiteside, Dean, Faculty of Medicine  

APPENDIX 29APPENDIX 31

CACMS/LCME UT Follow‐up Report  Appendix 1.A 

Curriculum Objectives Evaluation Introduction 

 Undergraduate Medical Education 

University of Toronto April 2010 

     

CACMS/LCME UT Follow-up Report, Appendix 1.A

1 | P a g e

Curriculum Objectives Evaluation 

Undergraduate Medical Education University of Toronto 

 As part of its ongoing program evaluation process, the University of Toronto’s Undergraduate Medical Education (UME) Program Evaluation Committee regularly evaluates the extent to which the current UME Curriculum Objectives are being achieved.  The most recent version of the table summarizing this evaluation is enclosed (March, 2010).  It continues to be informed by results from Part I and Part II of the Medical Council of Canada’s Qualifying Examinations (MCCQE) as well as student assessments conducted within the Toronto MD Program, particularly the most recent 2 years of assessments from clinical clerkship rotations and relevant courses.  The UME performance targets that guide the evaluation of the curriculum objectives are:  ●  That the mean standardized scores for University of Toronto medical graduates exceed the Canadian national average (which also includes Toronto’s graduates) on all aspects of Part I and Part II of the MCCQE;  ●  That Toronto be ranked within the top half of Canadian Medical schools according to the mean standardized score for all aspects of Part I and Part II of the MCCQE; and,  ●  That the vast majority of Toronto students meet or exceed expectations on the evaluated clinical clerkship components, pass the evaluated examination components, and have no professionalism lapses noted.  The notation in the table’s right hand column indicates the current assessment for each objective, with “√” indicating the objective component is currently being met, “Follow‐up Required” indicating that objective component is not being currently met and needs review by the Undergraduate Medicine Curriculum Committee, “±” indicating that the objective component is currently “indeterminate” (i.e. it is not clear at present whether that objective component is currently being met), and “IPR” indicating that the evaluation of the objective is currently in progress.  We are very pleased with our program evaluation results‐ overall, 84% (41 out of 49) of Toronto’s UME curriculum objectives are being fully met.  Of the remaining 8 objectives, 3 are currently indeterminate, and 4 are well in progress, and only one sub‐component of one objective is not being met and requires further follow‐up.  Further details regarding these 8 curriculum objectives are presented below.  A sub‐component of medical expert objectives 3a, 3b, and 3c are currently assessed as “indeterminate” because in the most recent year, for the first time in 8 years, our school’s ranking was in the lower half—ranked 9th out of 16—of Canadian medical schools on the Data Acquisition sub‐component on Part II of the Medical Council of Canada’s Qualifying Examination (MCCQE).  Recognizing the inherent year‐to‐year instability of school rankings, we will monitor this measure closely in the coming years and await results from the next exam before declaring whether the objective has been met.  Further, we also noted that in contrast to this MCCQE component ranking, Toronto students overall have received excellent assessments on the applicable directly comparable aspects of the clinical clerkship student evaluations, and do otherwise very well on the Part II exam.  

CACMS/LCME UT Follow-up Report, Appendix 1.A

2 | P a g e

A subcomponent of medical expert objective 3d is currently deemed as requiring follow‐up.  In each of the past 3 years, Toronto’s has ranked between 10th and 13th out of 16—of Canadian Medical schools on the Clinical Decision Making sub‐component on Part I of the MCCQE.   In response to this recent, recurrent performance, a series of assessments and responses have been initiated.  Because these Part I MCCQE results are not consistent with Toronto student’s excellent performance during the same period on the analogous component on Part II of the MCCQE (i.e. Problem Solving) and on the applicable clinical clerkship assessment components, we have considered the possibility that the Part I MCCQE results may be due to the lack of exposure that Toronto students currently receive to Key Features questions that are used for the Clinical Decision Making portion on Part I of the MCCQE.  As such, steps have been taken to increase familiarity of Toronto medical students with this form of assessment, including a presentation to the students at this year’s MCCQE preparation sessions.  In addition, faculty development sessions are being organized to facilitate the more frequent use of Key Features questions in applicable examinations throughout the curriculum.  Progress on this aspect of Part I of the MCCQE will continue to be monitored to better inform these and any other indicated curricular initiatives.  The remaining 4 curricular objectives (Scholar objectives 2b, 2d, 3a, and 3b) are currently “in progress.”  While curriculum relevant to these 4 objectives has recently been added and included in the UME curriculum as noted in the table, the relevant student assessments are only being introduced in the current academic year.  As such, overall assessment and tabulation of student achievement for each of these objectives are not currently available.

   

CACMS/LCME UT Follow‐up Report Appendix 1.B 

Curriculum Objectives Evaluation Table 

 Undergraduate Medical Education 

University of Toronto April 2010 

1. Medical Expert/Skilled Clinical Decision-Maker The medical graduate will be able to:

Curriculum Objective Performance Measures Achieved Results *OK?

1. Demonstrate knowledge of the scientific and humanistic foundations of medicine and be able to apply that knowledge to the practice of medicine. _____________________________________ 2. Demonstrate a thorough knowledge of etiology, pathogenesis, clinical features, complications, principles of prevention and management of common and life-threatening illnesses presenting throughout the age spectrum, including all of the core clinical presentations outlined by the Medical Council of Canada. _____________________________________ 3a. Demonstrate the ability to obtain and document both a complete and focused medical history, as the situation requires.

MCCQE-2: Overall Score Clerkship Ward Evaluations (Total) _________________________________ MCCQE-1: Total Clerkship Ward Evaluations: - Knowledge (Basic Science & Clinical) _________________________________ MCCQE-2: Data Acquisition Sub-score Clerkship Ward Evaluations: - History Taking

Since 2001, have exceeded the Canadian mean and rank has ranged from 2nd to 5th 2007-9† 2006-8† Outstanding: 9,779 9,177 Exceed Expectations: 18,990 17,802 Meet Expectations: 5,340 4,779 Below Expectations: 5 8 Unsatisfactory: 0 0 Total Evaluations: 34,114 31,766 ___________________________________ Since 2004, have exceeded the Canadian mean and rank has ranged from 3rd to 7th 2007-9 2006-8 Outstanding: 357 298 Exceed Expectations: 1,268 1,212 Meet Expectations: 424 402 Below Expectations: 2 1 Unsatisfactory: 0 0 Total Evaluations: 2,051 1,913 ___________________________________ Since 2001, have exceeded the Canadian mean and rank has ranged from 1st to 6th except in 2008 when ranked 9th

2007-9 2006-8 Outstanding: 645 604 Exceed Expectations: 1,140 1,075 Meet Expectations: 267 233 Below Expectations: 0 1 Unsatisfactory: 0 0 Total Evaluations: 2,052 1,913

√ √

___ √ √

___

± √

CACMS/LCME UT Follow-up Report, Appendix 1.B 1. Medical Expert/Skilled Clinical Decision-Maker (cont’d)

The medical graduate will be able to:

Curriculum Objective Performance Measures Achieved Results OK?

3b. Demonstrate the ability to perform and document both a complete and focused physical and mental status examination, as the situation requires. _____________________________________ 3c. Demonstrate the ability to interpret commonly-employed laboratory tests, including tests of blood and other body fluids, various imaging modalities, and other specific tests such as electrocardiography.

MCCQE-2: Data Acquisition Sub-score Clerkship Ward Evaluations: - Physical Examination _________________________________ MCCQE-2: Data Acquisition Sub-score Clerkship Ward Evaluations: - Diagnostic Test Interpretation

Since 2001, have exceeded the Canadian mean and rank has ranged from 1st to 6th except in 2008 when ranked 9th

2007-9 2006-8 Outstanding: 371 341 Exceed Expectations: 1,125 1,052 Meet Expectations: 353 324 Below Expectations: 0 0 Unsatisfactory: 0 0 Total Evaluations: 1,849 1,717 ___________________________________ Since 2001, have exceeded the Canadian mean and rank has ranged from 1st to 6th except in 2008 when ranked 9th

2007-9 2006-8 Outstanding: 288 242 Exceed Expectations: 1,163 1,150 Meet Expectations: 372 294 Below Expectations: 0 0 Unsatisfactory: 0 0 Total Evaluations: 1,823 1,686

± √

___

± √

CACMS/LCME UT Follow-up Report, Appendix 1.B 1. Medical Expert/Skilled Clinical Decision-Maker (cont’d)

The medical graduate will be able to:

Curriculum Objective Performance Measures Achieved Results OK?

3d. Demonstrate the ability to integrate the above history, physical and laboratory tests findings into a meaningful diagnostic formulation. _________________________________ 3e. Demonstrate therapeutic and on-going management skills with respect to health and disease.

MCCQE-2: Problem Solving Sub-score MCCQE-1: Clinical Decision Making Sub-score Clerkship Ward Evaluations: - Problem Formulation and Management Plan (Clinical Judgment) _____________________________ MCCQE-2: Problem Solving Sub-score Clerkship Ward Evaluations: - Problem Formulation and Management

Plan (Clinical Judgment)

- Technical and Procedural Skills

Since 2007, have exceeded the Canadian mean and rank has ranged from 3rd to 4th Since 2007, less than Canadian mean and rank has ranged from 10th to 13th 2007-9 2006-8 Outstanding: 477 488 Exceed Expectations: 1,120 1,034 Meet Expectations: 453 391 Below Expectations: 2 0 Unsatisfactory: 0 0 Total Evaluations: 2,052 1,913 ______________________________ Since 2007, have exceeded the Canadian mean and rank has ranged from 3rd to 4th 2007-9 2006-8 Outstanding: 477 488 Exceed Expectations: 1,120 1,034 Meet Expectations: 453 391 Below Expectations: 2 0 Unsatisfactory: 0 0 Total Evaluations: 2,052 1,913 2007-9 2006-8 Outstanding: 264 241 Exceed Expectations: 778 750 Meet Expectations: 365 329 Below Expectations: 0 0 Unsatisfactory: 0 0 Total Evaluations: 1,407 1,320

√

Follow-up

required √

____ √ √ √

CACMS/LCME UT Follow-up Report, Appendix 1.B 1. Medical Expert/Skilled Clinical Decision-Maker (cont’d)

The medical graduate will be able to:

Curriculum Objective Performance Measures Achieved Results OK?

4. Retrieve, analyze, and synthesize relevant and current data and literature, using information technologies and library resources, in order to help solve a clinical problem. _____________________________________ 5. Propose clinical decisions utlilizing methods which integrate the best research evidence with clinical expertise and patient values.

Clerkship Ward Evaluations: - Use of Evidence-Based Medicine _________________________________ Clerkship Ward Evaluations: - Use of Evidence-Based Medicine - Communication with Patients, Families and Community

2007-9 2006-8 Outstanding: 270 231 Exceed Expectations: 1,152 1,070 Meet Expectations: 377 327 Below Expectations: 0 0 Unsatisfactory: 0 0 Total Evaluations: 1,799 1,628 ___________________________________ 2007-9 2006-8 Outstanding: 270 231 Exceed Expectations: 1,152 1,070 Meet Expectations: 377 327 Below Expectations: 0 0 Unsatisfactory: 0 0 Total Evaluations: 1,799 1,628 2007-9 2006-8 Outstanding: 945 934 Exceed Expectations: 922 831 Meet Expectations: 180 146 Below Expectations: 0 1 Unsatisfactory: 0 0 Total Evaluations: 2,047 1,901

√

___ √ √

CACMS/LCME UT Follow-up Report, Appendix 1.B 2. Communicator/Doctor-Patient Relationship

The medical graduate will be able to:

Curriculum Objective Performance Measures Achieved Results OK?

1. Communicate effectively with patients, their families and the community through oral, written and other non-verbal means of communication, respecting the differences in beliefs and backgrounds among patients and students. 2. Establish professional relationships with patients, their families (when appropriate) and community that are characterized by understanding, trust, respect, empathy, and confidentiality. 3. Deliver information to the patient and family (as appropriate) in a humane manner, and in such a way that is easily understood, encourages discussion and promotes the patient’s participation in decision-making. 4. Gather information, negotiate a common agenda, and develop and interpret a treatment plan, while considering the influence of factors such as the patient’s age, gender, ethnicity, cultural and spiritual values, socioeconomic background, medical conditions, and communication challenges. Note: The same performance measures apply to each of these 4 curriculum objectives.

MCCQE-2: Patient Interaction (ie, Communication) Sub-score Clerkship Ward Evaluations: - Communication with Patients, Families and Community - Patient Education

Since 2003, have exceeded the Canadian mean and rank has ranged from 2nd to 7th 2007-9 2006-8 Outstanding: 945 934 Exceed Expectations: 922 831 Meet Expectations: 180 146 Below Expectations: 0 1 Unsatisfactory: 0 0 Total Evaluations: 2,047 1,901 2007-9 2006-8 Outstanding: 523 510 Exceed Expectations: 1,069 970 Meet Expectations: 216 201 Below Expectations: 0 2 Unsatisfactory: 0 0 Total Evaluations: 1,808 1,683

√

√

√

CACMS/LCME UT Follow-up Report, Appendix 1.B 2. Communicator/Doctor-Patient Relationship (cont’d)

The medical graduate will be able to:

Curriculum Objective Performance Measures Achieved Results OK?

5. Demonstrate the importance of cooperation and communication among health professionals so as to maximize the benefits to patient care and outcomes, and minimize the risk of errors.

MCCQE-2: Patient Interaction (ie, Communication) Sub-score Clerkship Ward Evaluations: - Team Participation (Contribution within Inter-Disciplinary Team) - Provision of Patient Care in Collaboration with all Health Care Providers

Since 2003, have exceeded the Canadian mean and rank has ranged from 2nd to 7th 2007-9 2006-8 Outstanding: 964 953 Exceed Expectations: 897 806 Meet Expectations: 186 149 Below Expectations: 1 1 Unsatisfactory: 0 0 Total Evaluations: 2,048 1,909 2007-9 2006-8 Outstanding: 730 692 Exceed Expectations: 947 880 Meet Expectations: 160 136 Below Expectations: 0 0 Unsatisfactory: 0 0 Total Evaluations: 1,837 1,708

√

√

√

CACMS/LCME UT Follow-up Report, Appendix 1.B 3. Collaborator

The medical graduate will be able to:

Curriculum Objective Performance Measures Achieved Results OK?

1. Describe the roles and expertise of all members of an interdisciplinary team that are required to optimally achieve a goal related to patient care, a research problem, an educational task, or an administrative responsibility. 2. Develop a care plan for a patient he/she has assessed, including investigation, treatment and continuing care, in collaboration with the members of the interdisciplinary team. 3. Participate in interdisciplinary team discussions, demonstrating the ability to accept, consider and respect the opinions of other team members, while contributing an appropriate level of expertise to patient care. Note: The same performance measures apply to each of these 3 curriculum objectives.

Clerkship Ward Evaluations: - Team Participation (Contribution within Inter-Disciplinary Team) - Provision of Patient Care in Collaboration with all Health Care Providers

2007-9 2006-8 Outstanding: 964 953 Exceed Expectations: 897 806 Meet Expectations: 186 149 Below Expectations: 1 1 Unsatisfactory: 0 0 Total Evaluations: 2,048 1,909 2007-9 2006-8 Outstanding: 730 692 Exceed Expectations: 947 880 Meet Expectations: 160 136 Below Expectations: 0 0 Unsatisfactory: 0 0 Total Evaluations: 1,837 1,708

√

√

CACMS/LCME UT Follow-up Report, Appendix 1.B 4. Manager

The medical graduate will be able to:

Curriculum Objective Performance Measures Achieved Results OK?

1. Participate effectively in health care organizations, ranging from individual clinical practices to Academic Health Sciences Centres, exerting a positive influence on clinical practice and policy-making in one’s professional community. ____________________________________ 2. Describe the governance, structure, financing, and operation of the health care system and its facilities and how this influences patient care, research and educational activities at a local, provincial, regional, and national level. ____________________________________ 3. Apply a broad base of information to the care of patients in ambulatory care, hospitals and other health care settings. ____________________________________ 4. Describe the rationale for wise stewardship of available resources, appreciating the overall framework for resource allocation, and the absolute and relative levels of resources in various components of the health care system.

Clerkship Ward Evaluations: - Awareness of Appropriate Use of Healthcare Resources _________________________________ Determinants of Community Health (DOCH) examination: - Performance on the Health Care System components of the DOCH 1 examination _________________________________ Clerkship Evaluations: - Team Participation (Contribution within Inter-Disciplinary Team) _________________________________ Determinants of Community Health examination: - Performance on the Continuous Quality Improvement (CQI) components of the DOCH 3 examination

2007-9 2006-8 Outstanding: 317 277 Exceed Expectations: 1,159 1,109 Meet Expectations: 516 455 Below Expectations: 0 0 Unsatisfactory: 0 0 Total Evaluations: 1,992 1,841 ___________________________________ 2008/9‡ 2007/8‡ > 80%: 73.7% 79.6% 60 – 79%: 25.9% 19.6% < 60%: 0.4% 0.9% ___________________________________ 2007-9 2006-8 Outstanding: 964 953 Exceed Expectations: 897 806 Meet Expectations: 186 149 Below Expectations: 1 1 Unsatisfactory: 0 0 Total Evaluations: 2,048 1,909 ___________________________________ 2008/9 2007/8 > 80%: 59.0% 52.2% 60 – 79%: 35.1% 46.8% < 60%: 5.9% 1.0%

√ ____ √

____ √

____ √

CACMS/LCME UT Follow-up Report, Appendix 1.B 4. Manager (cont’d)

The medical graduate will be able to:

Curriculum Objective Performance Measures Achieved Results OK?

5. Help to build better teams. ____________________________________ 6. Describe how population-based approaches to health care services can improve medical practice. ____________________________________ 7. Participate in planning, budgeting, evaluation and outcome of a patient care program. ____________________________________ 8. Participate in innovative approaches to clinical care.

Clerkship Ward Evaluations: - Team Participation (Contribution within Inter-Disciplinary Team) _________________________________ Determinants of Community Health (DOCH) examination: - Performance on the Health Promotion essay in DOCH 1. This essay requires students to examine the role of different health promotion approaches to improve the health of populations served by community agencies _________________________________ Students participate in applicable simulation exercises: Carter racing team (Year 1) and Lakeview (Year 3) simulations _________________________________ Year 3 students participate in applicable Institute for Healthcare Improvement (IHI) exercise/module

2007-9 2006-8 Outstanding: 964 953 Exceed Expectations: 897 806 Meet Expectations: 186 149 Below Expectations: 1 1 Unsatisfactory: 0 0 Total Evaluations: 2,048 1,909 ___________________________________ 2008/9 2007/8 > 80%: 88.0% 78.0% 60 – 79%: 12.0% 22.0% < 60%: 0.0% 0.0% ___________________________________ Participation: 2008/9 2007/8 Year 1: 100% N/A Year 3: 100% N/A ___________________________________ Participation: 2008/9 2007/8 Year 3: 100% N/A

√

____ √

____ √

____

√

CACMS/LCME UT Follow-up Report, Appendix 1.B 5. Health Advocate/Community Resource

The medical graduate will be able to:

Curriculum Objective Performance Measures Achieved Results OK?

1. Describe the determinants of health and principles of disease prevention and behavior change appropriate for specific patient populations within a community and internationally, and apply these to patient care responsibilities and broader patient care initiatives. ____________________________________ 2. Define and describe a population, its demography, cultural and socioeconomic constitution, circumstances of living, and health status; and understand how to gather health information about this population in order to better serve its needs.

Clerkship Ward Evaluations: - Recognition of Important Determinants of Health and Principles of Disease Prevention - Patient Advocate _________________________________ Determinants of Community Health (DOCH) examination: - Performance on the Epidemiology components of the DOCH 1 examination

2007-9 2006-8 Outstanding: 417 350 Exceed Expectations: 1,129 1,055 Meet Expectations: 263 246 Below Expectations: 0 0 Unsatisfactory: 0 0 Total Evaluations: 1,809 1,651 2007-9 2006-8 Outstanding: 604 568 Exceed Expectations: 1,070 998 Meet Expectations: 330 311 Below Expectations: 0 0 Unsatisfactory: 0 0 Total Evaluations: 2,004 1,877 ___________________________________ 2008/9 2007/8 > 80%: 61.9% 79.6% 60 – 79%: 28.8% 19.6% < 60%: 2.2% 0.9%

√

√

____ √

CACMS/LCME UT Follow-up Report, Appendix 1.B 5. Health Advocate/Community Resource (cont’d)

The medical graduate will be able to:

Curriculum Objective Performance Measures Achieved Results OK?

3. Respect diversity, be willing to work through systems, collaborate with other members of the health care team, and accept appropriate responsibility for the health of populations.

____________________________________ 4. Participate in community activities directed at improving health, utilizing the best evidence, effective teamwork and communication skills.

Clerkship Ward Evaluations:

- Provision of Patient Care in Collaboration with all Health Care Providers - Recognition of Important Determinants of Health and Principles of Disease Prevention _________________________________ Clerkship Ward Evaluations: - Recognition of Important Determinants of Health and Principles of Disease

Prevention

- Patient Advocate

2007-9 2006-8 Outstanding: 730 692 Exceed Expectations: 947 880 Meet Expectations: 160 136 Below Expectations: 0 0 Unsatisfactory: 0 0 Total Evaluations: 1,837 1,708 2007-9 2006-8 Outstanding: 417 350 Exceed Expectations: 1,129 1,055 Meet Expectations: 263 246 Below Expectations: 0 0 Unsatisfactory: 0 0 Total Evaluations: 1,809 1,651 ___________________________________ 2007-9 2006-8 Outstanding: 417 350 Exceed Expectations: 1,129 1,055 Meet Expectations: 263 246 Below Expectations: 0 0 Unsatisfactory: 0 0 Total Evaluations: 1,809 1,651 2007-9 2006-8 Outstanding: 604 568 Exceed Expectations: 1,070 998 Meet Expectations: 330 311 Below Expectations: 0 0 Unsatisfactory: 0 0 Total Evaluations: 2,004 1,877

√ √

____ √

√

CACMS/LCME UT Follow-up Report, Appendix 1.B 5. Health Advocate/Community Resource (cont’d)

The medical graduate will be able to:

Curriculum Objective Performance Measures Achieved Results OK?

5. Describe the importance of the individual physician/patient relationship, and develop it appropriately, as a means to identify and implement individual health and disease management strategies on an individual basis.

Clerkship Ward Evaluations: - Communication with Patients, Families and Community - Patient Education - Patient Advocate

2007-9 2006-8 Outstanding: 945 934 Exceed Expectations: 922 831 Meet Expectations: 180 146 Below Expectations: 0 1 Unsatisfactory: 0 0 Total Evaluations: 2,047 1,901 2007-9 2006-8 Outstanding: 523 510 Exceed Expectations: 1,069 970 Meet Expectations: 216 201 Below Expectations: 0 2 Unsatisfactory: 0 0 Total Evaluations: 1,808 1,683 2007-9 2006-8 Outstanding: 604 568 Exceed Expectations: 1,070 998 Meet Expectations: 330 311 Below Expectations: 0 0 Unsatisfactory: 0 0 Total Evaluations: 2,004 1,877

√

√

√

CACMS/LCME UT Follow-up Report, Appendix 1.B 5. Health Advocate/Community Resource (cont’d)

The medical graduate will be able to:

Curriculum Objective Performance Measures Achieved Results OK?

6. Be prepared to challenge clinical orthodoxy, or identify threats to population health and advocate for their amelioration.

Clerkship Ward Evaluations: - Recognition of Important Determinants of Health and Principles of Disease Prevention - Patient Advocate

2007-9 2006-8 Outstanding: 417 350 Exceed Expectations: 1,129 1,055 Meet Expectations: 263 246 Below Expectations: 0 0 Unsatisfactory: 0 0 Total Evaluations: 1,809 1,651 2007-9 2006-8 Outstanding: 604 568 Exceed Expectations: 1,070 998 Meet Expectations: 330 311 Below Expectations: 0 0 Unsatisfactory: 0 0 Total Evaluations: 2,004 1,877

√

√

CACMS/LCME UT Follow-up Report, Appendix 1.B 6. Scholar

The medical graduate will be able to contribute to the following scholarly activities:

Curriculum Objective Performance Measures Achieved Results OK?

1. Research: The medical graduate will: a. be able to pose a research question, help develop a protocol, assist in carrying out research, and disseminate the results. _____________________________________ b. demonstrate an understanding of ethics as it relates to medical research. _____________________________________ 2. Education: The medical graduate will: a. demonstrate the ability to engage in life-long, self-directed learning and critical inquiry.

Determinants of Community Health (DOCH) course: - Performance on Individual Year 2 Research Project Learning Plan—the research protocol and plan of how they will complete this project over the year _________________________________ Determinants of Community Health: - Year 2 Research Project ethics protocol - Performance on the Research Ethics components of the DOCH 2 examination _________________________________ Clerkship Ward Evaluations: - Self-Directed Learning

2008/9 2007/8 > 80%: 55.0% 50.0% 60 – 79%: 45.0% 50.0% < 60%: 0.0% 0.0% ___________________________________ 2008/9 2007/8 Completed: 100% 100% > 80%: 95.0% 62.6% 60 – 79%: 0.0% 28.8% < 60%: 5.0% 8.6% ___________________________________ 2007-9 2006-8 Outstanding: 724 646 Exceed Expectations: 1,083 1,009 Meet Expectations: 242 256 Below Expectations: 0 0 Unsatisfactory: 0 0 Total Evaluations: 2,049 1,911

√

____

√

√

____ √

CACMS/LCME UT Follow-up Report, Appendix 1.B

6. Scholar (cont’d)

The medical graduate will be able to contribute to the following scholarly activities:

Curriculum Objective Performance Measures Achieved Results OK?

2. Education (cont’d): The medical graduate will: b. compare and contrast the diverse learning approaches of peers, patients, and others, in order to effectively interact and collaborate. _____________________________________ c. assist in teaching others and facilitating learning where appropriate. ____________________________________ d. understand the importance of being mentors to those less experienced members of the health care team.

Opportunities for senior students to act as co-facilitators with residents and learn about and apply diverse learning approaches have been implemented in several courses and rotations (e.g., ethics and professionalism sessions) Clerkship Ward Evaluations: - Patient Education _________________________________ Clerkship Ward Evaluations: - Contribution to Rounds, Seminars and Other Learning Events _________________________________ Opportunities for senior students to learn about and serve as mentors to other health professional students and junior medical students are being implemented in applicable courses and rotations, as well as through the Office of Student Affairs.

The evaluation of student understanding and application of diverse learning approaches will be evaluated via a portfolio system being introduced in Sept 2010 2007-9 2006-8 Outstanding: 523 510 Exceed Expectations: 1,069 970 Meet Expectations: 216 201 Below Expectations: 0 2 Unsatisfactory: 0 0 Total Evaluations: 1,808 1,683 ___________________________________ 2007-9 2006-8 Outstanding: 420 402 Exceed Expectations: 939 931 Meet Expectations: 233 214 Below Expectations: 0 2 Unsatisfactory: 0 0 Total Evaluations: 1,592 1,549 ___________________________________ The evaluation of student understanding of mentoring is being implemented through both course evaluations, and feedback from mentoring opportunities.

IPR √

____ √

____ IPR

CACMS/LCME UT Follow-up Report, Appendix 1.B 6. Scholar (cont’d)

The medical graduate will be able to contribute to the following scholarly activities:

Curriculum Objective Performance Measures Achieved Results OK?

3. Creative Professional Activity (cont’d): The medical graduate will: a. be able to describe the importance of, and contribute to professional innovations, creative excellence, and exemplary professional practice. ____________________________________ b. demonstrate leadership potential by participating in the development of professional practices, such as practice guidelines or health policy development, and participation in professional organizations.

Opportunities for students to learn about innovative creative professional activity and practice are being implemented in several courses (e.g., Determinants of Community Health) and new clerkship rotations (Transition to Residency Program- TTR) _________________________________ Opportunities for students to learn about and demonstrate leadership skills are being implemented in several courses and rotations. Faculty lead for leadership curriculum hired.

The evaluation of student understanding of creative professional activity and practice is being developed concurrently with new courses such as TTR, within a portfolio system. ___________________________________ The evaluation of student understanding and demonstration of leadership potential is currently under development with new course Faculty lead, and will include leadership module completion (e.g., CMA)

IPR ____ IPR

CACMS/LCME UT Follow-up Report, Appendix 1.B 7. Professional

The medical graduate will be able to:

Curriculum Objective Performance Measures Achieved Results OK?

1. Recognize and accept the need for self-care and personal development as necessary to fulfilling one’s professional obligations and leadership role. _____________________________________ 2. Demonstrate altruism, honesty and integrity and respect in all interactions with patients, families, colleagues, and others with whom physicians must interact in their professional lives. _____________________________________ 3. Demonstrate compassionate treatment of patients and respect for their privacy and dignity and beliefs. _____________________________________ 4. Be reliable and responsible in fulfilling obligations.

Clerkship Professionalism Evaluations: - Duty: Reliability and Responsibility _________________________________ Clerkship Professionalism Evaluations: - Altruism _________________________________ Clerkship Professionalism Evaluations: - Altruism _________________________________ Clerkship Professionalism Evaluations: - Duty: Reliability and Responsibility

2007-9 2006-8 Meet Expectations: 16,133 13,345 1-2 Minor Lapses: 24 32 Major Lapse(s): 2 5 Total Evaluations: 16,159 13,382 ___________________________________ 2007-9 2006-8 Meet Expectations: 12,645 10,564 1-2 Minor Lapses: 4 9 Major Lapse(s): 0 0 Total Evaluations: 12,649 10,573 ___________________________________ 2007-9 2006-8 Meet Expectations: 12,645 10,564 1-2 Minor Lapses: 4 9 Major Lapse(s): 0 0 Total Evaluations: 12,649 10,573 ___________________________________ 2007-9 2006-8 Meet Expectations: 16,133 13,345 1-2 Minor Lapses: 24 32 Major Lapse(s): 2 5 Total Evaluations: 16,159 13,382

√

____ √

____ √

____ √

CACMS/LCME UT Follow-up Report, Appendix 1.B 7. Professional (cont’d)

The medical graduate will be able to:

Curriculum Objective Performance Measures Achieved Results OK?

5. Recognize and accept the limitations in her/his knowledge and clinical skills, and demonstrate a commitment to continuously improve his/her knowledge, ability and skills and leadership, always striving for excellence._____________________________________ 6. Describe and abide by the University/Faculty codes of professional conduct, and the relevant professional regulatory requirements concerning medical practice. _____________________________________ 7. Describe the threats to medical professionalism posed by the conflicts of interest which can occur in the practice of medicine. _____________________________________ 8. Demonstrate a sound grasp of the theories and principles governing ethical decision-making, the major ethical dilemmas in medicine, and an approach to resolving these. _____________________________________ 9. Demonstrate an understanding of the principles and practice of law as they apply to the practice of medicine.

Clerkship Professionalism Evaluations: - Self Improvement and Adaptability _________________________________ Clerkship Professionalism Evaluations: - Upholding Student and Professional Code of Conduct _________________________________ Clerkship Professionalism Evaluations: - Duty: Reliability and Responsibility _________________________________ Ethics/professionalism/medical jurisprudence questions on multiple course examinations (e.g., Structure and Function) _________________________________ Ethics/professionalism/medical jurisprudence questions on multiple course examinations (e.g., Foundations of Medical Practice)

2007-9 2006-8 Meet Expectations: 12,608 10,499 1-2 Minor Lapses: 15 46 Major Lapse(s): 0 3 Total Evaluations: 12,963 10,548 ___________________________________ 2007-9 2006-8 Meet Expectations: 12,598 10,495 1-2 Minor Lapses: 8 2 Major Lapse(s): 0 0 Total Evaluations: 12,606 10,497 ___________________________________ 2007-9 2006-8 Meet Expectations: 16,133 13,345 1-2 Minor Lapses: 24 32 Major Lapse(s): 2 5 Total Evaluations: 16,159 13,382 ___________________________________ Ethics and jurisprudence short answer questions on three in-course exams Foundations of Medical Practice: 2007-2008 Class mean 86.4% SD 9.7% 2009-2009 Class mean 86.7% SD 9.4% Structure and Function: 2009-2010 Class mean 87.0% SD 11%

√

____ √

____ √

____ √

CACMS/LCME UT Follow-up Report, Appendix 1.B 7. Professional (cont’d)

The medical graduate will be able to:

Curriculum Objective Performance Measures Achieved Results OK?

10. Develop the capacity to recognize common medical errors, report them to the required bodies, and discuss them appropriately with patients.

Clerkship Professionalism Evaluations: - Duty: Reliability and Responsibility

2007-9 2006-8 Meet Expectations: 16,133 13,345 1-2 Minor Lapses: 24 32 Major Lapse(s): 2 5 Total Evaluations: 16,159 13,382

√

Notes:

* “OK?” signifies whether the objective is currently deemed by Program Evaluation Committee as: i) achieved (√), ii) indeterminate (±), iii) in progress (IPR), or iv) requiring follow-up by the Undergraduate Medicine Curriculum Committee † 2007-9 includes all clinical clerkship students evaluated during that period (most would have graduated in June 2009)

2006-8 includes all clinical clerkship students evaluated during that period (most would have graduated in June 2008)

‡ 2008/9 includes all applicable students evaluated in the 2008/2009 academic year

2007/8 includes all applicable students evaluated in the 2007/2008 academic year

CACMS/LCME UT Follow‐up Report – Appendix 2 

 

Appendix 2 Timeline for Review of Educational Program Objectives 

The current Faculty of Medicine, University of Toronto Undergraduate Medical Education Goals and Objectives are organized into seven groups corresponding to the CanMEDS roles (and in several cases the Four Principles of Family Medicine).  The total number of objectives is 40.    A core principle of curriculum management is the evaluation of the ways in which the institution's goals and objectives are being taught and learned, and the ways in which we are ensuring that student progress towards the achievement of these goals and objectives is being appropriately measured.  In order to accomplish such an evaluation on a regular basis, the Undergraduate Medicine Curriculum Committee has approved the following process and timeline for review of the educational program objectives:   A Curriculum Evaluation sub‐committee is being struck, and each year the committee will tackle 6 ‐ 10 of the objectives, so that in any 7 year period of time, all of the objectives will have been reviewed.  There are several natural "clusters" of objectives (e.g. all of the communicator objectives are relatively closely related) that will facilitate such a review.  Subgroups of the committee will each tackle one or at most a few of the objectives and will explore three issues: how is the objective being taught, what is the evidence that students are learning it, and how appropriate are the measures that are being used to evaluate their learning?  The answers to these questions will be identified by review of our electronic curriculum map, by reviews of course materials, and by focus groups with students and teachers.    Each subgroup will then prepare a report, which will summarize the findings, and provide recommendations as to how to address any deficiencies.    For Objective #1 ‐ 2 (Demonstrate a thorough knowledge of the etiology, pathogenesis, clinical features, complications, principles of prevention and management of common and life‐threatening illnesses presenting throughout the age spectrum, including all of the core clinical presentations outlined by the Medical Council of Canada) the committee will utilize input from subcommittees of content experts.  These will be separately constituted over the same 7 year cycle.  They will involve inviting subject experts to form a task force to report on two types of issues: teaching, learning and assessment of learning of topics such as "cancer", "cardiovascular disease" or "child health"; and, teaching, learning and assessment of learning of cross‐cutting areas such as medical ethics, pharmacology, occupational health and aboriginal health, among others.  The profile of these content areas will be determined by the UMECC.