Supporting them through the residency application process...

of 17

© Faculty of Medicine, University of Toronto, 2014

Launching Our Students:

Supporting them through the residency application process

Physician Toolkit

August 2014

of 17


AUTHORS

This toolkit was developed for the Mississauga Academy of Medicine’s Faculty Development Program with thanks

to the authors:

Dr. Leslie Nickell, Associate Dean, Office of Health Professions Student Affairs

Faculty of Medicine, University of Toronto

Dr. Glen Bandiera, Associate Dean, Postgraduate Medical Education (Admissions and Evaluation)

Mississauga Medical Education Advisor,


Dr. Pamela Coates, Academy Director, Mississauga Academy of Medicine,


Dr. Jana Bajcar, Faculty Development Director, Mississauga Academy of Medicine,


ACKNOWLEDGEMENTS

The authors wish to thank the following individuals for their contributions to the development of this toolkit:

Alicia Lozon

Interim Director, Medical Education

Trillium Health Partners

Kristy Beaulieu

Project Manager, Medical Education

Trillium Health Partners

Lori Innes

Faculty Development Administrative Assistant, Mississauga Academy of Medicine


of 17


CONTENTS

AUTHORS ....................................................................................................................................................................... 2

ACKNOWLEDGEMENTS ............................................................................................................................................ 2

Overview & Objectives .................................................................................................................................................. 4

CaRMS Critical Timelines ............................................................................................................................................... 4

Key Facts About CaRMS Match ...................................................................................................................................... 4

Overarching Messages for Students: ............................................................................................................................. 5

Process and Key Facts About Submitting Letters of Reference ..................................................................................... 5

Tips for a Letter of Reference Conversation between Faculty and Student .................................................................. 6

Tips for writing an effective reference letter ................................................................................................................ 8

Sample Letters – Good letter & Analysis ..................................................................................................................... 11

Sample Letters – Poor letter & Analysis ...................................................................................................................... 13

Appendix A – Further Background Information .......................................................................................................... 15

of 17


OVERVIEW & OBJECTIVES

The Canadian Resident Matching Service (CaRMS) is a national service organization that provides fair and transparent application and matching service to medical students for entry into postgraduate medical training throughout Canada. Trillium Health Partners (THP) and the Mississauga Academy of Medicine (MAM) are committed to the future success of all learners. As the inaugural MAM class prepares to graduate in spring 2015, students will be preparing their CaRMS application packages in late summer/early fall. Letters of reference from physician preceptors are a key component of the student’s CaRMS application package and any teaching physician (at THP or in the community) may be approached to write a letter of reference. This toolkit has been developed, as part of the MAM Faculty Development Program, to provide critical context to physician teachers to effectively support our students through the residency application process.

CaRMS CRITICAL TIMELINES

August/September: application portal opens to 4th year students

November: completed student submissions due, including all Letters of Reference

January/February – interviews across the country

First week of March – Match Day

For exact dates, please visit: https://www.carms.ca/en/first-iteration-timetable

KEY FACTS ABOUT CaRMS MATCH

1. Student ranking of their preferred programs drives the match process. Students should always rank the programs in order of preference – programs will not know where they are located within a student’s rank order list (ROL).

2. In order to maximize chances of matching, students should rank all programs they would be willing to match to.

3. A match to a residency program is legally binding.

https://www.carms.ca/en/first-iteration-timetable

of 17


OVERARCHING MESSAGES FOR STUDENTS:

1. Apply broadly, ideally to more than one (1) discipline. 2. If applying to only one (1) discipline, apply to all programs across the country. 3. Even if applying to more than one discipline, apply out of GTA and out of province. 4. Utilize the career support services available (See Appendix A) to optimize their submission

package.

PROCESS AND KEY FACTS ABOUT SUBMITTING LETTERS OF REFERENCE

Each year, candidates upload documents to the CaRMS website (exam scores, personal letters, demographic information, CVs) and indicate which programs they wish to apply for. The candidates then request reference letters from supervisors. They can request as many as they wish. Each letter is assigned a unique reference request number and the candidate can ‘map’ each letter to a specific program. Some candidates may therefore ask a supervisor for more than one letter – each tailored to a specific program or specialty. The supervisor then completes the letter and logs into the CaRMS site to upload it, without showing it to the candidate. Letters of reference MUST NOT be shown to the student. Medical schools also upload official documents about the students’ performance to the site.

of 17


TIPS FOR A LETTER OF REFERENCE CONVERSATION BETWEEN FACULTY AND STUDENT

SHOULD YOU WRITE A LETTER OF REFERENCE?

If/when a student approaches you to write a letter of reference, first consider some of the questions/tips below which may support you in having a conversation with the student:

1. Can you meet with the student, preferably in person, to discuss the letter (before you write

it)? Meeting the student will likely trigger your memory of qualities you wish to emphasize and allow you to recall/discuss specific previous experiences or interactions that you may wish to detail in the letter.

2. Do you know the student well enough and has your experience with the student afforded you

an appropriate opportunity for assessment? Your exposure and/or experience with the student should be sufficient to allow you to provide specific examples to support your statements and comment on the areas most likely to interest the applicable program directors, who will be reviewing student applications. If you have provided formal evaluations for the student in the past, it can be helpful to ask for copies of those evaluations to remind you of what you previously said about the student.

3. Can you ask other colleagues who have worked with the student for suggestions of qualities

to highlight? Other physicians who taught the student and/or members of the interprofessional team who interacted with the student during their rotation might have valuable insights that would support a robust letter.

4. Do you have enough time to write the letter? Writing letters can be a time consuming process,

do not overcommit and then not follow through. Whenever possible, try to make the time as these letters are very important to the students – this is a high-stakes situation.

5. Are you able/willing to write the letter? If you cannot, or do not wish to write the letter, it is important to be open with the student. You may still be able to help them by identifying more appropriate referees. It is critical to note that a ‘neutral’ letter or one that conveys the student was ‘average’ can result in significantly negative outcomes for the student. Be honest if this is the case and allow the student the opportunity to choose another letter of reference.

of 17


ONCE YOU’VE AGREED TO WRITE A LETTER OF REFERENCE

1. Make notes at the time that the student makes the request. Keep these on file to consult and

support your memory when you write the letter later on.

2. Ask the student which program(s) the student will direct the letter towards (e.g. Is the student asking for a letter that will be used to apply to Paediatrics or Family Medicine OR will the letter go to both?).

3. Ask for a highlighted copy of the student’s CV and a summary outlining the times you have

taught the student, reminding you of special areas of interest, cases worked on, procedures performed, etc. This is particularly important if you are not able to meet directly with the student.

4. Ask the student to keep you updated on how their CaRMS application works out. CaRMS

applications can be stressful, so take the opportunity to confer words of encouragement and demonstrate your interest in their progress.

of 17


TIPS FOR WRITING AN EFFECTIVE REFERENCE LETTER

1. Write letters only for those students who you have personally interacted with and about

whom you can generate an informed opinion. Clarify with the student the context from which you will be writing the letter, and see if this fits with what is being asked by those requesting the reference, i.e. the student wishes a clinical reference, but you only know them from research involvement. If you feel you cannot provide the letter the student is asking for because of limited contact, suggest alternative individuals or approaches.

2. Help the reader understand what you are basing your letter on. In what context did you work with the candidate? For how long? What clinical or academic activities did you engage in? What degree of ‘face time’ did you have with the candidate during the rotation? Were you the only supervisor or were others involved in the assessment?

3. Briefly tell the reader about you. One or two sentences are enough. Describe your relevant

practice, how long you have been in your role, how many learners you have worked with at a similar level, and (if you have written more than 5 references letters before) how many?

4. Describe your impressions of the candidate based on your own observations. If you solicit

opinions of others to help you write the letter, indicate this in the letter.

5. Provide a broad assessment of the candidate that is relevant to the target programs (if known) and that cover the CanMEDS roles. CaRMS provides general guidance. In 2014, they specifically suggested commentary on: o Cognitive skills and knowledge o Problem solving and patient management o Behaviour and attitudinal skills o Communication skills and working relationships o Motivation and punctuality o Sense of responsibility o Procedural (or other) skills specific to the discipline (yours or the ‘target’ specialty) o Special qualities and unique contributions o Other areas include: receptiveness to feedback, ability to work in a team, self-awareness

and insight and knowledge of the specialty

6. Provide explicit examples to support your opinions. These may include: o Specific examples of cases seen/managed o Lists of procedures you have observed the candidate do o Specific interactions with health care providers or others where candidates demonstrated

behaviours of interest o Anecdotes about your interaction with the candidate

of 17


o Quotes or paraphrases of things others have said about the candidate (ensuring you indicate that the source of your information is input from others)

o Examples of the candidate’s work that has an impact you can comment on (For example: “Bill presented rounds on disease XXX and challenged us all to change how we approach it.“

7. If the candidate has asked for a letter in support of an application to a specific program, make

your comments relevant to that program and indicate this explicitly (e.g. procedures for surgery, multitasking for emergency medicine, adept at spatial integration for radiation oncology, etc.). It is important to clarify this directly with the student, as they may not have thought about this aspect. Students may ask you for several letters each for a different program.

8. If the candidate has asked for a general letter (one that is applicable for multiple disciplines or programs) then comment on general competencies, listing specific examples and indicate the breadth of fields you could envision the candidate succeeding in.

9. Include a balance of strengths and weaknesses. Nobody is perfect and a great resident is one

who can learn from their mistakes and be self-directed. Describe if/how you believe the candidate performs in this regard with examples. This allows the reader to see how the candidate responds to feedback or mistakes and shows you are honest and not ‘rosy-eyed’ about the candidate. Indicate if and how the candidate rectified weaknesses during the rotation/experience.

10. Compare the candidate to their peers/your previous learners at similar levels. Are they ahead

or behind? Are they performing like somebody more advanced than they? Where in the spectrum of learners you have taught would you rank them (e.g. ‘top 5-10%’ - if true)?

11. If the candidate provides you with a CV, resist the urge to reiterate parts of the CV again in your letter. The application will contain their CV and space in your letter is precious. Do not reference things that you did not already know about by virtue of working with the candidate. “I notice on the CV that they are a wrestler. They must be very agile.”

12. Indicate other relevant qualities or accomplishments you are aware of outside of medicine.

Did they share their outside interests? Do they have relevant accomplishments (sports, arts, etc.) that you have seen impact their performance in medicine (leadership, inquisitiveness, social activism, educational activities, etc.)? Do they demonstrate attention to wellness and balance?

13. Make a definitive summary statement about your impressions: Would you want to work with

this person? Would you want them in your discipline? Would you want them in your residency program? Are they acceptable or highly competitive based on your assessment of the field? Don’t comment on the “field” unless you have been able to demonstrate knowledge of the field

of 17


when introducing yourself. Avoid telling the reader what to do: “I suggest you rank this person highly.”

14. Be concise. Give just enough information to justify your point and convince the reader you have evidence to back up your view. One page is a good target for length, and effective letters rarely will be more than 1.5 pages.

15. Your role is to help the reader make an informed decision, not necessarily to ensure the

candidate gets into a given program. Your opinion on the nature, quality and relevance of the candidate’s attributes are what are important.

of 17


SAMPLE LETTERS – GOOD LETTER & ANALYSIS

September 29, 2011

Dear Program Director and Selection Committee Members,

Re: CaRMS Application Ref: Mr. John Doe

I had the pleasure of supervising John Doe on a one month emergency medicine clerkship elective at Hospital X in August 2011. For context, our department sees a diversity of patients across the age spectrum, as well as a large immigrant and ethnically diverse population. We are also a regional cardiac and stroke centre, but we do not see major trauma. I have been a staff emergency physician for 15 years, and have supervised approximately 6 clerks per year. John worked 6 shifts with me; I have also solicited feedback from colleagues with whom he has worked. I will not provide Mr. Doe a copy of this letter.

John stood out as somebody who has a clear interest in Emergency Medicine. He is able to establish effective rapport with a diversity of patients. He repeatedly demonstrated good bedside manner and effective team interaction, which were spontaneously commented on by our nurses. For example, on one occasion he was heard patiently guiding a speech-impaired patient through a difficult history, reassuring him that we would attend to their concerns. Upon completion of his assessments, John’s summaries and plans demonstrate a sound knowledge base in Emergency Medicine and diagnostic reasoning at the level of a good EM PGY1. He was particularly knowledgeable around acute respiratory and abdominal presentations and chronic medical conditions such as diabetes. One area that was noted to require attention was his ECG interpretation and, to his credit, he acknowledged this and spent extra time focusing on a methodical approach. His skills in this area improved appreciably over the rotation, and became better than average. He is very good at procedures; I watched him independently undertake wound repairs (including adhesives, staples, simple suturing and a complex facial repair), CPR, and abscess incision and drainage. He also has a good degree of common sense and can tolerate ambiguity well.

John demonstrated a strong sense of professionalism and commitment to his patients. He was punctual, introduced himself to all members of the team, and demonstrated appropriate humility as he became accustomed to the environment. He also demonstrated a strong and appropriate sense of confidence in his abilities. He was quick to acknowledge when he wasn’t sure about something or wanted extra help/supervision. John seemed quite comfortable in the ED environment, working at a solid pace, often following several patients at a time without neglecting to follow-up on any, and engaging in social conversation with the team when appropriate. He is socially aware and active, and seemed quite interested in a diversity of opinion. He was keen to see complex patients and handled several sensitive patient presentations well (intoxicated patient who needed calm encouragement to stay, and an angry patient who waited longer than they wanted to for care). He followed through on all of his patients beyond the end of shift and participated impressively well in handover of his patients.

of 17


I am convinced John has a sincere and deep interest in emergency medicine but we did not get the chance to talk in depth about specific academic interests. I know he has done research in the area of pediatric compliance and he has a breadth of interests within emergency medicine at the moment.

Overall, I found Mr. Doe to be a committed, bright, and thoughtful clerk who can integrate well and quickly into new environments. He can adapt to a variety of situations and I felt confident that I could count on him to treat his patients well and practice up to, but not beyond his limitations. I think he will make an excellent emergency physician and would rank him in the top 10% of clerks that I have worked with at his level.

Sincerely,

Dr. Jane Doe

Analysis of Good Letter

This is a strong letter about a strong candidate. The letter starts by providing exact context of the

experience as well as the perspective and experience of the writer. It details the nature of the clinical

exposure and the role others had in crafting the letter. It then details the candidate’s clinical

performance with clear examples, including an area he improved upon, linking it to his receptiveness to

feedback. It also notes spontaneous nursing commentary on his communication skills. The next

paragraph talks about his personality and professionalism. Explicit examples are given such as his ability

to monitor his own limits. The author uses words like ‘keen’ and ‘well’ to emphasize the quality of the

candidate’s performance, rather than simply documenting what he did and did not do. The next

paragraph acknowledges the limits of the author’s ability to comment, thus avoiding the reader making

the assumption that absence of comment means absence of ability on the candidate’s part. The letter is

highly specialty-specific and deals with some nuance of the specialty. It finishes with a couple of strong

statements about his performance relative to other students and the confidence the author has in him.

of 17


SAMPLE LETTERS – POOR LETTER & ANALYSIS

September 29, 2011

Dear Program Director and Selection Committee Members,

Re: CaRMS Application Ref: Mr. John Doe

I am writing this letter in support of Mr. John Doe’s application to your postgraduate residency training program. Mr. Doe completed a clinical rotation with me in emergency medicine as an elective student in August of 2011. During this time he shadowed me in the department when it was busy and participated in a wide range of patient care commensurate with his abilities. As per my personal policy, John will not receive a copy of this letter.

John did well in his Emergency Medicine experience. He was able to undertake good histories and physical examinations, and he was able to appropriately present his findings and plans. He seemed to particularly like the more medical cases although I noted that he was also good at the typical range of procedures we assign to students in the department. One area that was noted to require attention was his ECG interpretation. This got better over the rotation. John also demonstrated good problem-solving abilities. He saw and managed several complex patients and sensitive patient presentations.

John was punctual and attended all scheduled shifts and teaching sessions. He followed through on all of his patients, often staying beyond the end of shift to do so. He saw a reasonable number of patients, and often followed several of them at any one time. He demonstrated good interpersonal skills; I received no complaints about his teamwork or conduct in the department.

John has a number of outside interests. I note from his CV that he is a rock climber and helped to set up a community athletics program for disadvantaged youth. He also volunteers at a local nursing home to

provide companionship to the elderly. On top of all of this he is playing varsity hockey. I am not sure I would have managed all of that when I was a student!

Overall, I found Mr. Doe to be a bright and thoughtful student who will do well in any residency program. I think he has what it takes to succeed and will deliver good care to his patients. I recommend him to you for consideration of admission in the upcoming recruitment session.

Analysis of Poor Letter

This is a weak letter about a strong candidate. On first read, it can seem positive, but most program directors

would see this letter as minimally contributory and possibly even as a negative since it does not distinguish the

student’s performance. The reader has no idea about the nature of the elective (even what level the student was

of 17


at- the reader would need to compare this to the CV

to find out).The next paragraph describes behavior that would be relatively common among clerks and thus of

limited value (if the author meant to describe clear strengths in

these areas, this doesn’t come across). Was his ECG interpretation better because of teaching, self-study or

passage of time and is it eventually acceptable? The author uses terms like ‘reasonable’, and ‘good’, which all but

the poorest students would be. Valuable space is used up repeating information in the CV, which the program will

also receive and there is little connection between these activities and the candidate’s performance. Finally, the

fact that the author feels the candidate would do well in any residency may seem like a glowing accolade, but in

reality, if he is good for any program, he is not likely great for any program and it seems like the author hasn’t

thought much about the candidate’s specific strengths.

of 17


APPENDIX A – FURTHER BACKGROUND INFORMATION

CaRMS Background

Every year, Canadian medical students in their final year apply for residency positions that start the following July. Each candidate competes for positions in their program(s) of interest with all other graduates of Canadian medical schools and some from U.S. medical schools. There is a separate quota of positions set aside for international medical graduates (IMGs). This is all done through the Canadian Residency Matching Service (CaRMS), an on-line computerized match. CaRMS sets the timelines and implements policy decisions around eligibility and conduct of the match in collaboration with the Association of Faculties of Medicine of Canada (AFMC). Applicants and programs both pay fees to CaRMS to maintain and operate the system.

Ministries of health across Canada establish the number and distribution of funded residency positions in each province in collaboration with the relevant medical schools’ Postgraduate Medical Education Deans. In Ontario, there are six schools offering residencies (some provinces, such as B.C., have only one school, while some schools, such as Dalhousie, work across several provinces). Once schools and Ministries agree on positions, these are posted on the CaRMS site for candidates to apply to. Once the match is complete, the candidates and programs are contractually obligated to honour the assignment(s) they are given. Candidates in rare occasions can request to transfer out of a program later on in training, but it is expected that those entering residency programs will complete the program to which they have matched.

CaRMS Process

The timeline for each year’s match is available on the CaRMS website: www.carms.ca. Candidates upload documents to the website (exam scores, personal letters, demographic information, CVs) and indicate which programs they wish to apply for. Most candidates will apply to 10 or so programs across two or three specialties. The candidates then request reference letters from supervisors. They can request as many as they wish. Each letter is assigned a unique reference request number and the candidate can ‘map’ each letter to a specific program. Some candidates may therefore ask a supervisor for more than one letter – each tailored to a specific program or specialty. The supervisor then completes the letter and logs into the CaRMS site to upload it, without showing it to the candidate. Medical schools also upload official documents about the students’ performance in medical school to the site.

http://www.carms.ca/

of 17


Programs log into the CaRMS sites after the deadline for uploading documents has passed. The program will have access to all documents on the website. Some programs feel strongly that each component of the application be assessed independently from the other components. The program may thus decide to print or download the documents and give each type of document (letters, CVs, etc.) to different members of the selection committee to assess. Therefore, committee members who are assessing a reference letter may or may not have also seen or assessed other components of the application package. Assessors will make an assessment and ‘score’ the application package in one of two ways. Some programs use the application as a means to say ‘yes’ or ‘no’ to an interview and others score each application on a continuous scale and interview all candidates above a certain threshold (or interview a fixed number of applicants, going down through their application scores until all interview positions are filled). The latter group usually will factor the application score into the final ranking along with the interview scores, whereas the former model involves basing the final rank on the interview only, with the application having served its purpose in identifying interviewees in the first place.

Ranking Decisions

Once all interviews have been completed in early February, candidates rank all the programs that they are prepared to accept, in the order in their order of preference. Programs do the same for their preferred candidates. Programs are expected to have a process in place for scoring the applicants and deriving their rank order list based on the goals and outcomes the program is trying to achieve. The rank order lists are then submitted to CaRMS and the match is run in March.

In the first iteration of CaRMS, not all programs fill and not all candidates are matched to a position. About 85% of candidates get matched to one of their first three choices and about 5% are unmatched each year. The unmatched spots and candidates can enter the second iteration of CaRMS, which is completed approximately six weeks following the initial match results. In the second iteration, all candidates, regardless of graduating school (Graduates of Canadian schools and International Medical Graduates) compete against each other for the available positions. In preparation for the second submission, candidates may request a new reference letter from you, for the new program(s) to which they are applying.

Top Reasons for Students Not Matching

1. Too few programs/disciplines applied to or ranked 2. Narrow focus on highly competitive programs 3. Limited geographical location 4. Unrealistic career planning strategy 5. Over-confidence in their own competitiveness 6. Poor communication/interviewing skills 7. Limited or no utilization of career counselling resources

of 17


What Happens to the “UnMatched” Students?

1. Enter Second Match Iteration: During the second iteration of the match (done in mid-April), programs with remaining space are reopened to the match. In this second iteration, there are therefore limited choices available in terms of program, discipline and location. Additionally, there is increased competition as IMGs, Canadian’s Studying Abroad (CSAs), Resident Transfers and Re-Entry applicants are included in the second iteration.

2. Take one (1) year off and re-enter CaRMS first iteration the following year: During this year off, students cannot work clinically in any residency program, but can look for research opportunities, Masters Programs and/or shadow opportunities. A number of schools are offering post-MD programs, or extended MD programs, allowing the unmatched students to participate in some clinical electives; however, this opportunity is limited and not available to all unmatched students.

Supporting them through the residency application process...

Documents

Transcript of Supporting them through the residency application process...