Written Assessment PPT

8/7/2019 Written Assessment PPT

1/18

Newcastle University

School of Medical Sciences Education Development

Certificate in Clinical Education

Principles and Practice of Teaching (MED 8058)

Name: Matthew Rowett

Student number: 099066110

Date: 8th January 2010


2/18

Contents

Section Title of section Page

i Autobiography 1

ii First lesson plan (observed session) 2

Second lesson plan 4

Third lesson plan 7

iii Tutors report 9

Critical reflective account of observed session 11

iv Methods of evaluating teaching 12

Examples from practice illustrating development in the use ofevaluation.

13

SMART plans for future development in the use of evaluation. 14

References 15

Appendix 16


3/18

i. Autobiography.

I have been a psychiatry trainee for six and a half years and entered this speciality

from my pre-registration house officer year. I am now a registrar in general adult psychiatry.

I am currently working in a liaison psychiatry post. I work into James Cook university

Hospital in Middlesbrough. As a multidisciplinary team we provide mental health care for

patients with physical health problems or episodes of harming themselves intentionally. A

key role of the liaison psychiatrist is the education of other health professionals about the

interface between physical and mental health. This involves regular teaching of junior

doctors, medical students and nursing staff.

I have always been interested in medical education and have developed other opportunities to

teach. I have been involved in the production of training videos for examiners of the Royal

College of Psychiatrists practical examinations, I teach on a revision course for physicians

preparing for the first part of their professional examinations, I facilitate a seminar for dental

students about mental health once a year and I am currently developing a training package for

the voluntary sector in conjunction with a mental health charity. I also teach junior doctorsand medical students as part of the local medical development programme when required.

I plan to develop my role in medical education and eventually be in a position to plan and

deliver a programme of education to undergraduate and postgraduate students.

(Word count 233)


4/18

ii. Lesson Plans

The use of antidepressants in palliative care Observed session.

Aims:

y To familiarise Macmillan nurses with the medical treatment of depression in palliative care.

Learning outcomes:

y By the end of the session the learner will be able to discuss the appropriate options for the pharmacologi

palliative care.

y By the end of the session the learner will be able to state what augmentation strategies are available.

y By the end of the session the learner will be able to name where to find information about the doses of th

Time Topic Teacher activity Learner activity

14:30-

14:35

Introduction -Introduce self, title of session, aims and objectives.

-Outline structure of session.

-Ask student to introduce themselves (round robin).

-Groundrules All questions welcome etc...

-Listen to introduction.

-Introduce themselves.

14:35-

14:45

Activating /

assessing

prior

knowledge

Start interaction by asking learners to tell me what they

already know about antidepressants. Write this on the

flipchart. (If it appears the group is particularly quiet in

the introductions I might split them into smaller groups

for this task).

Recall prior knowledge abou

antidepressants and recount

this to the group.

14:45-

14:55

Mini-

lecture

Mini-lecture introducing principles of choosing

antidepressants, the characteristics of the groups of

antidepressants, a small formulary of antidepressants andthe strategies for augmentation.

-Listen to lecture.

-Complete partial handout.

14:55-

15:05

Group

exercise

Split group into groups of about 3 and give them

scenarios to consider treatment options (syndicate

groups)

Use information from mini-

lecture to choose treatment

options.


5/18

15:05-

15:20

Group

discussion

Ask each group to explain their chosen treatment.

Encourage the other groups to constructively comment on

the chosen treatment. Correct any areas of

misunderstanding, (This directly relates to the first

learning outcome). Ask about strategies if this doesntwork, (Tests second learning outcome).

In turn each group to explain

their choice briefly and then

to discuss their choice with

the larger group.

15:20-

15:25

Summary Recount key principles. Point out sources of further

information (on partial handout, e.g. NICE guidance and

BNF). Go back to learning outcomes and state how these

have been covered.

Listen to summary.

15:25-

15:30

Evaluation

of teaching

Ask learners to fill out evaluation forms. Fill out evaluation forms.

Teaching methods: Teaching was run as a small group interactive session, using a mini lecture and case based d

to both acquire information and then be able to apply this in order to meet the outcomes.

Assessment of learning: Learning was assessed informally through learners comments in the group discussion.

Rationale for teaching methods: Learning here was in the cognitive domain. Two outcomes concerned the ac

recalling this. The first outcome listed concerned the assimilation of this knowledge with prior learning and ap

context. The teaching methods were therefore varied to include the delivery of new information and providi

apply this. The group of nurses had some prior knowledge of antidepressants so in order for the session to be r

interactive approach. This activated their prior knowledge and enabled the learners to ask questions and discu

was a large amount of information to be delivered however I wanted to spend most of the time in discussion

knowledge and principles gained to their practice. I therefore included a mini-lecture near the beginning of th

handout to guide note taking to provide material for the later tasks of formulating their own management plan.


6/18

Whats it all about? - Why do psychiatrists ask the questions they do?

Aims: To increase the level of understanding about the aims of the psychiatrist in psychiatric consultation and th

Learning outcomes: By the end of the session the learner will be able to describe and discuss some of the influenstyle of the psychiatric consultation.

Entry behaviour: To have had some experience of the psychiatric interview, either as a medical student, service


10:45

11:00

Introduction

and

brainstorm

The facilitators introduce themselves. Outline structure of

session (Discussion, watch video, lunch at 12:00)

Ask learners to put on name badges.

Ask the group to split into three and write down what

they want from the session on flipchart paper. Put thesesheets on wall when completed.

Listen to introduction and pu

on name badges.

Split into 3 groups and write

on flipchart paper what they

want to achieve.

11:00

11:10

Stateobjectives

Use the groups comments to identify and discuss whatwill be covered, why and what is for another session.

Discuss with facilitators whwill be achieved in the

session.

11:10

11:25

Group

introduction

and

guidelines

Ask group to introduce themselves in a round robin and

give one guideline for the session each, e.g. We only

give constructive comments.

State guidelines will be up during lunch break and can be

added to then.

Each learner to state their

name and a guideline.

11:25

11:40

Groupwork.

Questions

Split group into 2 (one do and one should). Ask

groups to think about What questions should / do

psychiatrists ask? on their own for a minute. Then ask

the groups to discuss this between themselves and write

the questions down on flipchart paper.

Think of the questions. Then

discuss in groups and think o

5 questions for each do an

should group.


7/18

11:40

12:00

Groupwork.

Reasons

Ask the groups to swap their questions with the other

group. Then ask each group to write the reasons why

these questions are / should be asked.

Swap flipchart paper with

questions on. Consider these

questions and write down

why they think they

are/should be asked.12:00

12:30

LUNCH

Revisit

guidelines

Remind learners that guidelines can be added to on board

during lunch.

Mix with group. And talk about questions.

Eat lunch and discuss

questions they have come up

with.

12:30

13:00

Group

discussion

In large group look at the questions and use these as a

basis for discussion. Cover topics of:

-Multiple aims of the psychiatrist

-Structure of psychiatric history

-Language used

-Rapport-Diagnosis

-Understanding the patients experience-Different personalities

-Common aims with the patient-management

Talk about the reasons for th

questions and discuss as a

group.

13:00

13:20

Observerole play

Show short video and ask group to consider what thepsychiatrist is doing. Emphasise the doctor and patient

are ACTING! It is not meant to reflect real symptoms orreal practice.

Note down thoughts duringvideo, show twice if

necessary.

13:2013:40

Discussionof role play

Ask group to discuss their thoughts. Sit with group andencourage them to answer queries between themselves.

Discuss their thoughts aboutthe way the interview took

place within the group.

13:40

13:50

Summary /

assessmentof learning

Ask for points in a hat of one thing each person has

learned. Pick out the points and read aloud. Write thepoints on the board.

Write down learning points

and put them in the hat.

13:5014:00

Evaluationof teaching

Circulate evaluation forms. Must fill out to improvesession and receive certificate.

Complete the evaluation ofteaching.


8/18

Teaching methods: This was a longer session allowing time for a greater range of techniques. After the initia

with some group discussion work in smaller sub-groups. This led on to a short period of individual work to a

forming larger syndicate groups to consider opposing aspects of the question. This informed the interactive disc

of a psychiatric interview was then shown and discussed as a group.

Assessment of learning: This was done in three ways. Initially learning from the smaller group discussions

discussion informally through peoples comments. The application of this learning to a practical experience wa

about the observed psychiatric interview. Finally learning from the whole session was assessed by using a

identify and re-enforce their learning points.

Rationale for teaching methods: This session involved a lot of interactive discussion so initially small group

learners to engage in the process. This also allowed for a period of activation and assessment of prior knowledg

each groups work. As the group sizes increased it allowed a snowballing effect for the discussion points. A sy

two opposing views to be considered and allow a greater appreciation of the competing aims of the psychiatrist

two viewpoints which facilitated an interactive discussion. To reinforce the learning, make it more applicable tand provide a visual format a video of a psychiatric interview was shown and then discussed using points from t


9/18

Assessment of self harm.

Aims: To increase the confidence and competence of junior doctors working in medicine in assessing people w

Learning outcomes:

y By the end of the session the student will be able to list the risk factors for repetition of self harm.

y By the end of the session the student will be able to discuss the limitations of actuarial scales for risk ass

y By the end of the session the student will be able to list the essential questions they must ask the patient.

y By the end of the session the student will be able to name the physical investigations that should be perfo

y By the end of the session the student will be able to discuss the variety of factors that contribute to peopl

Entry behaviour: Foundation doctors in their first year as part of the Foundations of Clinical Practice program


10:0010:05 Introduction Introduce myself, the topic of the session and its format.Ask the group to form into three groups of four. Listen and then move chaform three groups.

10:05

10:20

Small group

work

Ask groups to discuss patients they have seen who have

harmed themselves. They should identify the issues

these patients raised for them and one case to discuss

with the larger group. While the groups are discussing

cases the teacher will move round the groups assessing

prior knowledge and identifying areas of need.

Discuss cases, the issues

and summarise one case t

present to the larger grou

10:20

10:50

Whole group

discussion, (to

be performed in

three cycles of

ten minutes).

Ask a group to present case and the issues raised. Ask

the groups to then answer these issues using their own

experience. Clarify any misunderstandings and facilitate

the discussion to cover the learning outcomes.

-Present the case.

-Reflect on the issues.

-Analyse the issues, (e.g.

did the patient refuse trea

-Plan for what could be dfuture (e.g. mental capaci

10:50

11:00

Summarise

learning

Recap of the learning objectives then a round robin of

the key learning points for individuals in the group.

State their key learning p

turn.

11:00 End session Highlight evaluation forms and further reading. (e-mail) Complete evaluation form


10/18

Teaching methods: This session used small group discussion and interactive case discussion based on learners p

group. A round robin at the end of the session summarised the main points and further reading was provided fo

Assessment of learning: Learning was assessed informally during the discussion and using the round robin form

Rationale for teaching methods: The session aims were to increase the learners confidence and competence. Th

experience so an interactive session based on the experiential learning cycle was used. The small group work ac

analysed the issues. The larger group work then allowed further analysis of the issues and planning for the futur

in the group, (consistent with the social constructivist theory).


11/18

iiia. Tutors report.

Strengths of the session:

y Lesson plan prepared and well structured. You reviewed the learning outcomes of the

session and discussed them in general terms.y You were in an unfamiliar environment and with unfamiliar learners. You did a quick

ice-breaker and moved into your presentation seamlessly.

y Good identification of background of learners. Your questions (concerning what anti-depressants they were familiar with) were general enough to stimulate safe responses

and gave you some idea as to their experience.

y It was helpful to investigate that they wanted to learn in the beginning. The morerelevant information is to adult learners, often the more successful the learning

experience is.

y Session organised and structured logically. You had an introduction to principles ofanti-depressants, reviewed different meds and then moved into interactive activities.

y

You demonstrated a confident and relaxed manner in your session.y You used lots of positive feedback when students responded (great, very good)

y You did a lot of work prior to this session. Your plan, handouts and case studies werea lot of preparation. Ideally, you can keep these resources in a teaching portfolio and

use them again.

y It was pitched at an appropriate level. They obviously had some (probably varied!) background in this area. Your questions pushed their understanding and required

them to apply this in a clinical setting.

y Lecture style teaching can be a very appropriate teaching strategy in certainconditions. You chose to use a mini lecture to review principles and characteristics of

anti-depressants in the beginning. This was entirely appropriate.

y It was a well controlled pace and you certainly kept to time. The structure of the

lesson was clear and the time allotted was well planned. You had to be flexible inyour plan as you were not sure how many students there would be or their exact

background. You did this flawlessly and created fantastic learning opportunities for

them.

y Your obvious interest and background in the area was a valuable asset for thestudents.

y You used varied teaching methods in appropriate ways-lecture, brainstorm, smallgroup work, case studies

y Good activation of prior knowledge and cued learning (what are the side effects oflorazepam?) This cued the students to recall the material that you knew they had (or

should have!) covered previously and put it in a clinical context.

y

You were approachable and non-threatening. According to Rogers and Maslow, this basic requirement is imperative to facilitate an environment conducive to learning.The students felt comfortable answering out loud and asking questions (One girl

asked Can I ask about 20mg dose of citalopram in an elderly woman?). I doubt you

would get this involvement if the students were threatened or intimidated by the

environment.


12/18

Points to consider

I hope you find the following comments helpful.

y You may have asked the students to share examples of cases concerning depression in palliative care they have seen have seen. It could have generated some interesting

discussion if you had asked for examples they had seen- either a present client or in thepast. Remember, one of Knowles (1980) basic assumptions of adult learners is that they

have real life experience to draw from. You might have attempted to facilitate this.

y You explained the topic, outlined what you wanted to cover and gave an overallintroduction. However, a brief explanation to the structure might have helped. You might

have explained there would be a mini lecture, group work, then a brief summary. In doing

this, the students knew what to expect and what was expected of them

y It was a well controlled pace, however, there could have been more time for questionsduring the lecture at the beginning. Asking questions can change timings, and I am awareyou were time constrained in this session! However, think about what you have to teach

them and what they could do on their own, or should already know. As we discussed, thestudents had some background knowledge on this subject. Instead of just transmitting

information, you could have increased interaction and activated prior learning by askingquestions i.e.

o What are the side effect of amitriptyline?- instead of listing themo "Why have you seen patients stopping anti-depressants?"

y Learners often value and can learn a lot from your experience. It may have beenworthwhile sharing more of your personal clinical experiences that were learning points

for you. It might have been helpful to include, for example, a case you remember inwhich someone at risk was identified and treated, or someone at risk who was missed and

the consequences.

y The session was well planned and organized. However, there seemed to be a bit of

confusion at the end of the group work. It was a little unclear who would be commentingon the groups treatment choice. After the first group went, you commented, but I think it

would have been more helpful, interactive and informative to you if the groups

commented on each others. In your lesson plan, you suggested that would be happening.

I think you were quite clear what you wanted and expected. However, the students

seemed to falter a bit. Plus, you were unfamiliar with the room and the number of

students- very difficult! Make sure you plan the activity and give clear instruction (or put

on flip chart or handouts) what you want the learners to do. Just keep this in mind next

time you are planning a group activity. That way everyone is doing what you want and

(hopefully) working cooperatively. Plus, it may make it easier for you and your time!

Overall, it was well taught. Your flexibility and preparation led to a successful experience.

I hope this feedback is helpful. Hopefully, you will learn from this experience!! Have youhave started reading for you learning module? If so, you may remember back to yourexperiential learning cycle . Identify the things that you need to learn from and plan what

you are going to do if the situation happens again.

Laura Delgaty


13/18

iiib. Critical reflective account.

Using my reflective diary and the tutor report I have identified areas for improvement.

When I started the session I stood at the front of the room with a flipchart. Instead I could

have used another member of the group as a scribe while I sat with the group. This would

have made the activation and assessment of prior knowledge more interactive. As suggested

in the tutor report this may also have been achieved by encouraging the learners to answer

questions during the mini-lecture. This would take longer but would build on prior

knowledge. Asking learners to share their experiences would also have helped.

The room was rectangular with a lot of furniture. I moved the desks to the side and put the

chairs in a circle, but removing the flipchart and using the board on the wall or removing

other furniture may have made the environment more comfortable. A more full explanation

of both the session structure and my expectations of the learners in the group exercise would

have made the session flow more easily as the learners would have known what was expected

of them.

To make delivery of information more accessible to the learners I could have used personal

clinical anecdotes to put facts into context and make the principles easier to remember.

In future sessions I will involve the group more by asking someone to be the scribe and using

the prior knowledge of the learners to answer each others questions. I will look closely at

how the room can be arranged before the learners arrive. I will use examples from my own

practice to illustrate points made. I will specifically examine the level of interaction when

reflecting on my teaching. I will also read resource material provided by the course on

promoting interaction. By doing this I will identify and be able to discuss ways of facilitating

interaction within a group.

(Word count 316)


14/18

iv. Extended essay

a. Methods of evaluation of teaching.

A search of the literature on evaluation of medical teaching reveals a number of sources

commenting on the characteristics of an effective evaluation tool, for example Morrison

(2003), Mohanna, Wall and Chambers (2004) and Snell et al (2000). With the aid of thesecriteria the advantages and disadvantages of different methods of evaluating clinical teaching

can be identified.

The two methods that will be discussed here are the use of a standard questionnaire produced

by the Royal College of Psychiatrists and keeping a reflective diary of teaching sessions.

Introduced in 2008 the Assessment of Teaching (AoT) is part of the Royal College of

Psychiatrists range of work-place based assessment (WPBA) tools designed to assess

competence. These are used as evidence in annual reviews to show competency progression.

The competency based curriculum of the Royal College of Psychiatrists has a number of

objectives specifically on teaching others and the AoT is mentioned as appropriate evidenceto show these have been achieved. It is therefore an important and widely used tool for

evaluating teaching among psychiatry trainees. A copy is provided in Appendix 1.

The AoT can be completed by anyone. It has ten items rated on a Likert scale linked to the

teachers stage of training. These are weighted towards using teaching aids, (three of the ten

items). There are no items rating content or learning outcomes. There is an area for

qualitative information which includes agreed action.

The advantages of this tool are that it is short with tick box ratings which means it is feasible

to complete at the end of a session and not overly burdensome. It collects both quantitative

and qualitative information. This allows data to be grouped for comparison and analysis, andalso elicits information in greater detail that is useful for the teacher to consider how to

improve sessions. Being paired with feedback increases the teachers motivation to use the

tool as the comments are directly relevant to him. The questionnaire applies to any psychiatry

trainee at any stage of their training at any site and is therefore able to be used widely. It is

specifically mentioned in the Royal College of Psychiatrists guidance on collecting evidence

to prove competency and is therefore directly linked to professional development. It can be

completed by anyone and therefore allows information from a range of sources to be

collected.

Disadvantages are that it has not been shown to be valid or reliable. Threats to the validity are

the narrow range of questions, distributed by the teacher allowing them to influence the

results and it is not completed anonymously. The reliability is adversely affected by the large

range of people who could complete the questionnaire, its reliance on assessing people

against their stage of training and the ratings are highly subjective using the completers

idea of what is acceptable as a standard. People are unlikely to know how teachers should

be performing or what their stage of training is. These factors mean cross comparison

between trainees is unreliable. There is no evaluation of how well the learning objectives


15/18

were achieved or if they were appropriate. This potentially allows a teacher to be scored

highly even though he did not cover the stated outcomes or the outcomes were unrelated to

learning needs.

This tool highlights some of the advantages of using questionnaires to evaluate teaching. A

questionnaire format is flexible and can be redesigned to fit its purpose and, as shown later,

many of the disadvantages of this tool can be addressed. Questionnaires can be used by a

range of interested parties, (teachers, course organisers, curriculum designers etc... [Snell et al

2002]), andtailored to their needs.

The use of a reflective diary fits with Kolbs theory of experiential learning and has the goal

of improving practice through learning from experience. It is relatively unstructured

compared to other methods of evaluation and, while this limits how conclusions can be

generalised to others, this makes it highly relevant to the individual. It can be tailored to the

learning objectives of the reflector and used as evidence for meeting goals of professional

development. It is flexible and can be used by all teachers in any situation. Reflective diaries

can be confidential and are directly relevant to the practice of the individual teacher. Thisprovides a high level of motivation for the teacher to buy-in to the process. One can also

include information from other sources, such as informal comments from students, as

material for reflection which increases its validity.

The individual nature of the reflective diary means it is of limited use to other interested

parties such as course providers. The burden of the time it takes to complete the process can

mean it is unfeasible in situations when time is limited. The reliance on qualitative data in

this method limits comparison to other teachers. There are no objective items for assessing

individual points, such as whether the teacher is addressing the learning needs of the student,

which results in a subjective account where significant areas for development can be missed.

b. Examples from practice illustrating development in the use of evaluation.

There were many disadvantages of the Assessment of Teaching tool from the Royal College

of Psychiatrists. When this was used it was found that some items were seen not to be

relevant and therefore not completed, the boxes for qualitative information were not

completed, (possibly because they were at the end and had confusing titles, for example

agreed action), and people were confused by the request for their registration number. An

alternative tool was therefore devised to address these issues and is shown in the appendix.

This new shorter tool directly assessed the learners perceptions of meeting the learningobjectives. It prioritised the qualitative data collection to maximise the completion rate and

used a Likert scale related to the learners agreement to assess factors related to the delivery

of the session. It included the statement I would recommend this session to a friend to

assess overall satisfaction with the session.


16/18

c. SMART plans for future development in the use of evaluation.

y Continue to use the newly developed tool to evaluate teaching after each session.

y Refine the tool to reflect the situation it is used in and the development of my

knowledge and experience in using evaluation tools.

y Use validated tools when appropriate, such as the L-PAST, to evaluate teachingsessions.

y Use multiple sources of information when evaluating teaching sessions.

y Request a copy of the evaluation conducted by course organisers on sessions I have

been involved with.

(Word count 1068)

References.

Morrison, J. (2003). ABC of learning and teaching in medicine: Evaluation. British Medical

Journal. 326:p385-p387.

Mohanna, Wall and Chambers (2004). Teaching made easy: a manual for health

professionals. Radcliffe medical press. Abingdon

Snell J, Tallett S, Haist S, Hays R, Norcini J, Prince K, Rothman A and Rowe R. (2000).

Medical Education. 34:p862-870


17/18

Appendix


18/18

The use of antidepressants in palliative care.

Teaching evaluation.

Were the objectives of todays session met?

1. To be able to discuss appropriate options for the

pharmacological treatment of depression in palliative

care.

Yes No

2. To be able to state what augmentation strategies are

available.

Yes No

3. To be able to name where to find further information

about antidepressants.

Yes No

Please state one thing you thought was good

about todays session.

1.

Please state three things you think would

improve the session.

1.

2.

3.

Strongly

agree

Agree Dont know Disagree Strongly

disagree

The session was

relevant to my

learning needs.

1 2 3 4 5

The approach

taken was

appropriate.

1 2 3 4 5

The content was

appropriate to

me.

1 2 3 4 5

The resources

were useful 1 2 3 4 5

I would recommend this session to a friend. Yes No

Written Assessment PPT

Documents

Transcript of Written Assessment PPT