“Strategies for Effective Clinical Teaching and Evaluation” Classroom/Clinical Assessment...

“Strategies for Effective Clinical Teaching and Evaluation”

Classroom/Clinical Assessment Techniques (CAT) (Part 3)

Patricia A. Mahoney, MSN, RN, CNE

This presentation is a portion of one presented at ASUCONHI May 2, 2007Used with permission of P. A. Mahoney – All rights reserved

http://www.asu.edu/

This Section Includes:

• One Minute Paper• Muddiest Point• Self Assessment• Group Work• Directed Paraphrase• Analogy

• Journaling• Writing• Simulation• Rubrics• Nursing Care Plans

Classroom/Clinical Assessment Techniques (CAT)

• Classroom assessment is both a teaching approach and a set of assessment techniques

• The more you know about what and how learners are learning, the better you can plan learning activities

• Simple, graded or non-graded, in-class activities that give both you and your learners useful assessment of learning

How is Classroom /Clinical Assessment Different?

• Classroom/Clinical assessment differs from testing/evaluation

• Aimed at course improvement, knowing where the learners are at

• Goal is to better understand your learners' learning and so to improve your teaching.

Classroom/Clinical Assessment

1. One ways to improve learning is to improve teaching. 2. To improve effectiveness, educators need to make

their goals and objectives explicit and then get specific, comprehensible feedback on the extent they are achieving those goals and objectives.

3. To improve learning, learners need to receive appropriate and focused feedback early and often; they also need to learn how to assess their own learning.

Classroom/Clinical Assessment :

4. Classroom Assessment does not require specialized training; it can be carried out by dedicated teachers from all disciplines.

5. By collaborating with colleagues and actively involving students in Classroom Assessment efforts, faculty (and learners) enhance learning and personal satisfaction.

http://honolulu.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/assess-1.htm

How do I use Classroom Assessment Techniques?

• Decide what you want to learn from a classroom assessment.

• Choose a Classroom Assessment Technique (CAT) that provides this feedback, is consistent with your teaching style, and can be easily implemented in your class.

How do I use Classroom Assessment Techniques?

Explain the purpose of the activity to learners, then conduct it.

After class/clinical, review the results and decide what changes, if any, to make.

Let your learners know what you learned from the CAT and how you will use this information.

Why should I use CATs?Provide short-term feedback about

the day-to-day learning and teaching process at a time when it is still possible to make mid-course corrections.

Provide useful information about learner with a much lower investment of time compared to tests, papers, and other traditional means of learning assessment.

Why should I use CATs?

Help learners become better monitors of their own learning.

Help break down feelings of anonymity, especially in larger courses.

Point out the need to alter study skills.

Provide concrete evidence that the educator cares about learning.

CATs

• Cross and Angelo recommend that CATs be ungraded

• We have found it more successful, especially early on, to "count" them. Even if only 1% of the final grade (perhaps with a grading scheme of check, plus, and minus) will encourage learners to take them seriously

• Techniques seem to work best when they are viewed as a source of feedback and not as a system for evaluating learner performance

• Achieving the right balance may take a bit of experimentation.

www.schreyerinstitute.psu.edu/Resources/class_assessment.asp

The One-Minute Paper Educator takes the last minute of class to ask• “What was the most important point made in

class today?”• "What was the most surprising or enlightening

moment in class today?” • “Name five significant points about _____.”• “How well did the discussions integrate with

the reading?” • “What unanswered question do you still

have?”

The One-Minute Paper

• The purpose is to elicit data about learners' comprehension of a particular class session.

• Helps alert when disjuncture occurs• Gives the timid learner an opportunity

to ask questions and seek clarification• Learning strategy & evaluation tool

What to do with the data

Review responses and note any useful comments

Next class periods emphasize the issues illuminated by your learners' comments.

Time required: o Preparation time:

Low

o In class time: Low

o Analysis time: Low

Angelo and Cross, P. (1993). Classroom Assessment Techniques, 2nd ed. San Francisco: Jossey-Bass, p. 148.

Muddiest Point

• Ask learners to write down what was least clear to them

• Requires learners to rate their own understanding across several topics

• Requires learners to ponder, if even momentarily, why one particular topic should be selected as leastleast understood

• Learning strategy & evaluation tool

Muddiest Point

• 25% of the class mentions the same Muddiest Point, might want to schedule added class time on the subject

• Might distribute an explanatory handout • Focusing on muddiest points too often can be

discouraging for both learners and educators because of the tendency to emphasize the negative.

Time required: o Preparation time: Low



o Can collect and scan perhaps 100 Muddiest Points in 15 minutes

Angelo, T. and Cross, P. (1993). Classroom Assessment Techniques 2nd ed. San Francisco: Jossey-Bass, p. 154.

Self Assessment

“How do I know what I think until I hear what I say?”

~ E.M. Forester

“Thinking about your thinking—become a critic of your thinking—discover your thinking, see its structure, observe its implications, and recognize its basics and vantage points.”

Paul (2000), p. xvii

Self Assessment

• Makes the learner privately confront personal attitudes, paradigms, and biases that may not have thought about

Self Assessment

• This technique is an excellent way to make visible different styles of thinking and different attitudes on controversial topics

• May want it to be anonymous

Self Assessment

• The educator presents students with alternative ways of looking at a controversial issue and asks them to indicate, by writing on a 3x5 card, which viewpoint applies to them and why.

• The responses are then shuffled allowing the overall results to be read publicly and discuss without compromising confidentiality.

Self Assessment

• Topics to consider

• Culture• Legal/ethical situations• Critical thinking

Self Assessment Questions:

• Which of the two statements A or B, more closely describes the person you really are inside?

• Which type of communicating, A or B, do you actually do more in your nursing practice? Why?

• Which kind of individual would you rather talk with in a purely social situation: A or B? Why?

A. "I don't take ideas for granted; I am a thinker. My

opinions are pretty well thought out, are based on objective experience, and have solid reasons supporting them. When somebody else offers an opinion, I like to play the devil's advocate and, for the purpose of uncovering and testing my thoughts, I often take the opposite position. I am sincerely willing to open my ideas to others for scrutiny and critique. Naturally, I often see flaws and weak points in opposing views because I have considered them previously. I enjoy a good argument and often learn a great deal from thinking about and responding to the tough critique of a worthy opponent."

B. "When I hear people voice opinions very different from my

own, I seek first to understand why and how they could possibly think that way. I try to look at the issue from their perspective, to get inside their heads, to walk in their moccasins, to see how their culture or world view could prompt them to express such values. Then, only after I can mirror back to them their view stated in my own words, do I try to explain my position. I learn a lot just from listening to people explain why they think and act as they do. They usually have good reasons for adopting a stance different than mine, and I find that so attractive that I often find myself rethinking my own position."

http://www.siue.edu/~deder/assess/cats/self5.html

An example I created

You are assigned to provide care for a client who has been in a continuous vegetative state for several years. The client is being sustained by enteral feedings. Her husband claims that his wife would not want to have her life sustained in the state by enteral feedings. However the client’s parents are insistent that the client, because of her religious beliefs (opposed to euthanasia) would want her life sustained.

A

“I believe that the client should have her life sustained since she did not have an advance directive and removing the tube would be against the client’s reported religious beliefs. I would refuse to remove the feeding tube.”

B

“I believe that the client’s feeding tube should be removed as this was the client’s wish. A client in a persistent vegetative state with no hope of recovery should not be maintained with a feeding tube. I would remove the tube if a medical order was written.”

Time required: o Preparation time: Medium to

High

o In class time: Low to Medium

o Analysis time: Low to Medium


Group Work

• Is a fact of life in nursing

• Active learning

• We all remember the high school version of group work where students who were regarded as less motivated parasitized the efforts of those who were more motivated

Group Work

• Group work assessment makes performance expectations visible to all. By reducing the threshold for expressing opinions, it allows students more readily to state the concerns – and recognize the strengths – that they collectively bring to a group.

Sample Form: Group Work Assessment

1. How many of the group members participated actively most of the time? ___/___

2. How many of the group members were fully prepared for group work most of the time?

___/___

3. Overall, how effectively did your group work together on this assignment?

Extremely Well Well Adequately Inadequately Poorly Not At All

4. Give one specific example of something you learned from the group that you probably wouldn't have learned on your own.

5. Give one specific example of something that other group members learned from you that they probably wouldn't have learned without you.

6. Suggest one specific, practical change the group could make that would help to improve everyone's learning.

http://www.siue.edu/~deder/assess/cats/grp13.html

Group WorkAssessment should really be used in the early-middle

of a project and again at the endAll groups have their disagreements; early

assessment can help make real problems visible before they fester into disasters

This technique requests positive as well as negative feedback; it is not a mechanism for producing blame.

A negative assessment without an effective answer to Question 6 has not addressed what this assessment is really about.

Time required:

o Preparation time: Medium




Directed Paraphrase

• The teacher asks the learner to summarize, in well-chosen words, a key idea that has been presented during the current or previous class period.

• The paraphrase part requires the learner to generate a new way to express the concept.

• The directed part specifies the [virtual] audience to whom the paraphrase is directed, thus revealing whether the learner understands the concept within the specified framework.

Directed Paraphrase

• A nursing learner might be directed to paraphrase the concept of drug clearance by the kidneys to a worried client

• The twin challenges are brevity and choice of language to match the needs of the specified audience


(if grading would require a rubric)

o In class time: Medium

o Analysis time: Medium


Analogy• Likeness or similarity between two things

that are otherwise unalike

• Enhances an understanding of complex concepts or abstractions by allowing learners to consider the concepts in a different context

Analogy

• Choose a work of art• Ask learners to assess it (fundamentals)• What do you see• Ask learners to list what they saw• Be sure not to ask for thoughts or feelings

Valiga, T., Bruderle, E. (1996). Using the Arts and Humanities to Teach Nursing: A Creative Approach. New York: Springer Publishing Co.

Analogy

• Can use literature, research findings, newspaper

• Can be used in a variety of settings (classroom, clinical, Web)

• Can be used with large groups, small groups, or as an individual assignment.

Analogy• Is an active learning strategy

• Stimulates critical thinking

• Does not require a great deal of preparation

• Facilitates learning of complex concepts

Analogy

• Does not appeal to all learners; some will not want to participate, and may consider the activity a waste of class time

• Some students have difficulty making connections in a meaningful way

• Use of literature for analogy can be difficult and time consuming for faculty


• if grading would require a rubric• finding appropriate material may be

difficulto In class time: Lowo Analysis time: Medium


Journaling

Journals are written dialogues between the self and the faculty

Journaling

• Engage the learner in transferring classroom discussions to clinical experience and allowing the evaluator to participate in the learner's experience from the learner's perspective. Using a triangular approach, whereby the student links personal and professional experiences with theory from the classroom and the literature, prevents students from writing what they think the teachers want to read.

Journaling

Clarify from the outset the guidelines for journal entries. These include the purpose and criteria for evaluation, which should be based on the clinical competency being assessed

Criteria for grading can be based on the level of analysis and synthesis in the entries which reflect the learners reading, classroom and life experiences

Journaling• This method of journal keeping will foster the

development of skills in introspection, reflection, and dialogue and provides an avenue for students to make sense of and to learn from their mistakes

• Faculty need to build a trusting relationship with the learner– responding in a sensitive manner to personal

disclosures and providing constructive written feedback on specific aspects of content rather, than global references to 'good work' (Heinrich, 1992).

Journaling, Clinical Logs

• Promotes reflection• Promotes critical thinking• Requires faculty feedback to be meaningful to

learner• Usually used to connect theory to practice• Faculty need to carefully design questions for

journaling• Can be done electronically

Time required: o Preparation time: Low to Medium

o In class time: Medium to High

o Analysis time: Medium to High

• Might consider grading the assignment


Writing

• Includes formal papers, research critiques, letters to editors/legislators

• Most effective in flexible in topic• Specific grading criteria needed• Grading can be subjective• Can have peers or faculty review draft• Time commitment for learner & faculty

Time required:

o Preparation time: Low Medium

o In class time: Medium to High

o Analysis time: High


Case Studies - SimulationsCan be used not only as a teaching tool,

but also to assess clinical competencies

Not only assess a student's ability to communicate in a logical clear and concise manner, but also assess the student's ability to present a holistic perspective of client care.

Case Studies - Simulations

Can also draw conclusions about the cognitive and affective domains

As with other forms of evaluation strategies, evaluation criteria should be set before the case study is written.

Learners should be informed of these details so that they clearly understand what is expected.

Time required: o Preparation time: Medium

o In class time: High

o Analysis time: High

o Might consider grading the assignment

Simulation

• Development is time intensive – primarily due to the detail that is required of script writing

• Set up and implementation requires the cooperation of multiple individuals

• If using SimMan it can be costly

• Learners need orientation/direction – instructions must be very clear

Simulation• Allows the learner to experience “real world”

patient situations without risks• Learners are required to assess and interpret the

situation, and make decisions based on information provided

• Usually conducted in a laboratory setting, simulation learning allows students to practice a variety of skills including assessment, psychomotor skills, and decision making

• Debriefing - instructor and student(s) discuss the situation offering feedback regarding what was done well and areas to think about

Simulation

• Stimulates critical thinking through assessment, analysis, and decision making; thinking occurs in a non-linear fashion

• Allows students to practice and learn real situations without risk to patients/clients

• Students can repeat the experience as needed to develop skills and confidence

Post-operative Assessment

Purpose: To simulate receiving report and making initial client assessment.

• Learners are given report (either verbal or taped), take notes

• Learners perform an assessment on a mannequin (one learner per mannequin)

• Learners write a narrative note of assessment• Review with learners what should have been assessed.

Wrong IV solutionIV wrong flow rateOutdated tubingMed (multivitamins)

added to infusionSyringe on bedside

standClient NPO but has

picture of ice waterNG tube not connected

to suction device

Oxygen not connectedRed line for phlebitisTEDS on one leg onlyLarge amount of urine

that has gross sedimentMannequin is on side

with large amount of red drainage underneath

Caring in Fundamentals

• Diapering• Feeding• Bed pan• Social talking with peer ignoring client

Fetal Monitoring Simulation

• You are working on the labor and delivery room and see the following fetal monitoring strips. The nurse should FIRST see which client?

Time required:

• Preparation time: Moderate – High– set up time

• In class time: Moderate – High

• Analysis time: Moderate


Assessing Various Learning Domains

Assessing & Evaluating the Cognitive Domain

Muddiest PointOne Minute PaperGroup WorkDirected ParaphraseAnalogies Learning ContractConcept/Mind MappingPost-Op AssessmentSimulationsWriting

Assessing & Evaluating the Affective Domain

Self-AssessmentGroup Work Journaling AssessmentAttitude surveysPortfolios

Assessing & Evaluating the Psychomotor Domain

Return demonstrationVideo tapePick out of hatPeer critique Learning contractSimulations

RUBRICS

“ If you aren’t sure where you’re going, you’re liable to end up some place else and not even know it.”

~Robert Mager~Robert Mager

Using Rubrics to Evaluate Learners’ Performance

Systematic scoring guideline with specific criteria

Needed when evaluating subjective assignments such as essays, projects, and portfolios

Learners often complain that the grading standards are vague, inconsistent, or unfair

Using Rubrics to Evaluate Learners’ Performance

A list of criteria, or "what counts" in a project or assignment

Gradations of quality, with descriptions of strong, middling, and problematic learner work

Collaboration Rubric

Beginning1

Developing2

Accomplished3

Exemplary4

Score

Contribute

Research & Gather Information

Does not collect any information that relates to the topic

Collects very little information--some relates to the topic

Collects some basic information--most relates to the topic

Collects a great deal of info--all relates to the topic

Share Information

Does not relay any information to teammates

Relays very little information--some relates to the topic.

Relays some basic information--most relates to the topic

Relays a great deal of info--all relates to the topic

Be Punctual Does not hand in any assignments

Hands in most assignments late

Hands in most assignments on time

Hands in all assignments on time

Beginning1

Developing2

Accomplished3

Exemplary4

Score

Take Responsibility

Fulfill Team Role's Duties

Does not perform any duties of assigned team role

Performs very little duties

Performs nearly all duties

Performs all duties of assigned team role

Share Equally

Always relies on others to do the work

Rarely does the assigned work--often needs reminding

Usually does the assigned work--rarely needs reminding

Always does the assigned work without having to be reminded

Beginning1

Developing2

Accomplished3

Exemplary4

Score

Value Others' Viewpoints

Listen to Other Teammates

Is always talking--never allows anyone else to speak

Usually doing most of the talking--rarely allows others to speak

Listens, but sometimes talks too much

Listens and speaks a fair amount

Cooperate with Teammates

Usually argues with teammates

Sometimes argues

Rarely argues Never argues with teammates

Make Fair Decisions

Usually wants to have things their way

Often sides with friends instead of considering all views

Usually considers all views

Always helps team to reach a fair decision

Clinical Learning Log/Reflective Clinical Journal Rubric

0 1 2

Content Clinical outcomes not assessed by learner or no reflection of experience provided.

Log/journal entry

assesses most of

the clinical

outcomes, but not

all. Superficial

reflection present,

but learner needs

more depth in

thought process.

Log/journal entry assesses each clinical outcome of assignment.

0 1 2

Reflection Demonstrates little effort toward seeking opportunities for reflection.

Examples do

not demonstrate learner learning or

Professional

growth.

Reflection demonstrates limited understanding of the nurse’s role in the particular clinical setting

Some connections established between theory and clinical practice.

Slight professional growth demonstrated in theory base and

clinical practice.

Poor examples of

learner learning and professional growth.

Reflection of clinical experience demonstrates insight and personal assessment. Reflection of clinical experience shows evidence of

enhancement of learner theoretical base and

clinical practice. Specific examples of learning

and professional growth provided by learner.

0 1 2

Critical Thinking

•Evidence of

critical thinking

principles and

nursing process

lacking and not

defended in

log/journal•does not incorporate principles into planned client care•Fails to evaluate the effectiveness of planned client care•Client plan of care is not revised as needed

•Some evidence of use of critical thinking principles and nursing process communicated but poorly defended in log/journal•incorporates some

principles into planned client care.•Partially evaluates the effectiveness of planned client care

•Following evaluation, client plan of care is not revised as needed and alternative solutions are determined but

not implemented

•Evidence of critical thinking principles and nursing process communicated and clearly defended in log/journal. •Incorporates principles into planned client care.•Evaluates the effectiveness of planned client care•Following evaluation, client plan of care is revised as needed and alternative solutions are determined/implemented.

Grading for Web Discussion

• Discussion is worth 10% of grade

• Average of ten weeks – Maximum of 10 points – Minimum of 0 points

Score Level of Participation During One Week

0 point No postings by assigned date

1 point Posted main topic information

Replied to one other learner posting

No depth of presentation, no research base, opinion only

Information posted only one time or several posts at one time

Comments were barely related to main discussion question and/or other learner posting

No constructive comments to help class discussion

All posts made within 24 hours of assignment due date

Posted main topic information and one response on same day


2 points Several posts, but all on same day

Time between posting indicated learner had read and considered substantial number of learner postings before responding

Replied to other learner postings and provided relevant responses and constructive feedback to the learner

Enhanced quality of discussion (e.g., illustrated a point with examples, suggested new perspectives on issues, asked

Questions that helped further discussion (e.g., cited current news events)


Referenced other research, gave examples, and evoked follow-up responses from other students


3 points Demonstrated leadership in discussions

Posted regularly during the week

Replied to main topic. Substantially enhanced quality of discussion (i.e. illustrated a point with examples)

Perspectives on issues, asked questions that helped further discussion (cited current news events etc.)

Replied to several other learner postings on a regular basis and provided relevant responses and constructive feedback to the learner posting


Referenced other research, gave examples, and evoked follow-up responses from other students

Another Version• CriteriaPoints

Participation (3 pnts): • Logs in at least twice during week of activity, posts relevant, clearly

written comments, correlates with readings, maintains professionalism in writing = 3.

• Logs on once, posts relevant comments, minimal correlation with readings=2.

• Logs on once, posts trivial or irrelevant comments, not applicable to readings, =1.

• Fails to participate in discussions =0.

Activity/Assignment (3 pnts). • Completes activity as described, appropriate level for RN-BSN student,

well written, submitted on time =3. • Somewhat fulfills assignment, questionable quality of work =2. • Does not follow directions for assignment, poor quality =1.• Omits assignment =0.

_______________________________________________• Total Points

0 1 2 3

0 1 2 3

Nursing Care Plans

Nursing Care Plan Grading CriteriaAssessment Assessment Tool

35% Complete and Accurate 35%______

Diagnosis Minimum of 1 Nursing Diagnosis which25% correlates with assessment data obtained 10%______

Nursing diagnosis correctly stated using NANDA approved diagnosis 5%_______

“Related to” correlates with clients data collected on the Assessment toll and is not a medical diagnosis 5%_______

“As evidenced by” correlates with assessment data documented on assessment form (only if actual diagnosis) 5%_______

Planning 1 short term and 1 long term mutual goal15% Each goal is:

Related to the nursing diagnosis 3%_______

Client centered 3%_______

Realistic for client situation 3%_______

Has a time frame 3%_______

Measurable 3%_______

Intervention25% Nursing measures

Contribute to achievement of goals 10%______

Comprehensive 10%______

Have scientific rationales 5%_______Evaluation Client centered

0% Correlates with goal

Total (75% is the minimum satisfactory grade) _________%

Steps in Developing a Rubric Make a preliminary decision on the dimension of the

assignment t be assessed Look at some actual examples of learners work to see if

you have omitted any dimensions that should be included Revise as need Define each dimension Develop a scale for describing the range for each

dimension Pilot the rubric on assignments done by previous learners Revise as needed Share rubric with learners

An Effective Rubric

Addresses all relevant content and performance objectives Defines standards and help students achieve them by

providing criteria with which they can evaluate their own work;

Is easy to understand and use Is applicable to a variety of tasksProvides all students with an opportunity to succeed at

some levelYields consistent results, even when administered by

different scorers

“Strategies for Effective Clinical Teaching and Evaluation” Classroom/Clinical Assessment...

Documents

Transcript of “Strategies for Effective Clinical Teaching and Evaluation” Classroom/Clinical Assessment...