An Introduction to Understanding the OT and OTA

Fieldwork Performance Evaluations

(FWPEs)

Karen Atler, MS, OTR Colorado State University, Fort Collins, CO

Roberta Wimmer, OTR/L Pacific University, Forest Grove, OR

2003 The American Occupational Therapy Association, Inc. www.AOTA.org

2

Task Force Members Carole Dennis, PhD, OTR

Ithaca College, New York Carole Hays, MA, OTR

Springfield Hospital Center, Maryland Becky Robler, MEd, OTR

Pueblo Community College, Colorado Karen Atler, MS, OTR, Co-Chairperson

Colorado State University Roberta Wimmer, OTR, Co-Chairperson

Pacific University, Oregon

3

Objectives Describe the

Entry-level practice competencies for OT and OTA students

Purpose, format, content, and scoring of the companion evaluation forms

New concepts and terminology used in the FWPE from the OT Practice Framework

Begin to score items on the FWPE

4

Task Force’s Charge Revise/develop evaluation tools to measure

assistant and professional Level II fieldwork student performance.

Expectations Conduct review of literature across disciplines Synthesize feedback on current AOTA FWE/OT

forms Incorporate 1997 NBCOT Practice Analysis results Address identified desired characteristics

5

Desired Characteristics Companion documents for assistant

and professional level that Measure entry-level competence Focus on occupation-based practice Reflect current and future practice Can be used in a variety of settings Provide feedback to students Can be easily used in a timely manner

6

NBCOT Practice Analysis 1997

What OTs & OTAs Do

Determining needs/priorities for interventions

Identifying/designing interventions

Implementing interventions Reporting/evaluating

intervention effectiveness Providing OT services for

populations Managing delivery of OT

services Advancing effectiveness of

the OT profession

What OTs & OTAs Need To Know

Human development and performance

Principles/strategies in the identification/evaluation of strengths and needs

Principles/strategies in intervention/treatment planning

Principles/strategies inintervention

Nature of occupation and occupational performance

Service management Responsibilities as a professional

7

Standards of Practice for Occupational Therapy Identifies minimum standards Identifies key performance areas for the

OT and OTA Professional standing and responsibility Referral Screening Evaluation Intervention plan Intervention Transition services Discontinuation

8

ACOTE: Minimum Standards and Outcomes

OT Be a generalist Achieve entry-level

competence Articulate, apply, and justify

occupation interventions Supervise and collaborate

with the OTA Keep current with best

practice Uphold the ethics, values,

and attitudes of the profession

Be an effective consumer of research and knowledge

OTA Be a generalist Achieve entry-level

competence Work under the

supervision of and in cooperation with the OT

Articulate, apply, and justify interventions related to occupation

Keep current with best practice

Uphold the ethics, values, and attitudes of the profession

9

Goal of Level II Fieldwork Education for the OT and OTA Student Develop competent, entry-level

generalists Include an in-depth experience in

delivering occupational therapy services

Be designed to promote reasoning, enable ethical practice, and develop professionalism

10

The Process Began with OTA evaluation Reviewed by experienced panel Submitted to COE Made revisions Completed pilot studies (2 OTA, 1

OT)

11

Design and Analysis of Pilot Studies: The Rasch Measurement Model

less able Student Ability more able

12

Results of Pilot Studies Good representation in pilot samples Students and educators preferred new form Good scale and response validity Inaccurate use of scale

Rating Scale Usage in Pilot Studies II and III

4 = Exceeds Standards3 = Meets Standards2 = Needs Improvement 1 = Unsatisfactory Rating Scale Descriptors

29345356131000

% Usage OTA% Usage OT

13

The FWPEs for OT and OTA Students Companion documents Terminology Content layout Purpose Design Rating scale Scoring system

14

FWPEs OT and OTACompanion Documents Focus

The occupational therapy process The clinical reasoning process Roles and responsibilities of the OT and OTA

Structure Collaborative process—student and FW

educator Same layout Same rating/scoring system

15

Terminology of the FWPEs Reflects

Standards of Practice for Occupational Therapy and ACOTE Education Standards

Occupational Therapy Practice Framework The glossary

16

Content Layout of FWPEs Summary Sheet Overview/instructions Organization of items Space for comments—midterm and

final Performance Rating Summary

Sheet

17

Content of OT and OTA Evaluations

OTA Fundamentals of practice (3) Basic tenets (3) Evaluation/screening (5) Intervention (6)

Communication (2) Professional behavior (6)

OT Fundamentals of practice (3) Basic tenets (4) Evaluation/screening (10) Intervention (9) Management of OT services

(5) Communication (4) Professional behavior (7)

18

Primary Purpose of the FWPEs Measures entry-level competence

Designed to differentiate the competent student from the incompetent student

Not designed to differentiate levels above entry-level competence

19

Purpose (continued) Provides student with accurate

assessment of his or her competence for entry-level practice over time Growth occurs over time Midterm and final scores reflect this

change Midterm scores: Satisfactory–

unsatisfactory Final scores: Pass–no pass

20

Purpose (continued) Provides feedback to student Provides opportunity for student

self-assessment

21

Design of the FWPEs The “doing” of the occupational

therapy process is evaluated, not the individual tasks in isolation

NOT all items are equal in level of difficulty (i.e., simple to complex)

Evaluation is supplemented with development of site-specific objectives

22

RASCH Ordering of Items OT

 

- - 2- - - - - - - - - - 1- - - - - - - - - - 0- - - - - - - - - --1

- - - - - - - - - --2

-

Inte

rper

sona

l; 4

1R

espn

ds 2

fdbk

38

Div

ersi

ty; 4

2Et

hics

; 1  C

olla

b c

supe

r;36

Wor

kbeh

39,

safe

ty 2

Step

s 3,

Res

p; 3

7        Ti

mem

ng40

; Leg

ible

34

  Clie

nt c

ente

red

inte

rv 2

2    O

ccup

n ba

sed

inte

r 23

  Sele

cts

rel o

ccu;

21

Prod

uces

wor

k; 3

1C

lear

doc

umen

tatio

n; 3

3O

rg g

oals

30;

ver

bl 3

2 D

ocum

ents

intrv

; 26

Col

labo

rate

s cl

nt; 7

Lang

uage

refle

cts

35

Doc

umen

ts e

val r

slts

17

OTb

elie

fs 4

Obt

ain

inf 1

2 R

ole

of O

T; 6

Rat

iona

le T

x 18

, A

dmin

iste

r ass

es 1

3 A

rtic

ratin

al e

val;

8A

rtic

valu

e oc

c 5

Mod

ifies

app

roac

h 24

Und

erst

fina

nces

; 29

Col

labo

rate

s O

TA 2

8O

cc p

rof 1

0, A

djst

ass

14

Est P

lan

16, A

sses

s ftr

11U

pdat

es; 2

5, E

vide

nce

19

Inte

rpre

ts e

val r

eslts

; 15

  Sele

cts r

el a

sses

met

hd; 9

  Ass

igns

resp

OTA

; 27

HARD

 

23

Rasch Ordering of Items OTAEasier

2- - - - - - - - - - 1- - - - - - - - - - 0- - - - - - - - - -

-1- - - - - - - - - -

-2-

Cult

ural

compe

tence;

25

Ethi

cs; 1

In

terp

erso

nal sk

ills;2

4 Re

spon

ds to

Feedba

ck;21

Safe

ty; 2

and 3

Wo

rk B

ehavio

rs; 22

Therap

eutic

Use Se

lf;16

Writ

ten Co

mmunic

ation;

19

Verb

al Com

munica

tion;

18

Self

-respo

nsibil

ity; 2

0

IIm

plemen

ts int

ervntn

; 14

OT

/OTA Rol

es 5Da

taGath

er

Sele

cts

Interv

ention

;13

Acti

vity A

nalysi

s; 15

Ev

iden

ce Bas

edPrac

tic 6

Pl

ans

Interv

ention

; 12

Re

port

s; 10

OT P

hiloso

phy; 4

Ad

mini

ster A

ssessm

nts;8

Esta

blishe

s Goal

s; 11

Mo

difi

es Int

rven P

lan;17

Interp

rets A

ssessm

ent;9

Harder

24

Rating Scale of FWPEs4 = Exceeds

Standards

3 = MeetsStandards

2 = NeedsImprovement

1 = Unsatisfactory

Performance is highly skilled and self-initiated. This rating is rarely given and would represent the top 5% of all the students you have supervised.Performance is consistent with entry-level practice. This rating is infrequently given at midterm and is a strong rating at final.Performance is progressing but still needs improvement for entry-level practice. This is a realistic rating of performance at midterm and some ratings of 2 may be reasonable at the final.Performance is below standards and requires development for entry-level practice. This rating is given when there is concern about performance.

25

Scoring System of FWPEs Each item must be scored Ethics and safety items must be

passed Each item rating recorded on

Performance Rating Summary Sheet All items summed up at midterm and

final Score compared to scales provided

26

Midterm and Final Scoresof the FWPEs

OTAOverall Midterm ScoreSatisfactory: 54 & aboveUnsatisfactory: 53 &

belowOverall Final ScorePass: 70 & aboveNo Pass: 69 & below

OT Overall Midterm ScoreSatisfactory: 90 &

aboveUnsatisfactory: 89 &

belowOverall Final ScorePass: 122 & aboveNo Pass: 121 & below

27

Rating Performance Using the FWPEs Case ScenariosJohn—OT Evaluation Mary—OT Intervention Sandra—OTA Evaluation/Screening David—OTA Intervention

28

Individualizing the FWPEs Designed for additional objectives

to be written to add clarification Site-specific objectives NOT supervisor-specific

If an item is very clear and meets the RUMBA test, then there is no need to write another objective

29

Objectives: An Example From FWPE for the OT Student 16. Establishes accurate and

appropriate plan School—Provides behavioral-based,

measurable OT goals during IEP process

Acute Care—Overall intervention plan is achievable within client’s length of stay

30

Summary Evaluations designed to measure

entry-level competence, NOT level of performance above competency

OT practice examined as a generalist Evaluations reflect the occupational

therapy process Performance develops over time