2011 New Handbook

7/27/2019 2011 New Handbook

1/38

AssessmentHandbook

Divisionof

Student

Affairs

BostonCollege

20102011


2/38

2

Originally compiled by Jennifer Lackie, M.Ed.10

Last updated January 2011

Assessment Planning Cycle and narrative adapted from materials byGavin Henning, Dartmouth College


3/38

3

Index

AssessmentPlanningCycle.

Step1:

Describe

the

program

and

the

issue

or

question..

Step2:Developgoalstatement..

Step3:AligngoalstatementwiththeDivisionalLearningGoals.

Step4:Identifyanddeveloplearningoutcomes..

Step5:Identifyanddevelopoperationaloutcomes...

Step6:Developstrategiestoreachlearningandoperationaloutcomes..

Step7:Identifyassessmenttechniquesandconductassessment..

Part1:SelectingtheAssessmentTechnique.

Part

2:

Developing

the

Assessment

Tool..

Part3:ConductAssessment/CollectData..

Step8:Interpretassessmentdata..

Step9:Report/presentassessmentdataandfindings...

Step10:Adjustprogrambasedonassessmentdata,asneeded.

Appendices:

DivisionalLearningGoals...

DivisionalMissionandVisionStatement.

QuickGuidetoDiscussingLearningOutcomes...

StudentLearningOutcomesbyBloomsTaxonomy.

BloomsTaxonomy..

FormativeAssessmentTechniques..

PreAssessmentConversationGuide..

StudentVoiceAssessmentProjectRequestForm.

5

6

7

8

9

13

14

15

15

18

19

20

21

22

23

25

26

27

28

29

35

36


4/38

4

This page intentionally left blank


5/38

5


6/38

6

Tobegintheassessmentplanningcycle,itisfirstnecessarytodeterminewhattoassess.

Youmaywanttoassesstheoveralleffectivenessofaparticularprogramoraddressa

certainquestionorissuethatexistswithinaservice/program.Therearesometimesmany

issueswrappedupintooneForexample:Yourinitialquestionmaybe,whatisleadingto

residencehallvandalism?Potentialissuesmightbealackofrespectfortheresidencehall,a

lackofconnectiontothehallcommunity,orbehaviorsresultingfromhighriskalcoholuse.

Onceyouhaveidentifiedtheissuesofconcern,youwillneedtodecidewhichspecific

questionyoureallywanttofocusyourplanningandassessmenton.

Then,you

should

describe

the

initiative,

activity,

or

program

that

addresses

this

problem

or

issue.Afewsentencesummaryincludingwhoisinvolvedwiththeprogram(thetarget

population)andageneraloverviewofwhattheprogramentailswillsuffice.

Step 1:Describe the issue or question you want to explore and an

initiative, activity, or program that is effected by this question

Example:

TheLeadershipInstituteisaseriesofprograms,workshops,speakersand

presentationsthroughouttheyearopentoallclubofficers.TheInstitute

hopestoaddresstheissueofstudentleaderbehavioroncampus.Whilethe

OfficeofCommunityDevelopmenthashighexpectationsforstudentleaders

to

be

campus

leaders

and

role

models

for

other

students,

the

students

some

timesperceivetheirrolesaspositionalonlyandlimitedtothedescriptions

outlinedintheclubororganizationsconstitution.


7/38

7

Step 2:Develop your goal statement based on the issue/question

Onceyouhaveidentifiedthequestionorissueyouwanttofocuson,youneedtodevelopa

broadgoalstatementforwhatyouwanttoaccomplish.Forexample,youmaywantto

increasethefeelingofconnectiontotheresidencehallcommunityorbetterunderstand

whetherXprogrammeetsitsmissionto.

Example:

ThegoalofthisprogramistochallengestudentsintheJesuittradition

tobecomeincreasinglycomplexandincreasinglyprincipledintheir

understandingofappropriatebehaviorforstudentleadersatour

institution.


8/38

8

Step 3:Align goal statement with the divisional learning goals

ItisassumedthatindividualdepartmentmissionsfitintothemissionoftheDivisionandthe

strategicplan.Therefore,alignmentwiththemissionofyourdepartmentanddivisional

learninggoals

ensures

that

the

Divisions

various

programs

and

offices

are

all

moving

towardssimilaraims.

Reviewyourdepartmentsmissionstatementandyearlygoals

ReviewtheDivisionalLearningGoals(appendix2)

Ifnecessary,reviewtheDivisionofStudentAffairsMissionStatement(appendix3)

Finishthestatement:ThisgoalsupportsthedivisionallearninggoalsbyByfinishingthis

statement,youwilldemonstratethealignment.Feelfreetowriteandexpanduponhow

theprogram/servicefitsinwiththemissionasneeded.

Itisalwaysimportanttoconsideryourstakeholdersduringthisstep.Thinkaboutwhoyour

stakeholdersareandincludetheseindividuals/officesinyourplanning.Whoisaffectedby

theprogramandwouldhavesomeinterestinit?Ensurethatasyoualignyourgoal

statementwiththedepartmentalmissionandlearninggoals,youaretakinginto

considerationtheinterestsandneedsofyourstakeholders

Example:

TheMissionStatementemphasizesthenurturanceofengagedlearn

ers.Practiceinterviewsofferaunique,handsonlearningexperience

thatfullyengagesstudentsinanintellectualandpersonalexercise,one

thatincludesimmediatefeedbackonthequalityoftheirresponses.

TheMissionStatementemphasizestheneedtofacilitatestudentlearn

ingandformationintheirfullestsense(integratingintellectual,ethical,

religiousandspiritual,andemotionalsocialdevelopment).Practicein

terviewsofferstudentstheopportunitytoreflectonthetotalityoftheir

experiencesatBostonCollegeandtolearntoarticulatetheseexperi

encesintheserviceoftheircareergoals. Program

Level

Department

Level

DivisionLevel

UniversityLevel

Developedfrom:StudentVoice,DesigninganAssessmentProject

15,11/4/2010


9/38

9

Step 4:Identify and develop learning outcomes for a specific issue or

question

InStep4,youwilldeveloplearningoutcomesforthespecificissueorquestionidentifiedin

Steps13.ThisisacriticalstepintheAssessment

PlanningCycle,particularlyastheDivisionof

StudentAffairsmovesforwardinitsStrategic

Planningandbecomesincreasinglyfocusedon

studentlearning.

Thenextsectionwillprovidesomeuseful

techniquesthatmightguideyourthinkingasyou

considerlearningoutcomesforthespecificissueor

questionyou

plan

to

assess.

Brainstorm:Oneoftheeasiestwaystobegin

writinglearningoutcomesistothinkaboutyour

program,activity,orserviceandbegin

brainstormingalistofallthethingsthatyouhope

theparticipantswilllearnfromparticipatinginyourprogramorservice.Brainstormalist

withoutworryingabouttheformatorqualityofyourwriting.Peopleoftenfinditmore

effectivetobrainstormwithasmallgroupofcolleagues.

Organize/Categorize:Onceyouhavecomeupwithafulllistofpotentiallearningbased

outcomesforyourprogram,activity,orservice,reviewthelist.Removeanyduplicatesand

sortandorganizetheremaininglearningoutcomesinmanageablegroups.Whenorganizing

yourlist,itislikelythatyouwillfindmultipleoutcomesthatdealwiththesameaspectofa

programorareverysimilarinotherways.Groupingandsortingyourideasmakesiteasier

whenitcomestoactuallywritingtheoutcomes.

SelectyourOutcomes:Wheneveryoubeginanewassessmentproject,itisrecommended

thatyouhavethreetofivelearningoutcomes.Ifyouroutcomesaremorecomplex(e.g.:

studentwillbeabletodemonstrateanincreaseinleadershipabilitiesasmeasuredbya

rubric),you

should

have

fewer.

If

your

outcomes

are

simpler

(e.g.:

student

will

be

able

to

list

thenamesofthreeoffices),youcouldhavemorethanfivelearningoutcomes.Reviewthe

organizedlistsyouhavebrainstormedanddeterminewhichlearningoutcomesyouwouldlike

toassess.Youwillmostlikelyselectthelearningoutcomeswhicharemostcentraland

importanttoyouractivity,programorservice.However,basedonyourownneedsandthose

ofyourstakeholders,other,lesscriticallearningoutcomesmaybechosen.

StudentVoice,HowtoWriteOutcomeStatements,

Slide11,10/22/09

Fig1:LearningOutcomesAssessment

LearningOutcomesAssessmentOutcomesassessmentisthemostvalidwayofdemonstratingtheeffectivenessofservices,programs,andfacilities,especiallyindefendingandpromotinghighereducation,andalsoinmeetingaccreditationstandards.Itisalsomostdifficult,complexandmisunderstoodofalltheassessmentmethodologies.

(Upcraft&Schuh,1996)


10/38

10

Thenextstepsonlyutilizetheoutcomesyouhaveselected.Youwillnotneedtherest!

Overtime,asyougatherassessmentdataandbecomemorecomfortablewiththeassessment

process,youcanaddadditionaloutcomesasyouseefit.However,atthebeginning,startsmall.

ChooseyourBloomverbs:Thekeytosuccessindesigninglearningoutcomesischoosingverbs

whichreflect

the

level

and

nature

of

learning

you

anticipate

will

result

from

the

program

or

servicethatisbeingassessed.BestpracticesinstudentaffairssuggesttheuseofBlooms

Taxonomytoselectverbsforlearningoutcomes.

Bloomidentifiedsixlevelsoflearning:Remembering,Understanding,Applying,Analyzing,

EvaluatingandCreating.Thelevelsbuildinincreasingorderofdifficultyfrombasic

memorizationtohigherlevelsofcomplexandcriticalthinkingskills.AsisevidentinFig.2,

movingupthepyramidreflectshigherorder

thinkingskills.

Usefullearning

outcome

verbs

are

categorized

by

theirlevelinBloomsTaxonomy.Alistof

categorizedverbsisavailableinAppendix5.

Whenchoosingaverbforthelearning

outcome,itisimportanttoconsiderwhatyou

intendtomeasure,asthatwillaffectthe

BloomsTaxonomylevelfromwhichyouselect

theverb.Forexample,ifyouwouldliketo

measurewhetherastudentknowstheoffices on

campusthat

can

help

them

with

their

study

habits,

you

would

use

averb

from

the

Rememberinglist.If,however,youwantedstudentstounderstandwhateachofthoseoffices

did,youwouldselectaverbfromtheUnderstandinglist.AttheRememberinglevel,you

areassessingwhetherstudentshavememorizedbasicinformation,attheUnderstanding

level,youareassessingwhetherthestudentsunderstandtheinformationthatthey

memorized.

Includeindicatorofsuccessinoutcomes:

Inorderfortheresultstobemeasurable,youneedtodeterminewhatexactlyindicatesthatan

outcomewasachievedornot.Youdothisbyarticulatingexactlywhatdefinessuccess.

ApooroutcomewouldbeAttheconclusionofXprogram,studentswillbeabletoidentify

campusresourcesthatcansupporttheiracademicachievement.Thisisnotagood

outcomebecausethereisnoindicationofwhatwemeanbyidentifycampusresources.If

Iidentifyonecampusresource,didImeetthelearningoutcome?

AbetteroutcomemightbeAttheconclusionofXprogram,studentswillbeableto

identifyfivecampusresourcesthatcansupporttheiracademicachievement.Thisoutcome

Fig2:BloomsTaxonomy


11/38

11

ismoreappropriatebecauseitspecifiesexactlywhatindicatesthatthestudenthas

achievedthelearningoutcome(theywillknowfivecampusresources).

Theindicatorofsuccessiscalledthedegree.Degreecanbeexpressedinmanyways,

dependingonthenatureofthelearningoutcome.Somecommonwaystoexpressdegree

includeusing

aspecific

number

(6,

3),

using

apercentage

or

specific

portion

of

the

total

(50%,

7outof10)orusingaqualifyingwordsuchassufficientlyoraccurately.Twoimportant

thingstonote:First,multipledegreescanbeusedperoutcome(seebelow)andsecondly,the

abovelistisnotexhaustivethereareadditionalwaystoindicatedegreebeyondthose

specified.

Formatting:Nowthatyouhavecreatedallofthecomponentsofalearningoutcome,youneed

toputtogetherthepiecesandactuallywriteit!Therearetwocommonformatswhichareused

towritelearningoutcomes:SWiBATandABCD.

SWiBAT:One

formula

that

is

successful

in

developing

learning

outcomes

is

to

begin

your

outcomesstatementwithStudentsWillBeAbleTo.Beginyouroutcomestatementwith

thisandthenaddanactionverbandacondition.Forexample,studentswillbeableto

successfullyfacilitateahallgovernmentmeetingasaresultofparticipatingintheresidence

hallcounciltraining.MakesureyouincludeyourdegreewhenusingtheSWiBATformula!

ABCD:AnothertechniquethatcanbeusedtodeveloplearningoutcomesistheABCD

format.UsingtheABCDformat,thelearningoutcomemusthaveaspecificidentified

Audience,aparticularBehaviorwhichtheaudiencewillbeabletoknowordo(thisisthe

verbyouselected),aConditionwhichwillleadtotheearningandaDegree,whichindicates

thelevel

to

which

the

particular

behavior

will

need

to

be

demonstrated

in

order

to

ensure

learningtookplace.AlthoughthistechniqueiscalledtheABCDformat,itisnotnecessary

tohavethelearningoutcomefollowtheABCDformatinexactorderaslongasthe

componentsareallthere.Onecould,forexample,writealearningoutcomeinBCDAor

ADBCorder.Figure4,below,providesanexampleofhowalearningoutcomewouldlookif

formattedinCABDorder.

ABCDStructureofaLearningOutcomeAudience/Who

Whodoestheoutcomepertainto?Behavior/What

Whatdoyouexpecttheaudiencetoknow/beabletodo?Condition/How

Underwhatconditionsorcircumstanceswillthelearningoccur?Degree/Howmuch

Howmuchwillbeaccomplished,howwellwillthebehavior needtobeperformed,andtowhatlevel?

(Heinich,etal,1996)

Fig3:ABCDStructure

StudentVoice,HowtoWriteOutcomeStatements,

Slide15,10/22/09

Fig4:ABCDStructureApplied

22

As a result of participating in the leadership workshop students

will demonstrate three of the five leadership criteria

as stated in Kouzes and Posners The Leadership Challenge.

Developing LearningOutcomes

ConditionAudience

Behavior (verb infuture tense) Degree

Henning,Wherearewegoing?,Slide22,10/22/09


12/38

12

Checkyourself:Wellwrittenoutcomeshavesomedefiningcharacteristics.SMARTisa

mnemonicdevicethatcanserveasachecklist.SMARToutcomesare:

Specific

Measureable

Aggressive,butattainable

Resultsoriented

Timebound

Afteryouhavewrittenyourlearningoutcomes,verifythattheyareSMART.Ifyoudo,youwill

bemoreeffectiveasyouplanandyourassessmentwillbeeasiertoimplement.

LearningOutcomeExamples:

ResidentAssistantswillbeabletosuccessfullydemonstratethree

techniquestheycanusetointerveneinadiscriminatoryjokeatthe

endofRespectWeekend.

AftertwomeetingswiththeUHSStaff,studentswithlatentTBwillbe

abletoaccuratelydescribefourprosandfourconsofreceivingthe

tuberculosistreatment.


13/38

13

Step5:Identifyanddevelopoperationaloutcomes

Unlikelearningoutcomesthatmeasurewhatstudentsareabletodo,know,orvalue,

operationaloutcomesareaggregateanddocumentprogresstowardstheoutcome.

Operationaloutcomesmayincludethetotalnumberofprogramspresentedornumberofstudentsserved.Forexample:30studentswillattendtheprogram;75%ofstudentswill

completesomething,etc.

Operationaloutcomestendtoberelatedtosatisfaction,quality,tracking,needsmet,and

usage.

Includespecificindicatorsofsuccesswhilewritingoperationaloutcomes.(Seestep4for

moreinformationonindicatorsofsuccess).

Examples:

TheCareerCenterwillrunpracticeinterviewsforatleast50

studentsduringthefallsemester.

75%ofthecustomerswhopurchaseticketsattheRobshamBox

OfficewillindicatethattheyaresatisfiedorbetterwiththeBoxOf

ficescustomerservice


14/38

14

Step 6:Develop strategies to reach learning and operational outcomes

Aspreviouslymentioned,successfulassessmentisinextricablytiedtogoodplanning.

Strategiesarethespecificstepsthatyouwilltakeorthethingsthatyouwilldotofoster

eachofthelearningoutcomes.

Alignstrategieswithoutcomesandgoals

Onceyouhaveidentifiedyourgoalsandoutcomes(bothlearningandoperational),

beginconsideringstrategiesthatwillfosterthosegoals.Forexample,ifyourgoalis

forstudentstodevelopleadershipskillsandoneoutcomeforthatistheabilityto

facilitateahallgovernmentmeeting,whatstrategieswouldhelpstudentsbeable

tofacilitateameeting?Onepossibilitywouldbeforthemtoobserveother

meetingsandidentifybestpracticesformeetingfacilitation.

Thingstoconsiderwhendevelopingstrategies

BestPractices:

The

strategies

used

should

be

rooted

in

theories,

conceptual

frameworkandbestpractices.Identifyingtheories,conceptualframeworksand

bestpracticesensuresthatyouarebeingintentionalinyourwork,andthatyour

workisgroundedinresearchandevidence.

Actualresources:Thestrategiesusedshouldberealisticwhenconsideringthe

resourcesthatareavailabletoyou,includingtime,finances,humanresources,

facilities,etc.

Stakeholders:Thestakeholdersoftheactivityyouareassessingmayhave

particularstrategieswhichtheyprefer.Considertheinterestsandneedsofyour

stakeholderswhiledevelopingyourstrategies.Additionally,someofyourstrategies

may

involve

additional

stakeholders

who

should

be

taken

into

account.

For

example,ifaparticulartrainingonselfimageinvolvestheDirectoroftheWomens

ResourceCenter,itisimportanttoverifythattheDirectorisavailableandwillingto

participate.

Numberofstrategies:Therewilllikelybemultiplestrategiesforeachgoal.

Example:

LO:RAswillbeabletosuccessfullydemonstratethreetechniquesthey

canusetointerveneinadiscriminatoryjokeattheendofRespectWeek

end.

Strategy1:Aspartoftheprogram,wewillrunaworkshopthatdelineates

thestepbystepprocessforinterveningindiscriminatoryjokes.

(Conceptualframework:ThisstepbystepprocessfollowstheBystander

InterventionProgrammodel)


15/38

15

Step 7:Identify assessment technique and conduct assessment

Duringthisstep,youwillidentityanddeveloptheassessmenttechniquesthatyouwilluse

tomeasurethesuccessofachievingtheoutcomesandgoal(s)foryourprogram. The

programyou

have

chosen

to

assess

should

drive

the

assessment

technique

you

select,

not

theotherwayaround. Duringthisstepyouwillselecttheassessmenttechniquewhich

youwillusetoconductyourassessment,createtheassessmenttool,andconductthe

assessmentusingthetoolsyouhavedeveloped.

Part1:SelectingtheAssessmentTechnique Whenselectinganassessmenttechnique,itisimportanttokeepthefollowing

considerationsinmind:

Directvs.IndirectAssessment:Assessmentcanbedirect,indirect,orboth.Whendirect

assessmentisused,studentswillbeaskedtodemonstrateexpectedoutcomes.Direct

assessmentisevidenceoriented.Whenindirectassessmentisused,studentswillbe

askedtoreporthowtheyperceivemasteryofparticularconceptsoroutcomes.

Indirectassessmentisperceptionoriented.

Itisimportanttoconsiderwhattypeofassessment(directorindirect)youwantto

conductbeforeyouselectyourtechnique,asthetechniquemayvarybasedonthe

typeofassessmentyouchoose.Pleasenotethatalthoughindirectmethodsmay

sometimesbeusefulandappropriategiventhelearningoutcomeyouintendto

measure,ingeneralitispreferabletousedirectassessmenttechniques.

Example:

IndirectAssessment: Studentvisitshealthservicesandmedicalprofes

sionalaskshowthestudentisfeeling.(Studentisbeingaskedtoreport

ontheirperceptionsoftheirhealth.)

DirectAssessment:Studentvisitshealthservicesandmedicalprofessional

takesthestudentstemperature,listenstothestudentslungs,etc.(The

medicalprofessionalisobservingthestudentshealth.)

or

IndirectAssessment:

At

apreliminary

interview,

career

counselor

asks

astudent howskilledtheyareininterviewing. (Studentisbeingaskedto

reporttheirperceptionsofaskill.)

DirectAssessment:Careercounselorobservesastudentdoingapractice

interviewandratesthestudentsinterviewingskills.(Counselorisobserv

ingthestudentsskills)


16/38

16

Quantitativevs.QualitativeData:Beforeselectinganassessmenttechnique,itisalso

importanttoconsiderwhattypeofdatayouwouldliketogather.Quantitativedata

focusesonnumbers,whilequalitativedatafocusesontext,narrativeandstudent

stories.Therearebenefitsandchallengestoeach.Whilequantitativedataiseasierto

reportandanalyze,qualitativedatacanbemuchmoreindepthandrobust.Itisalso

simplerto

generalize

quantitative

data

to

alarger

population

and

can

be

easier

(in

time,resources)toconduct.Ontheotherhand,duetoitsrobustnature,qualitative

datamayallowforagreaterunderstandingofstudentlearninganddevelopment.

Howwillyouusethedata?:Priortoconductinganassessment,thinkaboutwhowill

usethedataandhow.Yourstakeholders(andthekindofdatatheyliketosee)and

howyouplantousethedatawillinfluenceyourtechnique.

Availableresources:Evaluateavailableresourcesbeforeselectinganassessment

technique.Someassessmenttechniquesrequiregreatertime,financialresources,etc.

Culturalsensitivity:Certaintechniquesmaybemoreinvasivethanothers.Some

students,forexample,mayfeeluncomfortabletalkingaboutparticulartopicsina

groupsetting

(like

afocus

group).

Keep

this

in

mind

when

selecting

and

designing

the

assessmenttechnique.

AssessmentTechniques

Therearemanydifferentassessmenttechniqueswhichcanbeusedinanygivensituation.This

sectionoutlinesseveralofthemajortechniques,butitshouldnotbeconsideredanexhaustive

list. Appendix6alsolistsanumberofassessmenttechniqueswhichcanprovideaquickpulse

onhowaparticularprogramorserviceisgoing.

Tracking/ExistingData

Thistechniqueisthesimplest,butalsomayreveallessinformationthansomeofthe

otherassessmenttechniques.

Trackingincludesgraduationandretentionrates,housingoccupancy,income/

expenditures,staffing,numberofprograms/workshops,enrollmentdata,SATscores,

participationinprograms,attendanceatevents,usageofservices,numberofphone

calls,etc.

Mostofficesalreadytrackcertainpiecesofinformationlookatwhatyourofficeis

alreadydoingandusethatdataaspartofyourassessment.

Otherofficesmayalsobetrackingcertaininformation.InstitutionalResearch,for

example,has

awealth

of

information.

Before

beginning

an

assessment

project,

it

is

alwaysusefultounderstandwhatinformationotherofficeshavethatcaninformyour

ownassessment.

Trackingisoneofthesimplesttechniquestouse.Verylittleexpertiseindatacollection

oranalysisisneededandminimaltimeisrequired.


17/38

17

Surveys

Therearetwomajortypesofsurveys:nationalsurveys(thinkNSSEandCIRP)andlocal

surveyswhicharedevelopedbyyou,youroffice,ortheinstitution.

Surveysaregoodforselfreportsaboutperceptions(howsatisfiedwereyou?),

behaviors(howmanytimesdidyoudoX?)andobservations(doyouthinkthatcampus

isawelcomingplace?).Surveysarealsousefulfortestsorquizzestomeasurewhat

studentslearnedandforpre/posttests.

Althoughsurveyscanbeusefulforgroupsofanysize,theyaremostappropriatefor

largegroups.Moreinformationmaybegleanedfromsmallergroupsusingother

assessmenttechniques.

Onemajorprobleminhighereducationissurveyfatigue.Therefore,itisalwaysagood

ideatoreviewotherassessmenttechniquesanddetermineifanyofthemare

appropriate.Althoughitmaybenaturaltosendoutasurvey,theremaybemore

usefultechniqueswhichcanprovidethesame,orbetter,results.

You

must

consider

that

when

students

are

self

reporting

information

it

should

be

takenwithagrainofsalt.Researchshowsthatstudentsaremorelikelytoreportdoing

sociallyacceptablebehaviors(studying),andlesslikelydoingsociallyinappropriate

behaviors(takingillegaldrugs).

Uponrequest,StudentVoicewillconsultwithyouonsurveysyouhavedeveloped.

FocusGroups

Focusgroupsareusefulwhenyourgoalistounderstandtheperceptions,beliefsor

opinionsofagroupofstudents(orotheraudience).Theycanbeusedasadirector

indirectmethodofassessmentandcangatheralotofdatainashortamountoftime.

However,because

the

results

from

afocus

group

can

not

be

generalized

to

alarger

population,theyarelessusefulincreatingstatisticalreports.

Focusgroupsarenotexpensive,butdorequireadegreeofskillinfacilitation,

particularlyifthegroupdynamicsoftheparticipantsarechallenging.Evenwithstrong

facilitation,focusgroupdiscussionsareunpredictableandmayveerinunanticipated

directionsmakingtheselesscontrolledthanothertechniques.

Focusgroupscanbeconductedwith610people.Inordertovalidatefindings,the

samequestionsshouldbeusedwithmultiplefocusgroupsandwithdifferent

individualsinvolvedineach.Asaresult,focusgroupstakemoretimeandplanning

thanotherassessmenttechniques.

Finally,since

focus

groups

collect

qualitative

data,

they

require

more

time

to

analyze

theresultsthanfortechniqueswhichcollectquantitativedata.


18/38

18

Rubrics

Arubricisascoringtoolusedtoevaluatestudentperformanceorwork.Itisauseful

assessmenttechnique,particularlyatthehigherlevelsofBloomsTaxonomy.

Rubricsincludedescriptionsofwhatsomethinglookslikeatdifferentlevelsofmastery,

andthusprovideconsistencybetweendifferentraters.

Rubricsallowassessorstogatherrichdata,facilitateprovidingfeedbacktostudentson

theirperformance,andallowforadirectmeasureoflearning.

Datacanbecollected,andanalyzedbyarubricthroughobservation(i.e.,roleplay,

training,groupwork)orthroughartifactanalysis(i.e.,journal,resumeorreflection

paper).

Therearemanydifferentkindsofrubrics.Oneofthebenefitsofrubricsisthattheyare

highlyflexible.

StudentVoicewillassistyouwiththedevelopmentofyourrubric,butitisalwaysbest

practicetohavemultipleindividualsreviewandtestyourrubricbeforeyouuseit.

Passingarubric

through

several

tests

prior

to

use

will

confirm

that

your

scales

are

appropriate,thedimensionswillgiveyoutheinformationyouneed,andthe

descriptionswithineachdimensionareclear.

Ensureinterraterreliability:Beforeyouusearubric,itisimportanttotrainthe

individualswhowillbeconductingtheassessmentwithyourrubric.Youwanttoverify

thattheyknowhowtherubricworksandensurethatalltheassessorsunderstandthe

rubricinthesameway.Youmayhavetheassessorsdoapracticerunononeindividual

andverifythattheyratedthestudentsimilarly.Keepinmindthattheassessorsdonot

havetomatchupexactlywiththeirratings,buttheyshouldbeinthesameballpark.If

majordiscrepanciesexist,itiscriticalthattherubricbeadjustedtoclearupany

confusion.

Part2:DevelopingtheAssessmentTool AnumberofresourcesareavailableonBCsassessmentwebsitetohelpdevelopvarious

assessmenttools.StudentVoicealsohasanumberofwebinarstohelpasyoubegin

developingyourassessmenttool.

VPSAAssessmentWebsite:http://www.bc.edu/offices/vpsa/Assessment.html

StudentVoiceWebsite:www.studentvoice.com

Makesurethattheassessmenttechniqueyouchosetargetstheoutcomesyouhopeto

achieve.It

is

easy

to

get

bogged

down

with

satisfaction

assessment,

so

ensuring

your

questionsarefocusedonwhatyoureallywanttoknowiscritical.

Finally,considerwhatkindofinformationiscriticaltoyourstakeholders.Thinkabout

whetheryourstakeholdersaremoreinterestedinquantitativedataorqualitativedata.If

yoursupervisorneedstoknowaboutX,makesureyouincludethatintheassessment!Be

surethatyourassessmenttechniquewillallowyoutoreportbasedontheseneeds.


19/38

19

Part3:Conductassessment/collectdata Usingthetechniqueyouidentifiedinthisstep,conductyourassessment.StudentVoice,

willbeaninvaluableresourceinhelpingyouconductyourassessmentandcollectthe

data.

Timing:Whencollectingdata,considerhowtheacademiccalendarmightaffectresponse

rates.Studentstendtobemuchbusiertowardtheendofeachsemesterandasaresult

maynotbeaswillingtocompletevoluntaryassessments.

Rememberthatstudentsaremuchmorewillingtoparticipateinassessmentswhichthey

feelhassomesaliencetothem.Assessmentswhichstudentsperceivetobeirrelevantwill

generallyhavealowerresponserate.Therefore,highlightingthesalienceofan

assessmentandhowthedatawillbeusedisimportantinordertoincreaseresponserates.


20/38

20

Step 8:Interpret Assessment Data

Makesenseofthedata

Inthisstep,youmustinterpretthedataandmakesenseofitbyansweringquestionssuch

as:

Basedonthedata,didwereachouroutcomes?Ifnot,towhatextentdidwereach

ouroutcomes?

Basedonthedata,arethereadjustmentsweshouldconsiderfornexttime?

Itisoftenbesttoanalyzeyourassessmentwithasmallgroupofcolleagues.Everyone

shouldlookatthedataseparately,andthencometogethertosharefindings.Younever

knowwhatyoumightcomeupwiththisway!

Consideralternativehypotheses

Makesuretoconsideralternativehypotheseswhenevaluatingthedata.Perhapstheissue

youchose

to

focus

on

is

alcohol

use

and

you

have

just

implemented

anew

program.

You

reviewyourdataandyoufindoutthatthereisadecreaseinusageacrosscampusafter

thefirstyearofimplementationoftheprogram.Canyouattributethedecreasetoyour

program?Maybe,maybenot.Youmaywanttochecktoseeiftherewereanypolicyor

enforcementchangesthatmayhavealsocontributedtolowerusagerate.

Waystoevaluate

Lookforpatternsandgapsinyourdata,includingdifferencesbetweensubgroups(males

vs.females,participantsvs.nonparticipants,sophomoresvs.juniors).

UseStudentVoice StudentVoice

has

great

tools

to

help

you

analyze

your

assessment

data.

No

need

to

rememberstatisticsfromyearsago,justuseStudentVoicetohelpyoumakesenseofyour

data.


21/38

21

Step 9:Report/present assessment data and findings

AstheDivisionmovesforwardinitsstrategicplan,itisimportantthatassessmentresultsand

findingsare

shared

with

colleagues

and

other

stakeholders

in

the

Division

who

might

benefit

fromyourfindings.

Identifyyouraudience

Whoneedstoseethereport?Theanswerdependsuponwhatthereportisabout

andtheinterestedstakeholders.Thinkaboutwhosequestionsyouaddressedand

thosewhosequestionsyoudidnot.

Thinkaboutwhowouldbeaffectedbytheimplicationsofthedata,bigpictureand

smallpicture,andsendsomeversionofthereporttothoseindividualsandoffices.

Thinkabout

your

content

and

format

as

it

relates

to

your

audience

Differentpeoplemayalsowant/needslightlydifferentinformation,soitis

importanttotailoryourreporttoeachperson.Youwilllikelyhavemultiplereports

foranyoneassessment.Whenwritingeachreport,considerwhatinformationthe

recipientwillfindmostusefuldonotoverwhelmyourstakeholdersbysendingall

ofyourfindingstoeveryone!

Examplesofwaystopresentyourcontentbasedonaudience:

o Aonepageexecutivesummaryforbusyadministrators

o Apresentationforhealthservicesstaffsothattheycandiscusstheimplications

o Afullreportonthewebtodisseminatetothecampuscommunity

o Apressreleasetogettheinformationtothepublic

Timing:decidewhenthereportwillbedisseminated

Yourtimingisjustasimportantashowtheinformationwillbedistributed.For

example,itmaybeveryineffectivetodisseminatetheinformationatthebeginning

orendofasemesterorduringabreak.Ifapplicable,alsoconsiderthebudgetcycle

andhowyourreportmightimpactresourceallocation.

Considerwhatyoucandotokeepthereportfrombeingshelved

Itisimportanttoconsiderhowyoucangetpeopletoreadthisreport.Thismay

relatebacktoyourassessmenttechniqueandpresentationformat.Forexample,if

theVP

is

anumbers

person

but

your

director

likes

to

read

about

student

stories,

youmaydecidethatyouneedbothquantitativeandqualitativeresultsto

effectivelydemonstratethedata.

Youmayalsodecidetogetothersinvolvedinthebeginning.Sometimesan

assessmentwillbecriticizedonitsmethodbecauseoneormoreconstituenciesdid

notexpecttheresultsortheywerenotpreparedfortheimplications.Forexample,

ifimplicationscouldbeincreasedresidencehallfeesitmightbehelpfultotalkwith

thestudentsenatepriortotheassessment.Youwouldwantthemtobe

supportiveoftheassessmentsothattheyaremorelikelytoaccepttheresults.


22/38

22

Step 10:Make adjustments based on assessment data

Nowthatyouhavethedata,youcanreviewyourfindingsandidentifyparticularaspectsof

theprogram

(or

even

the

program

as

awhole)

that

may

need

to

be

adjusted.

You

may

find

thatyouroutcomesarenotpossibletoreachorthataparticularstrategyisnotworking.

Makingchangesisnotalwaysnecessarysometimesyouwillfindyourprogram/service

wasnotonlyhighlyeffectiveinitsintendedoutcomes,butstudentswerelearningeven

morethanyouthought!Ifthisisthecase,youmayneedtoaddsomeoutcomestoyour

list.

TherearefourpossibleactionsthatyoucantakeinStep10:

Keepcurrentprogramasitis

Tweakprogram

Replacewith

abrand

new

program

Cutprogram

Sometimesyouwillrealizethatyouneedtoassessyourprogramdifferently.Onceany

adjustmentsaremade,theassessmentcyclewillbeginagainforanotherroundbutwitha

slightlydifferenttwist!

Assessmentshouldneverstopsoyoumustidentifywhattoassessnext.Asstudents

constantlychange,ourprogramsandserviceswillneedtobetweakedandadjustedto

continuetobemosteffective.Thinkaboutyournextcycleofassessment.Youmayrealize

awaytoimproveyourassessmenteffortsoryoumayjustwanttoreassesswitha

differentsetofstudentsandseeifyougetthesameresults.

o Forexample,ifyoupreviouslyassessedresidencehallcounciltraining:

Nowyouknowthatstudentsincreasedtheirmeetingfacilitationskills,so

youmaywanttounderstandhowthishappened.Whatwasmostuseful?

Alternately,youmaywanttoassessdifferenttypesofleadershipskillsaside

frommeetingfacilitation.


23/38

23

Thefivedivisionallearningdomainsare:

Intellectual

ReligiousandSpiritual

Ethical

Emotionalsocial

InterculturalandGlobal(InclusiveCommunity)

Learninggoalstatements:

1. Studentswilldevelopgreatercognitivecomplexityandmaturity(Intellectual)

2. Studentswillunderstand,andmorefullyintegrate,ethicsandvaluesintotheirlives

(Ethical)

3. Studentswillengagemorefullywiththeirownfaithorbeliefsanddevelopagreater

awarenessofthereligiousandspiritualidentitiesofothers(ReligiousandSpiritual)

4. Studentswilldevelopgreateremotionalandsocialmaturity(Emotionalsocial)

Studentswillunderstandtheirownculturaltraditionanddevelopgreaterculturaland

globalcompetency(InterculturalandGlobal)

Examplesofoutcomesforeachgoalstatement

INTELLECTUAL:Studentswilldevelopgreatercognitivecomplexityandmaturity.

Examplesof

next

level

outcomes

include,

but

are

not

limited

to:

StudentswhoparticipateinlearningexperiencesprovidedbytheDivisionofStudentAffairswill:

Demonstrateanalytical,reflective,andcreativethinking

Constructnewknowledgeandintegratethisknowledgeintoestablishedideasandex

periences

Thinkcriticallyandproactivelywithregardtolongtermcareergoals

Demonstratetheabilitytolistentoothersandbeopentonewperspectives

Integrateclassroomlearningwithexperientialandpersonallearning

ETHICAL:Studentswillunderstand,andmorefullyintegrate,ethicsandvaluesintotheirlives.

Examplesof

next

level

outcomes

include,

but

are

not

limited

to:


Beabletoclarifyandarticulatepersonal,work,andlifestylevaluesandbeliefs

Developanincreasedunderstandingofsocialjusticeissues

Demonstratetheabilitytochallengeunfair,uncivil,orunjustbehaviorofothers

Actincongruencewithpersonalvaluesandbeliefs

Appendix 1:Divisional Learning Goals


24/38

24

SPIRITUALANDRELIGIOUS:Studentswillengagemorefullywiththeirownfaithorbeliefsand

developagreaterawarenessofthereligiousandspiritualidentitiesofothers.

Examplesofnextleveloutcomesinclude,butarenotlimitedto:


Articulateandexploreissuesofpurpose,meaning,andfaith

DemonstrateanunderstandingoftheJesuittraditionandhowitrelatestoreligious

andspiritualidentity

Compareandcontrastvariousbeliefsystems

Seekoutinvolvementwithindividualsandgroupsfromdifferentspiritualandreli

gioustraditions

Questionactsofincongruencewithreligiousandspiritualbeliefs

EMOTIONALSOCIAL: Studentswilldevelopgreateremotionalandsocialmaturity.



Demonstraterealistic

self

appraisal,

self

understanding,

and

self

respect

Giveevidenceofunderstandingandaffirmingtheimportanceofcivicengagement

andparticipation

Describetheirpersonalstrengthsinrelationtocareeroptions

Demonstratetheabilitytotaketheperspectiveofothers

Seekoutopportunitiestoparticipateinservice/volunteeropportunitiesthatare

characterizedbyreciprocity

Demonstrateanabilitytomanagetheirownbehavioralandemotionalhealth

Takeresponsibilityfortheirindividualchoicesandactions

INTERCULTURALAND

GLOBAL:

Students

will

understand

their

own

cultural

tradition

and

de

velopgreaterculturalandglobalcompetency.



Criticallyreflectupontheirownculturaltraditionsandvalues

Demonstrateanunderstandingofdifferentculturaltraditionsandvalues

Seekoutinvolvementwithindividualsandgroupsfromdifferentculturaltraditions

Giveevidenceofculturalcompetency


25/38

25

Appendix 2:Divisional Mission and Vision Statement

MissionStatement

Themission

of

the

Division

of

Student

Affairs

is

to

facilitate

student

learning

and

formation

in

theirfullestsense(integratingintellectual,ethical,religiousandspiritual,andemotionalsocial

development),promoteaninclusivecommunityofengagedlearners,andadvancetheJesuit,

CatholicheritageandvaluesofBostonCollege.

VisionStatement

TheDivisionofStudentAffairswillbeafullpartnerintheprocessoflearningandformationat

BostonCollege.Byprovidingadiversityofexperientiallearningopportunities,buildingcom

munitiesofrespectandsupport,encouragingandsupportinglivesofintegrity,offeringser

vices

that

remove

barriers

to

learning,

promoting

students

readiness

to

learn,

and

collaborat

ingwiththeDivisionsofAcademicAffairsandMissionandMinistry,StudentAffairswillhelp

studentsengagefullywiththeUniversityseducationalopportunitiesandprogressintheir

journeyofpersonalformation.

StudentAffairsenhancestheJesuit,Catholicmissionoftheuniversitythroughitsprograms,

interactionswithstudents,andmentoring.TheDivisiondoesthisbypromotingalifecommit

tedtoserviceandsocialjustice,sponsoringdialogueonimportantstudentissues,supporting

religiousandspiritualdevelopment,promotingmoraldevelopmentandbehaviorgroundedin

distinctprinciples,andsupportingtheintegrationoffaithwithjustice.Ultimately,theDivision

desirestheactualizationoftheuniversitysmissioninthelivesandhabitsofstudents.

StudentAffairsstrivesforexcellenceineveryaspectofitsorganization,operations,andpro

gramandservicedesignanddelivery;theDivisioniscommittedtoevidencebasedpractice

andtheuseofassessmentdatatostrengthenprogramsandservices.Throughreflective,stra

tegicplanning,carefulallocationofresources,andcomprehensivestaffdevelopment,theDivi

sionworkstoprovidelearningexperiences,programs,andservicesthatproducesoundstu

dentoutcomes.

Inallofitswork,StudentAffairsseekstoknowandunderstandstudents,comprehendand

addresstheirneeds,andrepresentandadvocatefortheirneeds;itintendstoworkwithstu

dentstocreateastrong,healthylearningenvironmentandaninclusive,participatorycampus

culture.One

special

focus

of

the

Divisions

work

is

on

recognizing

and

addressing

the

needs

of

underrepresentedstudents.

StudentAffairsintendstocreateandsustainacaringcommunitywithintheDivision;torecog

nizethecontributionsandachievementsofallstaffandleadership;andtoencourageanap

propriateandsustainableapproachtoselfcareforallmembersoftheDivision.


26/38

26

Afteryouhavewrittenyourlearningoutcomes,lookthemoverandaskyourselfthefollow

questions:

Didyouidentifytheprogram,serviceoractivitythatyouplantoassess?Didyouidentify,

ifnecessary,thespecificaspectoftheprogram,serviceoractivity,thatwillleadto

studentlearning?(ThisiscalledtheCondition)

Didyouidentifythegroupyouwanttoassess?Perhapsthismightbeallstudents,ora

specificgroupofthem.(ThisiscalledtheAudience)

Didyouidentifywhatyouraudienceshouldbeabletoknow,do,orvalueafter

participatinginthecondition?Whenyouwroteitinthelearningoutcome,didyou

includeaBloomverb?(ThisiscalledtheBehavior)

Didyouidentifyhowmany,much,ortowhatdegreeyouexpectyouraudiencetodo

thebehavior?(ThisiscalledtheDegree)

*Note:Theaudience,behavior,conditionanddegreecanbewritteninthe

learningoutcomeinanyorder!

CheckYourself!

IsthelearningoutcomeSMART?

Specific?

Measurable?

Aggressivebut

attainable?

Resultsoriented?

Timebound?

Doesthelearningoutcomefollowthe3

Ms?

Manageable?

Measurable?

Meaningful?

Answerkey:TheresponsestoSMART,the3MsandQ1andQ2

shouldbeyes,allotherresponsesshouldbeno.

Responseswhichdeviatefromtheseindicatethatitis

probablynecessarytoeditthelearningoutcome.

1. Doyouhavemorethanonelearningout

comeperstatement?(Hint:ifyouhavethe

wordand,youmight!)

2. Doyouuseverbslikelearn,

appreciate,valueordevelop?

3. Doesitfocusonoutcomes,notprocesses?

4. Isthebehaviorthelearningsomething

theytheaudiencewasunabletodobefore

exposuretoaprogram?

5.

Areyour

learning

outcomes

wordy

and

complex?(Hint:ifyouhavetoreadthe

learningoutcomemorethanoncetoun

derstandit,itprobablyis!)

Appendix 3:Quick Guide to Discussing Learning Outcomes


27/38

27

Category Definition RelatedBehaviorsandVerbs

Remembering Recallingorremember

ingsomethingwithout

necessarily

understand

ing,usingorchangingit

Arrange,cite,count,define,describe,draw,

identify,label,list,match,memorize,name.

outline,

point

(to),

quote,

recall,

recite,

rec

ognize,record,relate,repeatreproduce,

select,state,tabulate,tell,trace,underline

write

Understanding Understandingsome

thingthathasbeencom

municatedwithoutnec

essarilyrelatingittoany

thingelse

Accountfor,annotate,calculate,change,

classify,compute,convert,defend,distin

guish,discuss,estimate,explain,group,gen

eralize,giveexamples,infer,interpret,para

phrase,predict,reiterate,reword,rewrite,

review,summarize,translate

Applying Usingageneralconcept

tosolve

problems

in

a

particularsituation;using

learnedmaterialinnew

andconcretesituations

Add,apply,adopt,calculate,change,clas

sify,collect,

complete,

compute,

conduct,

construct,demonstrate,discover,divide,

examine, execute,extend,illustrate,imple

ment, interview,graph,makeuseof,ma

nipulate,modify,operate,perform,pre

pare, produce,react,relate,roleplay,

show,solve,translate,use

Analyzing breakingsomething

downintoitsparts;may

focusonidentificationof

partsor

analysis

of

rela

tionshipsbetweenparts,

orrecognitionoforgan

izationalprinciples

analyze,arrange,breakdown,catalogue,

classify,compare,contrast,diagram,experi

ment,differentiate,discriminate,dissect,

distinguish,identify,

illustrate,

infer,

meas

ure,outline,pointout,relate, select,sepa

rate,sort,subdivide,test

Evaluating judgingthevalueofma

terialormethodsasthey

mightbeappliedina

particularsituation;judg

ingwiththeuseofdefi

nitecriteria

accept,appraise,assess,arbitrate,argue,

award,choose,compare,conclude,con

trast,criticize,defend,describe,discrimi

nate,evaluate,explain,grade,interpret,

judge,justify,order,presentacasefor,pri

oritize,rank,rate,recommend,referee,re

ject,

review,

select,

support,

test

Creating/

Generating

creatingsomethingnew

byputtingpartsofdiffer

entideastogetherto

makeawhole.

Assemble,blend,build,change,categorize,

combine,compile,compose,conceive,cre

ate,design,develop,devise,drive,establish,

formulate,generate,group,hypothesize,

integrate,modify,order,organize,plan,pre

dict,prescribe,produce,propose,rear

range,reconstruct,reorder,revise,specify,

tell,transform.write

Jennifer Lackie

Adapted from: http://www.k-state.edu/assessment/slo/blooms.htm, University of Georgia

Appendix 4:SLO Verbs by Blooms Taxonomy Level
http://www.k-state.edu/assessment/slo/blooms.htmhttp://www.k-state.edu/assessment/slo/blooms.htm


28/38

28

Knowledge

ArticulateDefine

List

Label

Locate

IdentifyState

Reproduce

Arrange

OrderRecall

Relate

Comprehension

Describe

Discuss

Explain

LocateRecognize

Report

TranslateClassify

Indicate

Review

Application

Demonstrate

Employ

IllustrateSchedule

Sketch

Use

ApplyInterpret

CompareAnalysis

AppraiseCalculate

Debate

Diagram

Differentiate

RelateSolve

Test

Classify

ExperimentCriticize

Contrast

Synthesis

ArrangeAssemble

Collect

ComposeCreate

DesignFormulate

Organize

Plan

Propose

Evaluation

Appraise

Assess

ChooseCompare

EstimateEvaluate

MeasureScore

Justify

Select

Predict

Appendix 5:Blooms Taxonomy


29/38

29

Appendix 6:Formative Assessment

Althoughassessment

has

traditionally

been

conducted

towards

the

end

of

aprogram,

assess

mentcanbeusedatothertimesaswell.Formativeassessment,orassessmentalongtheway,

canhelpyoubetterunderstandwhetheryourgoalsarebeingmetasyouprogressthrougha

programortheyear.Thistoolwill helpyouidentifysuccessesandchallengesandallowyouto

makeadjustmentsasneeded.Ifstudentsarenotunderstandingaconcept,itisbettertoreal

izethisduringtheyearratherthanwishingthatyouhadknownthiswasaproblemearlier.

Thisguideidentifiesseveralformativeassessmenttechniques,whichvaryinlength,amountof

preprequired,andtheamountofinformationyoucangleanfromtheresults.Notallofthe

techniqueswillbesuitableforeveryprogram,buteachhasitsstrengthsandcanbeusedin

studentaffairs

programming

in

different

ways.

After

picking

atechnique,

it

may

be

modified

tobestmeetyourneeds.

Assessmenttechniquesshouldbechosenbaseduponthespecificcontentorlearningwhichis

tobemeasured.Indecidingwhethertouseaparticulartechnique,considerwhichhasthepo

tentialtogiveyouusefulinformation.Howdoyouthinkthefeedbackyouwillreceivewillbe

helpfultoyou?Ingeneral,donotaskformoreinformationthatyouneedorinformationyou

arenotwillingtouse.Althoughformativeassessmenttechniquesaregenerallyquick,itisal

waysappropriatetoaskacolleaguetoreviewthetechniqueinadvancetoensureitwillcollect

thedesiredinformation.

Afteryoureviewthefeedbackyoureceivefromtheseassessments,itisimportanttosharetheresultswithyourstudentsandidentifyhowyouplantoutilizethoseresults.

Pleasenotethatforthesakeofefficiencyandspace,eachoftheseinstructionshavebeen

writtenusingthetermprogram.However,theyalsoapplytoworkshops,lectureseries,for

mativediscussions,interventions,classes,etc.


30/38

30

FormativeAssessmentTechniques(byskillareaassessed)

PriorKnowledge,RecallandUnderstanding

BackgroundProbe

MisconceptionPreconception

Check

OneminutePaper

MuddiestPoint

SynthesisandCreativeThinking

OneSentenceSummary

ProblemSolving

Problem/IssueRecognition

WhatsthePrinciple?

DocumentedProblemSolutions

TheTechniques(inalphabeticalorder)

Applicationcards

Purpose:Toverifythatstudentsunderstoodthematerialandcanapplyittoareallifesitua

tion.

Todo:Afterteachingaboutanimportanttheory,principle,orprocedure,askstudentstowrite

downatleastonerealworldapplicationforwhattheyhavejustlearnedtodeterminehow

welltheycantransfertheirlearning.Quicklyreadthroughtheapplicationsandcategorize

them.Consider

picking

out

abroad

range

of

examples

and

present

them

to

the

group

to

en

couragepeertopeerlearning.

BackgroundProbe

Purpose:Tocollectspecificandusefulfeedbackonstudents'priorlearningonatopic.Canalso

beusedtogetabetterunderstandingofwhystudentsareparticipatinginaparticularpro

gram.

Todo:Inordertogetageneralideaofwherestudentsarecomingfrom,havethemlistclasses

theyhavetakenonaspecifictopic,trainingstheyhavebeento,and/orexperiencestheyhave

haddoingX,YorZ.Thisisaneffectivewayofquicklygettingapulseonwherepeoplearein

their

experiences

so

that

you

can

tailor

material

appropriately.

ChainNotes

Purpose:Togetonthespotfeedbackwhileprogramisinprogress.Usefulinlargegroups

whenyouwanttogetaquickreadonwhatpeoplearethinkinginthemiddleoftheprogram.

Todo:Atsomepointaftertheprogramhasstarted,passaroundanenvelopeonwhichyou

(administrator/staffmember)havewrittenonequestionabouttheprogram.Thestudents

haveallbeengivenindexcardsbeforehand.Whentheenvelopereachesastudent,heorshe

spendslessthanaminutewritingaresponsetothequestion,thendropsthecardintheenve

lopeandpassesiton.Thisassessmenttechniqueresultsinarich,compositerecordofeach

individualstudentsreactionstotheprograminaction.

ApplicationandPerformance

ApplicationCards

DirectedParaphrasing

StudentGeneratedTestQuestions

StudentAwarenessoftheirAttitudesandValues

SecretPolling

EverydayEthicalDilemmas

LearnerReactionstoTeaching

Chainnotes


31/38

31

Directedparaphrasing

Purpose:Toensurestudentsunderstoodaconcept,process,idea,etc.

Todo:Askstudentstowritealaymans"translation"ofsomethingtheyhavejustlearned

gearedtoaspecifiedindividualoraudience toassesstheirabilitytocomprehendandtrans

ferconcepts.Categorizestudentresponsesaccordingtocharacteristicsyoufeelareimportant.

Analyzetheresponsesbothwithinandacrosscategories,notingwaysyoucouldaddressstu

dentneeds.

DocumentedProblemSolutions

Purpose:Tounderstandwhatthestudentwasthinkingandwhatstepsheorshetooktosolve

aproblem.Thisisalsousefulinhelpingstudentsclarifytheirthinkingandbecomemoredelib

erateintheirproblemsolving.

Todo:Developclearcutinstructionsfortheclassandrecognizethatthestudentswillneed

moretimeforthistaskthanwasrequiredbyyourselforyourcolleagues.Preparethestudents

forthisactivityandassurethemthatthisisnotagradedexercise.Emphasizethatyouareas

interestedin

how

they

approach

aproblem

as

in

obtaining

acorrect

answer

and

that

this

ac

tivitywillhelpthem(andyou)analyzetheirproblemsolvingskills.

MisconceptionPreconceptionCheck

Purpose:Toassessstudentsmisconceptionsorpreconceptionsonaspecifictopicatthebegin

ningofaprogam.Itcanalsobeusedattheendofaprogramtoseeifmisconceptionsare

clearinguporgrowing,whetherpreconceptionsarebeingreshapedandtoseeifstudentsare

improvingintheirabilitiestoassessrelevantinformationandfilteroutinaccuracies.

Todo:Askstudentsaquestion(orafewquestions)beforeandafteraparticularprogram.This

canbedoneinpopcornstyle,aformal(ungraded)quiz,groupwork,etc.

MuddiestPoint

Purpose:Tounderstandwhatstudentsfoundmostconfusing.Thisexerciseasksstudentsto

reflectupontheprogram,todecidewhatwasunclear,andtoselfassesstheirlearning.

Todo:Atendofprogram,askstudentstoidentifyand/ordescribethemuddiestpointofa

workshop(orsegmentoftheworkshop)youareinterestedin.Passoutslipsofpaperorindex

cardsforstudentstowriteon.Collecttheresponsesasorbeforestudentsleave.Respondto

thestudents'feedbackduringthenextclassmeetingorassoonaspossibleafterward.Youcan

specificallyaskstudents"Whatpointremainsleastcleartoyou?"or"Whatimportantques

tiondoyouhavethatremainsunanswered?"

Oneminute

paper

Purpose:Togetinformationfromstudentsattheconclusionofaprogram.Dependingonthe

question(s)youask,thisrequiresstudentstorecallandunderstandthematerial,evaluate

themselvesonwhattheyrecallandselfassesstheirlearningandunderstanding.

Todo:Towardstheendofaworkshop/program/class,askstudentstoansweronahalfsheet

ofpaperaquickquestionthatwillhelpyougaugehowmuchandwhatthestudentslearned.

Examplesinclude: "Whatisthemostimportantpointyoulearnedtoday?",Whatwasmost

helpfulinyourlearningaboutX?orWhatareyoumostand/orleastsatisfiedwith?.


32/38

32

Onesentencesummary

Purpose:Toseethemainideasthatstudentstookfromtheprogram.

Todo:Askstudentstosummarizethemainideasthey'vetakenawayfromalecture,

discussion,orassignedreading.Studentssummarizeknowledgeofatopicbycon

structingasinglesentencethatanswersthequestions"Whodoeswhattowhom,

when,where,

how,

and

why?"

The

purpose

is

to

require

students

to

select

only

the

de

finingfeaturesofanidea.Evaluatethequalityofeachsummaryquicklyandholistically.

Notewhetherstudentshaveidentifiedtheessentialconceptsoftheclasstopicand

theirinterrelationships.Shareyourobservationswithyourstudents.

PrePostTest

Purpose:Toassessstudentlearningbeforeandafteraprograminordertodemon

strateprogression.

Todo:Havestudentscompleteaminitestordoaparticularexercisebeforeaprogram

begins.Havestudentsrepeatthetaskattheendoftheprogram.Analyzethepretest

andpost

test

to

better

understand

the

growth

in

learning

the

students

experienced

as

aresultoftheprogram.

Problem/IssueRecognition

Purpose:Toassesswhetherstudentsareabletoidentifytheprobleminasituation.

Todo:Presentacommonproblem(orseveralexamplesofdifferentproblems)tostu

dentsandaskthemtoidentifythebasictypeofproblemrepresentedbyeachsample.

Theycandosobyashowofhands,groupworkandchoosingthebestanswer,writing

themdown,etc.Thisisparticularlyhelpfulwhenteachingabouthelpingskills.

SecretPolling

Purpose:To

poll

students

who

seem

hesitant

to

respond

to

questions

publicly

or

when

youwantresponsesthatarenotswayedbytheinfluenceofotherstudents

Todo:Askamultiplechoicequestionandassignanumber(15)toeachresponse.

Havestudentsholdupthenumberoffingersthatcorrelateswiththeirresponseto

theirchestsothatonlythepresentercanseetheirresponse.Samplequestionsinclude

IfXhappened,whatwouldofficeproceduresrequirethatyoudo?Putonefingerupif

youthinkY,2fingersifyouthinkZ.Ormoreserioustopicslikehowmanyofyou

haveexperiencedgenderdiscriminationwithinthepastyear?1ifyouhave,2ifyou

havenot

SelfConfidence

Worksheet

Purpose: Togaugeastudentscomfortlevelonaparticulartopic.

Todo:Thiscanbeadministeredeitheratthebeginningofaparticularsessionor

throughoutthesession(s)tomeasureprogress.Inadvance,designasurveywherestu

dentscanratehowcomfortabletheyfeelinanyparticularareas(forexample,public

speakingskills,challengingafriendonhis/herinappropriateuseofalcohol,bookinga

roomthroughEBMS,etc).Studentscanthenratetheirconfidenceonascaleofnone,


33/38

33

low,medium,orhigh.Youcanthenreviewtheselfevaluationsandtailorstrategies

accordingly.

StudentGeneratedHandbook

Purpose:Toseeifstudentsunderstoodaconceptwellenoughtoteachittoothers.

Todo:

Have

students

write

ahandbook

or

manual,

of

any

length

3steps

to

30

pages

regardingacertaintaskthatisgearedtowardsotherstudentswhowouldbelearning

thematerial.Thismaybeparticularlyhelpfulwhentrainingstudentworkersandhav

ingthemwritedowntheinstructionsforwelcomingindividualswhoentertheoffice

orrespondingtofrontdeskinquiries.Reviewthehandbooks/guidesforconcepts

thatthestudentsgraspedormayhavemissed.Givegroupfeedbacktostudentsand

reviewanytopicsorareaswhichprovedtobeparticularlychallenging.

StudentGeneratedTestQuestions

Purpose:Toseeifstudentsunderstoodmaterialenoughtobeabletoworkwithit,ap

plyit,

etc.

Todo:Allowstudentstowritetestquestionsandmodelanswersforspecifiedtopics,in

aformatconsistentwithatypicalcoursesexams.Thiswillgivestudentstheopportu

nitytoevaluatetheworkshop,activity,orprogramtopics,reflectonwhattheyunder

stand,andwhataregoodtestitems.Makearoughtallyofthequestionsyourstudents

proposeandthetopicsthattheycover.Evaluatethequestionsandusethegoodones

aspromptsfordiscussion.

WhatsthePrinciple?

Purpose:Toassesswhetherstudentscanidentifytheprinciple(s)onewoulduseto

solveaproblem.

Todo:ThistechniquefollowstheProblemRecognitiontechniqueinthatonceatypeof

problemiscorrectlyidentified,studentsmustidentifywhatprinciples,techniques,or

standardsintheclassmustbeappliedtosolveorresolvetheproblem.


34/38


35/38

35

Thisconversation

guide

can

be

astarting

point

for

assessing

programs

and

services

in

yourdepartment. Itmaybehelpfultotalkthroughthesequestionsinastaffmeeting,

withyoursupervisor,orwithcolleagueortwoasyoubegintoplanforassessingyour

program.

GUIDINGQUESTIONS

Whatistheprogramorserviceyouwanttoassess?

Whatisitthatyouwanttoknowaboutthisprogram?Whatisthespecificissueor

potentialproblem?

Howisthisprogramalignedwithyourdepartmentsgoals?

Whatdoyouwantstudentstolearn,know,orbeabletodoafterparticipatinginthis

programorinitiative?Thesewillbecomethebasisforyourlearningoutcomes,for

whichyouwillmakelearningoutcomestatements.

Doyouhaveanyoperationaloutcomesinmind,likeattendanceexpectations,reduc

tioninX,etc.?

Whoneeds

to

be

involved,

or

know

about,

this

assessment?

How

might

you

include

themintheplanning?

Whattechniqueswillyouusetoconductyourassessment? Doyouneedfurther

trainingorinformationinordertodothis?Consulttheworkshopscheduleoronline

webinarscheduleonStudentVoice(www.studentvoice.com)

Whendoyouwanttoconductyourassessment? Checkwithyourdirectortosee

whatelsemightbegoingoninthedepartmentoracrossthedivisionwhichmight

influenceyourtiming.

Appendix 7:Pre-Assessment Conversation Guide
http://www.studentvoice.com/http://www.studentvoice.com/


36/38

36

Pleasedescribewhyyouwanttoundertakethisassessmentbyrespondingtothefollowingitems:

1.Whatisthepurposeofyourassessment?

2.Whatistheprogramorservicebeingassessed?

3.What

are

the

specific

department

or

program

goals

to

which

this

is

related?

4.UnderwhichoftheDivisionalLearningDomain(s)doesthisassessmentfall?(Intellectual,

ReligiousandSpiritual,Ethical,EmotionalSocial,orInterculturalandGlobal)Note:notallprojects

willfallunderaspecificlearningdomain,particularlyifyouareassessinganoperationaloutcome.

5.Whatexistingdatahavebeenconsidered(e.g.,datafromInstitutionalResearch,Student

Services,otherStudentAffairsdepartments)?

6.Whatisyourtargetsample?

7.What

assessment

techniques

will

you

use?

(e.g.

survey,

focus

group,

rubric,

interview,

mobile

datacollection)

8Willtheassessmenttoolidentifyrespondentsorwillrespondentsremainanonymous?

9Howwillyouusethecollecteddata?

10.Withwhomandhowdoyouplantosharethecollecteddata?

11.Isthisaonetimeassessmentorwillitberepeated?Ifrepeated,pleaseindicatetheproposed

schedule.

TitleofProject:

NameofProjectsponsor:

Department:

Todaysdate:

Proposeddateofassessment

launch:

TitleofProject:

NameofProjectsponsor:

Department:

Todaysdate:

Proposeddateofassessment

launch:

DIVISIONOFSTUDENTAFFAIRSSTUDENT

VOICE

ASSESSMENTPROJECTREQUEST

FORM

Appendix 8


37/38

37

INSTRUCTIONSFORCOMPLETINGANDUPLOADINGPROJECTREQUESTFORM

Theprojectrequestformismeanttobeastartingpointforyourproject. Itisimportantto

bethoroughandthoughtfulincompletingalloftheitems,yetyoushouldalsobebrief.It

shouldtakelessthan30minutestocompletetherequestform. Thisformwillbeusedin

yourinitial

consultation

with

the

StudentVoice

consultant

and

will

be

reviewed

and

ap

provedbyKatieODairandJessicaGreenefromInstitutionalResearch.

1. Filloutallitemsontheformandsaveittoyourdesktoportotheserver.Youmaycon

sultwithyourassessmentcommitteeliaisonorKatieODairatanytimeforassistance.

LogintoStudentVoice:www.studentvoice.com.

1. GotothemyStudentVoicetab

2. Ontheleftside,clickrequestaproject

3. Filloutallrelevantprojectinformation. Foranewproject,clicknewproject

4. IfyouareusingaPDA,checkallandlistthemobiledevicenumbers(Katiewillhave

thesewhen

you

check

it/them

out)

in

the

comment

box.

This

will

tell

StudentVoice

to

sinc

theprojecttothePDA.

5. Underemailinformationclickunsureifyouwouldliketotalkaboutadministeringthe

assessmentwiththeSVconsultant.

6. Underadditionalassistance,clickyesifyouwanthelpfromaconsultantonthedesign,

phrasing,andadministrationoftheassessment.

7. ClickNext

8. UnderprojectnoteswriteinanythingyoudlikeyourSVconsultantorKatieorJessto

knowaboutspecifictothisproject. Clicksubmit.

9. Youwillbetakentoyourprojectdashboard.UnderProjectfilesuploadyourProject

RequestFormbybrowsingyourfilesanduploadingit. Ifyouhaveadraftsurveyorques

tionsto

submit,

make

sure

you

upload

those

as

well.

Make

sure

you

press

upload

after

youbrowse! Youwillknowitwassuccessfulwhenyouseeyouruploadeddocumentsap

pear.

10. Oncecompleted,KatieODairandJessicaGreenewillreviewandapproveeachrequest

andyourStudentVoiceconsultantwillcontactyou. Thiswillbedonewithinoneweekof

uploadingyourform.

11. Pleasenote,someprojectsmayrequiretheadditionalstepofpreparingdocumentsfor

theInstitutionalReviewBoardprocess.KatieandJessicawillassistwiththisstepifrequired.

12. TheStudentVoiceconsultantwillthenworkwithyoutocompleteyourassessment
http://www.studentvoice.com/http://www.studentvoice.com/


38/38

38

2011 New Handbook

Documents

Transcript of 2011 New Handbook