Post on 11-Apr-2019
ThisworkislicensedunderaCreativeCommonsAttribution-NonCommercial4.0InternationalLicense.
1
“Spendanotherdayinourclasstalkingaboutthisresearchplease”:Studentinsightsfromaresearch-baseddesignthinkingexercise
AtmanCynthiaJ.a,ArifAhmera,ShroyerKathrynE.a,TurnsJenniferA.a,Borgford-ParnellJimaaUniversityofWashington*Correspondingauthore-mail:{atman,ahmer,kshroyer,jturns,bparnell}@uw.edu
Abstract:Thispaperexploreshowguideddiscoverycanbeusedtoconnectinsightsfrom the ever-growingbodyof researchondesignprocesseswithdesign teaching.This paper focuses on a specific instance of a guided discovery activity in whichengineeringstudentswereinvitedtoengagewithselectedtimelinesfromastudyofdesignerprocesses;guidanceincludedpromptsattwopointsintime.Thegoalwasto see if the students could discovermeaningful insights about the design processandwhatfeaturesofdesignprocessescontributetoqualitysolutions.Thestudentsin this study succeeded in discovering six meaningful insights about the designprocess. The distribution of students’ insightswas not the same at the two time-points,suggestingthattheguidance is important inwhatstudentsdiscovered. Ourfindingsspeaktothevalueofthespecificguideddiscoveryactivitythatwestudied,andtheoverallideaofdevelopingactivitiesusingguideddiscovery.
Keywords:designpedagogy;guideddiscovery;designrepresentations;designexpertise
1.IntroductionAnever-growingbodyofresearchdocumentsinsightsintofeaturesofdesignprocessesthatcontributetosuccessfuldesignoutcomes.Intheirsynthesispaper“EngineeringDesignThinking,Teaching,andLearning,”Dymandhiscolleagues(2005)havecalledattentiontodivergent-convergentquestioning,thinkinginsystemsterms,anddecisionmakingasimportantaspectsofsuccessfuldesignprocesses.Inanotherexample,twooftheauthorsandtheircolleagueshaveinvestigatedhowtheamountoftimespentgatheringinformationandthefrequencyoftransitionsacrossdesignactivitiescontributetosuccessfuldesignprocesses(Atmanetal.,1999;Atmanetal,2007).Stillfurther,anexaminationofanyissue
AtmanCynthiaJ.,ArifAhmer,ShroyerKathrynE.,TurnsJenniferA.,Borgford-ParnellJim
2
inDesignStudiesinevitablyuncoversadditionalresearch-basedconsiderationsabouthowfeaturesofthedesignprocesscontributetosuccessfuldesignoutcomes.
Thispaperisconcernedwithhowsuchfindingscanbeleveragedtoinformdesignteaching.Onegeneralpathwaycouldbeembeddingthefindingsintheenvironment.Forexample,oneapproachwouldbetoprescribeadesignprocessthatdirectsstudentdesigners’attentiontowardstheveryactivitiesthathavebeenshowntobeimportantinthebodyofresearchondesignthinking.Anillustrationofthiscouldbeastudentdesignerbeinggivenasystemthathelpswitheffectivequestioning.Arelatedapproachwouldbetocreateresources,suchasvirtualagents,thatcouldremindadesignerwhentheyaredeviatingfromsuchdesirableapproachesinordertohelpthedesignergetbackontrack.Withthecurrentfocusonproblem-basedlearningandprojectworkforteachingdesign,theembedthefindingsintheenvironmentapproachmayseemfamiliar.Forinstance,muchoftheconversationaroundgettingprojectbasedlearningrightinvolvesnavigatingquestionsofhowtostructureanenvironmentsostudentssucceed.
Incontrast,anotherpathway(exploredinthispaper)involvesembeddingthefindingsintothedesigner’smentalmodelofthedesignprocess.Inotherwords,inthispathway,theideaistohelpthestudentdesignerunderstandresearch-basedfindingsaboutdesignthinkingsothatthefindingsarelateravailablewhentheyengageinmetacognitive-leveleffortssuchasplanningofdesign,reflectioninaction,andexecutivecontrol.Suchmetacognitive-leveleffortrelatestodesigners’abilitiesto“usedesignstrategieseffectively”—keyperformancedimensionsidentifiedintheextensiveliteraturereviewbyCrismondandAdams(2012,p.745).Withthispathway,ageneralquestionishowtohelpthestudentdesignercometounderstandandappreciatethesefindings.Astraightforwardstrategytoachievethisobjectivewouldbetopresentresearchfindingstostudentsoraskthemtoreadresearcharticles.However,thisstrategyisnotoftenreportedtobesuccessful.
Inthispaper,weexploreanalternativemethodofembeddingthefindingsintothedesignerthroughaformofguideddiscoverylearning.Discoverylearningisaninductivelearningapproachinwhichstudentsreceiveaproblemtosolvewithlittleornoguidancefromtheteacher.Guideddiscoveryisatypeofdiscoverylearninginwhichthestudentreceivesaproblemtosolvebuttheinstructoralsoprovidesfocus,coaching,feedback,andothersuchguidancetodirectthestudents(Mayer,2004).Guideddiscoveryattemptstoremediatethechallengesthathavearisenthroughcompletelyopendiscoverylearning(seeAlfierietal.,2011;Bruner,1961;DeanandKuhn,2007Kirschneretal.,2006;andMayer,2004)
Inthisguideddiscoveryactivity,studentsinteractwithresearch-basedvisualrepresentationsoffirst-year(freshmen)andgraduating(senior)studentdesignprocessestoseeiftheycandeveloppersonalinsightsaboutdesignthataresimilartofindingsdiscoveredbytheresearchers.Weprovideaworkedexampleforoneoftheconferencethemes,specificallyusingdesignresearchasanactiveforcethatallowsdesignstudentstorethinktheirideasaboutdesign.
“Spendanotherdayinourclassplease”:Studentinsightsfromaresearch-baseddesignthinkingexercise
3
Pragmatically,ourapproachdoesnotrequireachangingofprojectstructuresasanembedthefindingintheenvironmentapproachmight.Sinceprojectsarehardtostructure,itcanbechallengingtorestructuretheenvironment.Ourapproachprovidesstudentdesignerswiththeopportunitytodirectlyexploreresearchfindingssothattheycanlearnaboutimportantaspectsofdesignprocesses;aspectsthatmaycometopositivelyinfluencetheirownfuturedesignactivities.Giventhepotentialbenefitofsuchaneducationalactivity,wesetouttoanswerseveralquestions:
• Willstudentsdiscoveranyinsightsaboutdesign?Whatinsightsaboutdesignwilltheydiscover?
• Howcanwecharacterizethetypeofguidanceprovidedtothestudents?Whatwillbetheeffectoftheguidance?
• Whatwillbestudent’sreactionstotheirdiscoveriesandtotheexperienceofbeingaskedtodiscover?
Inthispaperweexplorethesequestionsthroughtheinstantiationof,andexperimentationwith,aguideddiscoveryactivity.Thepaperrepresentsanempiricalproof-of-concept,showcasingwhatsuchanactivitymightlooklikeandgatheringevidencethatdemonstratesthatsuchanactivitysuccessfullyhelpsstudentslearnaboutdesign.
Therestofthepaperproceedsasfollows.Firstwedescribeourspecificinstanceofguideddiscoveryofdesignprocesses.Thecontributionofthepaperisastudyoftheguideddiscoveryactivityinaction—themethodsandfindingsoccupythebulkofthepaper.Intheconclusionwediscussthesignificanceofthework.
2.TheResearch-BasedGuidedDiscoveryActivityResearchhasdemonstratedthatengineerswithdifferentlevelsofexperience(first-yearstudents,graduatingstudents,expertpractitioners)exhibitdifferentpatternsofdesignactivitieswhentheysolveadesignproblem(Atmanetal.,2007).Thesedifferencesaremadevisuallyapparentwhentheyaredisplayedastimelinesthatindicatethetimespentindifferentdesignactivitiesandthenumberoftransitionsamongdesignactivities.Timelinesfromsixengineeringstudents,threefirst-yearstudentsandthreegraduatingstudentsformthebasisoftheguideddiscoveryactivity(seeFigure1).
AtmanCynthiaJ.,ArifAhmer,ShroyerKathrynE.,TurnsJenniferA.,Borgford-ParnellJim
4
Figure1–Timelineactivityhandout.Timelinesrepresenttypicallow-performing,average-performing,andhigh-performingfreshmanandseniorengineeringstudents.
Thestepstotheactivityareasfollows(totaltimeis50to60minutes):
1. Aneducatorgivesabriefpresentationaboutthedevelopmentofthedesignprocesstimelinesusedinthetask(10minutes).
2. Studentsareeachgivenaworksheet(thetimelineactivityhandout)withquestionsonbothsides(Figure2).StudentsaregivenfiveminutestoindividuallyanalysethetimelinesandrespondtoPrompt1onthefrontofthesheet(Figure1).
3. Studentsthendiscusstheirresponseswiththeirprojectteams(10minutes).
4. Theeducatorthenleadsadiscussionwiththefullclassaboutstudentinsights.Additionalresearchfindingsarepresented,includingresultsfromasampleofexpertengineeringpractitioners,andthestatisticallysignificantresultsfromthecomparisonacrossthethreegroups(20to25minutes).
5. Studentsthenturnthepageoverandtake5to10minutestorespondtoPrompts2and3(seeFigure2foranexamplescanandtranscribedanswerfromastudentparticipantfromtheclassdiscussedinthispaper).
“Spendanotherdayinourclassplease”:Studentinsightsfromaresearch-baseddesignthinkingexercise
5
Figure2–ActivityworksheetandStudent22’sResponse
AtmanCynthiaJ.,ArifAhmer,ShroyerKathrynE.,TurnsJenniferA.,Borgford-ParnellJim
6
ThePromptsareasfollows:
Prompt1:ACTIVITY:Inthedesignprocesstimelinesshownabove,whatsimilaritiesanddifferencesdoyouseebetweenthefreshmenandseniorengineeringstudents?Dothesesimilaritiesalsoinvolvethequalityscores?Howso?
Prompt2:REFLECTIONACTIVITY:Whatwasthemostimportantthingyoulearnedtoday?Why?
Prompt3:REFLECTIONACTIVITY:Howcanweimprovethistalkforfutureaudiences?
Inthecaseofthisactivity,guideddiscoverylearningwasimplementedinsteadofpurediscoverylearningtoincreasethelikelihoodthatstudentswithlimiteddesignexperiencewouldbeabletouncoverinsightsinashortperiodoftime.Theamountofdata,representationofdata,andguidingquestionswerespecificallyscopedtodirectstudentsthroughtheactivitywhilestillallowingforautonomytodiscovertrends.Forexample,insteadofpresentingstudentswithalargeamountofrawdata,sixspecificdesignprocesseswerevisuallyrepresentedintheformoftimelines.
Ratherthansimplylisteningtoalectureorbeingprovidedwithageneralizeddesignprocessdiagram,studentswerepurposefullyguidedtoinductivelyuncovertrendsfromdesignprocessdata.ByexploringthetimelinesindependentlypriortorespondingtoPrompt1andsubsequentlywiththeirteammates,studentsdeveloprelevantinsightsintodesignprocesses.Followingthis,theeducatorthenfacilitatesadiscussionwiththefullclassandcomparesthestudentinsightstoresearchfindings.Students’finalperspectivesonimportantlessonsarethensolicitedwithPrompts2and3.Inotherinstanceswheretheauthorshavepresentedthisexercise,audiences(students,educatorsandpractitioners)haveidentifiedthestatisticallysignificantdifferencesacrossthetwostudentgroupsinthetimelinesjustfromthisguideddiscoverytask.Audiencememberobservationsarereinforcedwhentheylearnabouttheexperimentalresults,increasingbothconfidenceandexcitementwiththeirdiscovery.
Student22’sresponse,whichispresentedinFigure2,demonstratesboththebreadthanddepthofinsightsthatstudentscangainwiththisexercise.
3.Methods
3.1ParticipantsTwenty-fourmechanicalengineeringstudentsinathird-yearIntroductiontoDesigncourseatalargeresearchuniversityparticipatedinthisclassroomactivityinthespringof2015.
3.2CodeBookTheresponsesfromthetwenty-fourstudentswereanalysedbasedontheirwrittenresponsestoPrompts1and2.TheresponsestoPrompt3werenotincludedinthisanalysis(thepurposeofPrompt3istosolicitinputaboutteachingactivityratherthanstudent
“Spendanotherdayinourclassplease”:Studentinsightsfromaresearch-baseddesignthinkingexercise
7
learningaboutthedesignprocess).Theresponsestothefirsttwopromptswerecodedforpresenceorabsenceoftendesigninsightsineachresponse.Theseinsightswerebasedonninecodesusedinapreviousanalysisofthisactivity(Borgford-Parnell,2010).Anadditionalcode(Time)wasaddedandseveralcodeswereclarifiedresultinginafinalsetoftencodes.
Inourfindingsfromthisdataset,onlysixofthetencodeswereprevalent(Breadth,ProblemDefinition,GatheringInformation,Modelling,Iteration,andTime).Inthispaper,wethereforefocusonthosesixcodes.
CodeBook
Name Definition Examples–Prompt1 Examples–Prompt2
BreadthBreadth(accomplishingmoreofthedesignprocess)correlateswithhigherqualitydesign
“Highqualityscoreswerevariedinthedesignprocess,withallstepsprevalentthroughoutthetimeline”-17
“Toproduceagooddesignyoumustsplityourtimemoreevenlybetweenthedifferentstages.”-11
ProblemDefinition
Moretimespentonproblemdefinitionactivitycorrelateswithhigherqualitydesign
“Betterscoresreturnedtotheproblemdefinitionthroughouttheprocess…”-2
“Alwaysgobacktotheproblemandaskyourselfifyouarereallysolvingit.”-11
GatheringInformation
Moretimespentongatheringinformationactivityequatestohigherqualitydesign
“Somethingthatseemedtopromotehighqualityworkwastocontinuetokeepgatheringinformationandgeneratingideasthroughoutthewholeprocess.”-12
“Lastlyitisimportanttocontinuetoaskquestions&gatherinformationthroughouttheprocess.”-23
Modeling Everyonespendsthemosttimeinmodelingactivity
“Modelingtookupthemajorityofthetimeformostpeople.”-3
“…thoughevenexpertsspendthemosttime/effortintomodeling.”-16
IterationIterationand/ortransitionscorrelatewithhigherqualitydesign
“Thehighqualitydesignsbouncedaroundalot…”-19
“Thedesignprocessismosteffectivewhenundertakenasanonlinear,cyclicalprocess.Thedifferentactivitiesor‘phases’ofthedesignprocessshouldbeintegratedandrevisitedthroughoutdesigning.”-6
TimeTotaltimespentonthedesignprocesscorrelatestohigherqualitydesign
“Studentsthatspentlesstimehavelowerqualityscore.”-10
“I[learned]thatputtinginenoughtimeisvaluableuptoacertainpoint…”-15
Figure3-Descriptionsandexamplesofthesixcodes
3.3CodingProcessThecodingprocesswascarriedoutbytwotrainedresearchersthatindependentlyassigneddesigninsightcodesatthesentenceorsentencegrouplevelsforeachofthe24setsofstudentresponses.Sincewecodedresponsestotwoquestionpromptsforeachstudent,theactualnumberofresponsesthatwereanalysedwas48.Thiscodingprocessinitiallybeganusingtheaprioricategoriesthatweredevelopedusingabottom-upapproachinapreviousstudythatisextendedbythisresearch(Borgford-Parnell,2010).Theresearcherstrainedonaseparatesetofstudentresponses.Oncefinished,thetworesearcherscomparedresults,arbitratedanydiscrepanciesonthistrainingsettoagreement.Minorrefinementsweremadetotwoofthecodedefinitionsanda10thcode(Time)wasadded.
AtmanCynthiaJ.,ArifAhmer,ShroyerKathrynE.,TurnsJenniferA.,Borgford-ParnellJim
8
Aftertrainingwascomplete,theresearchersreplicatedthisprocessontheforty-eightstudentresponsesusedinthisstudy.Afterthisfirstiteration,theresearchersleveragedtheiremerginginsightstomakeminorrefinementstofourofthesecodesforthesakeofspecificityandinclusiveness.Additionally,theresearchersnoticedpatternsofdifferencesinthelevelofstudents’engagementandinsightsbeingdemonstratedwithinthetexts.Opportunistically,theresearchersbeganlookingforthosepatternsandannotatingforstatementsthatseemedtogobeyondtheacquisitionofdesigninsights.Statementsinthiscategoryhadtoindicatethatthestudentwasnowmorecuriousaboutdesignorwassomehowrelatedtotheirself-identityasadesigner.Theresearchersreported22agreementsoutof24judgementsforthisBDIcode(BeyondDesignInsights)andconsequently,wehavechosentoincludeitaspartofourdiscussionduetosomeoftheinsightsitsurfaced.
Uponcompletingtheiranalysis,theresearcherscomparedresultsandarbitrateddiscrepanciestoagreement.Twojudgements(outof480)requiredexternaladjudicationthroughamajorityvotebyathirdresearcher.Forthe6mostprominentinsightsdiscussedinthispaper(atotalof288judgements)26werebrieflydiscussed,10warrantedfurtherdiscussionorclarification,and1requiredadjudicationbyathirdresearcher.Priortothisarbitration,theinitialinter-raterreliabilityforthesesixcodeswasalsocomputedusingCohen’sKappa(1968)andfoundtobe0.78,0.59,0.88,0.85,0.83and0.86respectively.
4.FindingsDespitetherelativelyshortnatureofthedesigntimelineexercise,itprovokedavarietyofstudentreactions—manyofwhichwerecharacterizedbybuildingastrongerappreciationforconsideringafullerrangeofdesignactivities.Inwhatfollows,wefirstdescribethehighlevelresultsofouranalysis,includingattentiontohowstudentreactionsshiftedbetweenthetwoprompts.Wethendescribetheinsightsthatstudentstakeawayfromtheexercise.
Figure4showsthedistributionofthe89designinsightsthatresearchersobservedacrossthe48studentresponsesthroughouttheactivitybycategory.
Figure5providesadditionaldetailontheseresultsbyshowingthenumberofstudentsreportingeachinsightforPrompt1(thelightbars)andPrompt2(thedarkbars).RecallthatPrompt1capturesthestudents’initialinsightswhilePrompt2isdesignedtocapturefinaltakeawaysafterapresentationofresearchfindingsandagroupdiscussion.AllbutfiveofthestudentsaddedatleastonenewideatotheinsightstheyincludedintheirresponsetoPrompt2.Asthefigureshows,someinsightssuchasBreadthandGatheringInformationwerementionedwithsimilarfrequencyinresponsetoPrompt1andPrompt2.Otherinsights,suchasIteration,ProblemDefinition,Modelling,andTimeweredifferentiallyprominent.
“Spendanotherdayinourclassplease”:Studentinsightsfromaresearch-baseddesignthinkingexercise
9
Figure4-Thetotalnumberofstudentinsightsbycategory
Figure5-Thenumberofstudentinsightsbycategoryandprompt
Below,weprovideadescriptionofthestudentresponsesthatcomprisethesixcodedcategories.WealsopresentourfindingsfortheBDI(BeyondDesignInsights)codeinthissection.Thestudentquotesthatwehighlightinthissectionhavebeenselectedforbeingrepresentative,interestingandfortheirillustrativepower.
0 4 8 12 16 20 24
Modeling
ProblemDe9inition
Time
GatheringInformation
Iteration
Breadth
NumberofStudentswhoExpressedDesignInsight
DesignIn
sight
NumberofStudentswhoExpressedDesignInsight(forwholeexercise)
0 4 8 12 16 20 24
Modeling
ProblemDe9inition
Time
GatheringInformation
Iteration
Breadth
NumberofStudentswhoExpressedDesignInsight
DesignIn
sight
NumberofStudentswhoExpressedDesignInsight(forPrompt1andPrompt2separately)
Prompt1Prompt2
AtmanCynthiaJ.,ArifAhmer,ShroyerKathrynE.,TurnsJenniferA.,Borgford-ParnellJim
10
4.1StudentInsight:Timematters(Time)Oneoftheinitialinsightsreportedbyfourteenofthestudentsafterexaminingthedesigntimelinesrevolvedaroundequatingoveralltimespentwithgoodqualitydesign.Thiswasgenerallyexpressedinafairlysuccinctmannerbystudentswithstatementssuchas“themoretimeyouspend,themorelikelyyoucangetagoodscore”and“studentsthatspentlesstimehavealowerqualityscore”(students4&20).Interestingly,whilstfourteenstudentsconsideredthisanotableobservationtoshareinitially,onlytwoofthemchosetodiscussitasapointoflearninglateroninresponsetoPrompt2.Thiswouldseemtoindicatethatmanyofthesestudentsdidn’tconsiderwhattheycouldmakeofthisobservationtobeatakeawayimportantenoughforthemtoincludeastheirresponsetoPrompt2.
Threeofthestudentsalsofurtherqualifiedtheirobservationsbynotinghowtimespentstopsbeingafactorpastacertainpoint.Forinstance,student15wrotehowhe“learnedthatputtinginenoughtimeisvaluableuptoacertainpointbutthenamoreiterativedesignprocessbecomeshelpful”.Thesenuanceseffectivelyforeshadowtheotherthemeswhichthestudentswroteaboutregardinghowtimeshouldbespentindesignwhilestillpayinghomagetotheideathatacertainamountoftimeexpenditureisinevitableifqualityisdesired.
4.2StudentInsight:Timespentgatheringinformationraisesquality(GatheringInformation)Withinthethemeofhowtimeshouldbespent,twothirdsofthestudentswroteabouthowdesignerswith“goodscoresgatheredinfothroughouttheprocess”.Therelativelyevensplitofresponses(eleveninPrompt1andonlyanadditionaltwoinPrompt2)suggeststhatstudentscametothispointbythemselvesandwereattentivetoitssignificance.
Oneminordistinctionthatwecouldseeinthestudentresponsescamefromhowthemajorityofthemsawhighscorersandexpertsgatheringinformationcontinuouslythroughoutthedesignprocesswhileothersperceivedthattheinformationgatheredinanearlierphasecarriedmoreweight.Sowhilethreestudentswrotethattheylearned“towaituntilmoreinformationisgathered...tostartmodelingideas”(student8),otherswroteofhavinglearnedthatthereisnoendpointtotheinformationgatheringphase.Forexample,student5wrotethattheylearnedto“continuegatheringinformation,eventowards[the]end”,whilestudent7expressedthatshewould“trymyhardesttoavoidmodelingtooearly,andalsowhenIstartmodelingto‘comeupforair,’sotospeakandtakealookattheinformationgivenandmaybetheinformationnotgiven”.
4.3StudentInsight:Keeprevisitingtheproblem(ProblemDefinition)Interestingly,whileinformationgatheringandproblemdefinitionfrequentlygohandinhandasdesignactivities,therelationshipbetweendesignqualityandproblemdefinitionactivitieswastoucheduponbycomparativelyfewerstudents.OnlytwostudentsbroughtuptheproblemdefinitionactivityasafactorindesignqualityscoresinresponsetoPrompt1,whileanotherninedidsoafterdiscussioninPrompt2.Thissharpdifferencewouldindicatethat
“Spendanotherdayinourclassplease”:Studentinsightsfromaresearch-baseddesignthinkingexercise
11
observationalonewasnotenoughforstudentstomakealinkbetweenproblemscopingandfinaldesignqualityandthatdiscussionsbroughtthestudentstosurfacecertainkindsofinsights.Thiswasfurtherreinforcedbytherelativelyuniformtoneofstudentresponseswhichgenerallyexpressedthisthemeintermsofrevisitingproblemdefinition:“Itiseasytogetcaughtupinmodelingbutifyoudon'tgobacktotheproblem,youwillmostlikelyfailtomakeagooddesign”(student4).Thislackofvariationsuggeststhattheclassdiscussionpossiblyhelpedshapethestudentresponsesatthelanguagelevel.
4.4StudentInsight:Everybodyspendsthemosttimemodellingthesolution(Modelling)Significantly,manystudentresponsesseemedtorecognizethatmodelling—theactivitywhichconsumedthemostdesigntime—wasnegativelycorrelatedwithquality.Forinstance,student16offeredaconclusionthat“themostimportantthingIlearnedtodaywasthatwhenyoudesignaproduct,anidea,oranything,youhavetotrytonotonlyfocusonmodeling”.Sevenofthestudentswereparticularlyattentivetohowthetimelinesshowedmodellingasthemosttimeintensiveactivityformostdesignersinthesample.Forexample,students16and3respectivelynotedthat“evenexpertsspendthemosttime/effortintomodeling”and“modelingtookupthemajorityofthetimeformostpeople”.Thisobservation—thatnobodygetsawaywithnotdoingasignificantamountofsolutionmodellingduringdesign—canpotentiallyhelpstudentsdemystifyandbetteridentifywiththeworkprocessesofmoreexperienceddesigners.
4.5StudentInsight:Breadthisthekey(Breadth)Althoughstudentslinkedseveraldesignprocessactivitiestodesignquality,themostprolificinsightemergedfrom22ofthe24studentslinkinghighscoreswithaholisticdesignprocessintermsofactivities.Ineffect,studentswerenotinghowthemoreanactivitysuchasmodellingappearedasacontinuousblockonthetimelines,thelesstimewasthenavailableforotherimportantactivities.Theequalsplitandoverlapofstudentswhochosetocoverthisthemeintheirinitialobservationsversustheirchieftakeawayswouldindicatethatthisinsightwasbothreadilyapparentandveryimportanttothestudents.
Therichdiversityofexpressionusedaroundthiscodeisanotherinterestingfeaturethatwarrantselaboration.Forinstance,somestudentslikestudent12wroteaboutnot‘gettingstuck’on‘oneidea’orphasewhilereflectingupontheirowndesignprocess:“[Ilearned]thatspendingtimeonallaspectsofdesigniscriticalforaqualityproduct…Thisissoimportanttomebecauseitiseasytogetexcitedaboutacertainideaandforgetwhatthemainthingwasabout.”Otherstudentsofferedtheirconclusionsthroughideasregardingevendistribution,revisiting,balance,andintegrationwithrespecttodesignactivities.Itisdifficulttofirmlyconcludewhetherthiswidervarietyofphrasesindicatesadeeperormoregenuinelevelofengagement,butneverthelesswefoundthiscombinationoffrequencyanddiversitytobetelling.
AtmanCynthiaJ.,ArifAhmer,ShroyerKathrynE.,TurnsJenniferA.,Borgford-ParnellJim
12
4.6StudentInsight:Iterationandtransition(Iteration)Thesecondmostsystematicallyprevalentsetofinsightsarosefromnineteenofthestudentswritingabouthowgooddesignprocessesareiterativeinnature.Studentsexpressedthisconceptinavarietyofways.Someresponses,likestudent17’sreferredtocyclingorrepetition:“Thereseemstobeaclearcorrelationbetweenthequalityofthedesignandtheabilityofthedesignertokeepcyclingthroughallstepsoftheprocess”.Otherresponsesusedtermssuchas‘jumpingbackandforth’(student4),‘goingcircular’(student21),being‘nonlinear’(student19),etc.
Manystudentsalsoexplicitlycalledoutthepatternstheyobservedinhighqualityscoresasiterative.Forinstance,student3referredtoiterationdirectlyinthecontextofreflectinguponhisowndesignprocess:"Alsoseeingtheiterativestepsthatweremadewillmakemefeelbetteraboutsteppingbackandlookingeverythingover".SinceonlyeightresponsescontainedthiscodeinPrompt1versusseventeeninPrompt2,itseemsthatalthoughthiswasavaluabletakeawayformanystudents,itwasnotinitiallyapparent.Thatsomeamountoftimeordiscussionwasprobablyhelpfultosurfacingthisinsightcouldbeexplainedbythefactthattheiterationpatterncouldonlybeobservedafterabsorbingallsixtimelinesandsteppingbackforsynthesis.
4.7BeyondDesignInsights(BDI)Apartfromtheaboveinsights,morethanathird(10outof24)ofthestudentresponsescontainedelementsthatextendedbeyondlearningaboutdesignprocesses.Inmanyways,thesewerethestatementsthatthestudentswrotewhichdidnotdirectlyaddressthepromptstheyweregiven,butpointedtoreactionsthatweconsidertobesignificant.Unsurprisingly,mostoccurrencesofthisBDIcodeshowedupinthesecondresponsethatwassolicitedfollowedaclassdiscussionthatfeaturedresearchfindings,furthersupportingourfindingsconcerningthevalueofguideddiscovery.Inthisprompt,thestudentswereaskedtoengageinareflectiveactivitywheretheyselectedthemostimportantthingtheylearnedintheexercise.However,therewereanumberofinstancesevenduringthePrompt1responseswheresomestudentsmovedpastthedataobservationpromptandplacedthemselvesinthesituationofthedesignersintheexerciseorattemptedtogiveadviceonwhatnottodo.Forinstance,student13’sresponseendedwith“formeIwouldspendmoretimewith[the]decision[activity]thantheothersdid”.
OneofthemostfrequentlyexpressedBDItakeawaysatanoveralllevelwasanappreciationfor‘showdon’ttell’,whichtiedintoseveralotherthemes.Forinstance,thequoteforthetitleofthispapershowsthisparticipantexpressingthisappreciationwhiletyingitintowhatweinterpretasakindofpreparationforfuturelearning:“Supervaluable!Muchmorecompellingtoseerealdata,detail,makesmebelieve,insteadoftuningout'prescribed'info,can'ttrusthowtheyderiveditb/cdon’tknow.Spendanotherdayinourclasstalkingaboutthisresearchplease!”(student22).Elsewhere,studentresponsesalludedtohowsteppingoutsideoftheseprocessesandreflectinguponthemmightaffecttheirconfidenceorfuture
“Spendanotherdayinourclassplease”:Studentinsightsfromaresearch-baseddesignthinkingexercise
13
decisionmaking.Forexample,student3wroteofhavinglearned“howotherpeoplespendtheirtimeindesign.Ididn'thaveanyideahowothersdidit.SeeingwhatpartswerethemostimportantforthequalityscorewilldefinitelyshapethewayIdesigninthefuture”,whilestudent10noted,“Inowrealizethatspendingtimeatthisprocesswillpayoff”.
Figure6–Partialscansofstudent7and22’swrittenannotationsuponthedesigntimelinehandouts
Reviewingstudentresponsesfromthisbroaderperspectivealsohelpedusnoticehowstudentsweremakingmeaningofthetimelinesoutsideoftheprescribedspacesandpromptsthattheyweregiven.Someclearinstancesofthiscamefromthreeofthestudentsannotatingandsketchingdirectlyontheircopiesofthedesigntimelinestodiscoveradditionalinsights.Forinstance,intheimagescansabove(Figure6),wecanseestudent7attemptingtomapdesigntoashapebyconsideringthegeometricgradientorslopeofsenior3’sdesignprocess.Likewise,student22attemptstoprojectcertainpatternswithinthetimelinesontodesignbehaviourssuchaspivoting.Thesemorevisualattemptsatdiscoverycanhelpusappreciatethedifferentmodalitiesandbehavioursstudentswereengagingintoarriveattheirdiscoveries.
5.DiscussionThestudentsinthisstudysucceededindiscoveringsixmeaningfulinsightsaboutthedesignprocess,asdescribedinthefindingssection.Thestudents’abilitytoarriveatsuchinsightsconfirmsthatguideddiscoverywaspossibleandcorroboratespreviouswork(Borgford-Parnell,2010).Asalludedtointheintroduction,webelievethatstudentsdiscoveringandarticulatingtheirinsightswillbehelpfulfortheminfuturedesignactivities.Specifically,beingabletorecognizeeffectivestrategiesmeansthatthereispotentialforthemtomonitorandadjusttheirowndesignactivitywithincreasedconfidence.Forexample,seenfromahighlevelperspective,thestudents’insightsfocusonthesignificanceofspendingtimeandhowthattimeisspent.Implementingthatinsightmaypositionstudentstomonitorhowtheyarespendingtime,recognisewhentheyarenotspendingtimeeffectively(e.g.,spendingagreatamountoftimeinmodelling), andmakeexecutiveleveldecisions
AtmanCynthiaJ.,ArifAhmer,ShroyerKathrynE.,TurnsJenniferA.,Borgford-ParnellJim
14
abouthowtospendtimegoingforward(e.g.,decidingtorevisittheproblemdefinitionorgathermoreinformation).
Thedistributionofstudents’insightswasnotthesameforthetwoprompts.Prompt1guidedstudentstoarticulatetheirobservations,andPrompt2guidedthemtoreflectandarticulatewhatwasmostimportanttothem.Examiningthefindings,wecanseethattherewerethreepatternsofchange:(a)aninsightbeingprevalentforPrompt1andnotprevalentatPrompt2(e.g.,Time),(b)aninsightnotbeingprevalentforPrompt1andbeingprevalentforPrompt2(e.g.,IterationandProblemDefinition),and(c)abalancedpatterninwhichaninsightwasequallyprevalentforbothprompts(e.g.,BreadthandGatheringInformation).Surfacingthesepatternsofchangehelpsusbetterunderstandtheguideddiscoveryprocessovertimeintermsofwhatwaseasyforindividualstudentstodiscoverearlyonversuswhattheycoulddiscoverwithdiscussionswithpeersandtheeducator.Additionally,webetterunderstandwhatdiscoveriesstudentsrecognizeearlyonbutlaterreplacebymoreinterestinginsights,ascomparedtostudentsearlydiscoveriesthatcontinuetoretaintheirimportance.Thiswayofframingdifferenttypesofguidancecanhelpwiththedesignoffutureguideddiscoveryactivities,andalsoraisesquestionsabouthowdifferentpromptsmightfacilitatedifferentdiscoveries.
Asthedescriptionsofthestudentcommentsthatmadeupthecontentofthecodecategoriesindicate,therewasanindicationthatforsomestudents,theirinsightsweresomewhatpersonalinnature(forexample,insightsrelatedtoidentity,toimaginedfuturedesignactivities,andtoasenseofpersonalrelevance).Thisisinterestingbecausesuchpersonalpositivereactionmayincreasethelikelihoodthattheseinsightsarenotephemeral,butrather,becometheprinciplesthatstudentslaterusetogroundtheirowndesignprocesses.Interestingly,thepromptof“themostimportantthingyoulearned”doesnotnecessarilyfacilitatesuchpersonalreactions—learningforstudentscanoftenbeassociatedwiththetypeofknowledgethatistestedbytestsandexplainedintextbooks.Giventhatthepromptdidnotaskstudentstoprovideinsightintotheirpersonalreactions,theobservationthatone-thirdoftheresponsesfeaturedsuchapersonaldimensionmightactuallybeanunderestimationofthepersonalreactionsbeingfeltbythestudentsinthisproject.Thisleadsustowonderwhatwemighthaveobservedifthepromptshadbeendifferent.Forinstancewemighthavesurfacedevenmorepersonalinsightswithpromptssuchas“Whatwouldyoudodifferentlyifyouwereinafuturedesignsituation?”or“Doyouthinkofyourselfasadesigner?”or“Didtoday’sactivitiesinfluenceyourthoughtsaboutyourselfasadesigner?”
6.LimitationsandFutureResearchOnelimitationofthestudyisthatourunderstandingofwhatthestudentsdiscoveredismediatedentirelythroughwhattheywroteonthesheetofpapertheyweregiven.Seeingthatsomestudentsactuallymadeannotationsdirectlyonthetimelinesthemselvessuggeststhatwealsoneedtoconsiderotherartefactsandwaysthatstudentsmighthavebeen
“Spendanotherdayinourclassplease”:Studentinsightsfromaresearch-baseddesignthinkingexercise
15
engagingwiththetimelines.Onepossiblemeansofcapturingsomeoftheseadditionalinteractionswouldbetovideo-recordtheinstructionalevent.Thiswouldmakeitpossibletodiscernanyunscriptedquestionsandinteractionsthattookplacebetweenthestudentsandtheeducator.Additionalattentiontothoseinteractionscouldhelpinscriptingtheguidanceprocesssothatitisdoneinamoreexplainableandrepeatablemanner.Finally,giventhetime-constrainednatureoftheactivity,itwasnotpossibletofollowthestudentsintoanactualdesignactivityandseeiftheirnewinsightshelpedthembesuccessful.Futureresearchcanexplorewhetherthestudentinsightsfromtheshorterexercisedescribedinthispaperwillbehelpfulforstudentsoveralongerterm.
7.ConclusionInthispaper,wedocumentwhatwelearnedaboutusingguideddiscoverytohelpstudentdesignersdevelopinsightsaboutdesignprocessesthatarebasedondataandfindingsfrompriorresearch.Thispapercontributestotheevidence-baseforthespecificguideddiscoveryactivitythatislaidouthere.Ourfindingsleadustobelievethatdesigneducatorswhotrytorepeattheactivitywiththeirstudentsmaydiscoverthatadditionalaffordancesoccurwhenthereissharedvocabularyandsharedreferenceforideassuchasproblemsolvinganddesignprocessovertime.Thisactivityrepresentsoneexampleofcreatinganactivitywherestudentsinteractwithrealdataandhavetheopportunitytodiscoverpatternsthattheymaylaterleverageintheirowndesignwork.Seenasanexample,thiscanhelpothereducatorscreatesimilarguideddiscoveryactivitiesandsuggestfutureresearchdirectionsintotheuseofguideddiscovery.
Acknowledgements:WewouldliketogratefullyacknowledgeTerriLovinsforherassistanceonthispaperandthestudentswhoparticipatedinthisstudy.ThisworkwassupportedbytheNationalScienceFoundation(grantsRED-9358516andROLE-0125547),theBoeingCompany,theCenterforEngineeringLearning&TeachingattheUniversityofWashington,theMitchellT.andLellaBlancheBowieEndowment,andTheLeonaM.andHarryB.HelmsleyCharitableTrust.
8.ReferencesAlfieri,L.,Brooks,P.J.,Aldrich,N.J.,&Tenenbaum,H.R.(2011).Doesdiscovery-basedinstruction
enhancelearning?JournalofEducationalPsychology,103(1),1.Atman,C.J.,Adams,R.S.,Cardella,M.E.,Turns,J.,Mosborg,S.,&Saleem,J.(2007).Engineering
designprocesses:Acomparisonofstudentsandexpertpractitioners.JournalofEngineeringEducation,96(4),359–379.http://doi.org/10.1002/j.2168-9830.2007.tb00945.x
Atman,C.J.,Chimka,J.R.,Bursic,K.M.,&Nachtmann,H.L.(1999).Acomparisonoffreshmanandseniorengineeringdesignprocesses.DesignStudies,20(2),131-152.
Bruner,J.S.(1961).Theactofdiscovery.HarvardEducationalReview31(1):21–32.Borgford-Parnell,J.,Deibel,K.,&Atman,C.J.(2010).Fromengineeringdesignresearchto
engineeringpedagogy�:Bringingresearchresultsdirectlytothestudents.InternationalJournalofEngineeringEducation,26(4),748–759.
AtmanCynthiaJ.,ArifAhmer,ShroyerKathrynE.,TurnsJenniferA.,Borgford-ParnellJim
16
Crismond,D.P.,&Adams,R.S.(2012).Theinformeddesignteachingandlearningmatrix.JournalofEngineeringEducation,101(4),738.
Cohen,J.(1960).Acoefficientofagreementfornominalscales.EducationalandPsychologicalMeasurement,20(1),37-46.
DeanJr,D.,&Kuhn,D.(2007).Directinstructionvs.discovery:Thelongview.ScienceEducation,91(3),384-397.
Dym,C.L.,Agogino,A.M.,Eris,O.,Frey,D.D.,&Leifer,L.J.(2005).Engineeringdesignthinking,teaching,andlearning.JournalofEngineeringEducation,94(1),103-120.
Kirschner,P.A.,Sweller,J.,&Clark,R.E.(2006).Whyminimalguidanceduringinstructiondoesnotwork:Ananalysisofthefailureofconstructivist,discovery,problem-based,experiential,andinquiry-basedteaching.EducationalPsychologist,41(2),75-86.
Mayer,R.E.(2004).Shouldtherebeathree-strikesruleagainstpurediscoverylearning?AmericanPsychologist,59(1),14.
AbouttheAuthors:
CynthiaJ.AtmanisdirectoroftheCenterforEngineeringLearning&TeachingandprofessorofHumanCenteredDesign&Engineeringatthe University ofWashington. Her research focuses on engineeringdesign learning, reflection, and considering context in engineeringdesign.
AhmerArif isaPhDstudentat thedepartmentofHumanCenteredDesign&EngineeringattheUniversityofWashington.Hestudiesthedesignofdigitaltoolstosupportreflectivepractices.
Kathryn E. Shroyer is a PhD student in the department of HumanCenteredDesign&EngineeringattheUniversityofWashington.Herresearchinterestsincludeengineeringeducationanddesignthinking,teaching,andlearning.
Jennifer A. Turns is a professor of Human Centered Design &Engineeringat theUniversityofWashington.Her research interestsinclude engineering education, reflection, helping engineeringeducators, and application of ideas from complexity science to thechallengesofengineeringeducation.
Jim Borgford-Parnell directs the Office for Advancement ofEngineeringTeaching&LearningattheUniversityofWashington.Hetaughtdesign,research,educationtheory,andpedagogycoursesforover 35 years. He currently does research and instructionaldevelopmentinengineeringeducation.