THE JOURNAL OF TEACHER ACTION RESEARCH 14 · Abstract The analysis of three years of reflecting on...

THEJOURNALOFTEACHERACTIONRESEARCH 14

JournalofTeacherActionResearch- Volume3,Issue2,2017,<practicalteacherresearch.com>,ISSN#2332-2233©JTAR.AllRights

USINGTEACHERACTIONRESEARCHINHIGHSCHOOLCHEMISTRYTODEVELOPNOVELASSESSMENTTOOLSChadHusting

SycamoreHighSchool,Cincinnati,OH

JordanHarshman,Ph.D.

UniversityofNebraska-Lincoln

EllenYezierski,Ph.D.

MiamiUniversity

AbstractTheanalysisofthreeyearsofreflectingonteachingpracticesinablogleadateachertorealizethatassessmentplaysacriticalroleindaytodayteachingandcouldbeimprovedinhisclassroom.Theliterature

doesnotprovideverymuchonthe"nutsandbolts"ofusingassessmentdatatoinformteaching;however,

data-driveninquiryprovidesausefulframeworkforaddressingthisproblem.Amethodandtoolwere

developedtofacilitaterapidandmeaningfulstudentdatacollection,scoring,andinstructionaldecision-

makinginahighschoolchemistrysetting.Thetoolcapturesgroupsofstudents’knowledge,understandings,

andmisconceptionsatthebeginningoftheyearinanefficientandeffectivemannerbyleveragingeasily

accessiblesoftware(Excel).Thestudentdataarelinkedtotestedandresearch-drivencurriculummaterials

withastrongbasisinchemistryeducationtheory.Theassessmentmethodsandtool,itsimpetus,design

considerations,functionality,andimplementationaredescribedwithhopeofintroducingteachersto

approachesthatcanimprovehowtheycoordinateassessmentwithcurriculumandinstructiontoimprove

studentlearning.

Keywords:assessment,formativeassessment,data-driveninquiry,teacheractionresearch,collaborative

actionresearch,blogging,qualitativeanalysis,highschool,science,chemistry

Introduction



Foralmostonehundredyears,criticalreflectiononteachinghasbeennotedasoneofthe

mostimportantprocessesforimprovinginstruction(e.g.,Dewey,1933;Yost,Sentler,

Forenza-Bailey,2000).Recently,scholarshaveexaminedweblogsor“blogs”astoolsto

promoteandfacilitateteacherreflection(Ray&Hocutt,2006;Loving,Schroeder,Kang,

Shimek&Herbert,2007;Killeavy&Moloney,2010;Luik,Voltri,Taimalu&Kalk,2011).

Althoughblogshavebeenshowntobegoodforcriticalevents,theymaynotelicitdeep

reflectionwithoutexplicitprompting(Wopereis,Sloep&Poortman,2010).Blogginghasalso

beenusedasaresearchtoolinethnographicfieldwork(Saka,2008),whichprovidesauseful

frameworkfortheanalysisofteacherblogs.Thisstudyincorporatesbloggingandqualitative

analysiswithintheframeworkofteacher(oraction)research,whichhasbeenshowntobea

highlyeffectivemeansofpromotingteachergrowthandhasbeenacommoncomponentof

professionaldevelopment(e.g.,Lytle&Cochran-Smith,1992).

LiteratureReview

InTobiasandBaffert’swork(2009),theyhaverecentlyhighlightedtheneedforelevating

theprofessionalstatusofscienceandmathematicsteachers.Actionresearchmayalsobea

vehicleforthisend,asteacherresearchhasbeenviewedbyBurtonandBartlett(2005)and

ZeichnerandNoffke(2001)asahighlyeffectivemeansofcollaboratingwithteachersto

improvetheirprofessionalismandthestatusoftheteachingprofessioningeneral.Zeichner

andNoffke(2001)alsofoundthatteacherresearchcontributestotheknowledgebase.Such

findingssuggestthatactionresearchcanproducemanyintellectualandsocialbenefitsto

theteacheraswellastheirstudentsandtheacademiccommunity.Thesimilaritybetween

teachingandresearchhasbeencalledoutbyHuberman(1996)andFreeman(1998)andhas

recentlyframedakeyworkbyMeijer,Oolbekkink,Meirink,andLockhorst(2013).Meijeretal.(2013)describeshowthelinkagesbetweenteachingandresearchparallelkeyideasinhighereducationfirstintroducedbyBoyer(1990)whocoined“thescholarshipofteaching

oflearning”thusexpandingthetraditionalideaofresearchasonlythescholarshipof

discoverytofourareas.Coppola,BanaszakHoll,andKarbstein(2007)describehowteaching

andresearchareintegratedandinformed,intentional,impermanent,andinheritable.

However,whentheresearcherisnottheclassroomteacher,muchcanbemissingfromthe

research.Teachershavetheknowledgeofthestudents,classroom,andschoolenvironment

aswellastheneedsofallstakeholdersincludingthemselves(Meijer,Oolbekkink,Meirink,&

Lockhorst,2013).Toimproveteacherlearning,researchquality,scholarlyimpact,

authenticity,andimpactonstudents,collaborativeactionresearch,inparticular,ishighly

valued(Meijer,Oolbekkink,Meirink,&Lockhorst,2013),inwhichteachersworkwith

scienceeducationresearchfaculty.

Thestudyheredetailsacollaborativeactionresearchprojectcarriedoutbyahighschool

chemistryteacherandachemistryeducationresearchfacultymemberandgraduate

student.ThestudywasguidedbyAndersonandHerr’s(1999)fivevaliditytypesfor

practitionerresearch(outcome,democratic,catalytic,dialogic,andprocessvalidity)witha

majoremphasisonoutcomevalidity,“whethertheresearchundertakenleadstooutcomes



forteachersandfortheschool.”Assuch,wewillnotonlypresentthemethodsandfindings

ofthestudyinatraditionalmanner,butwealsorevealthemajorconsequenceofthe

knowledgegeneratedbythiscollaborativeendeavor.Theaimofthislastpieceisnotonlyto

demonstrateoutcomevaliditybutalsoistoprovideararelyseenproductinreportsof

actionresearch.Thedescriptionofhowtheteacherrespondedtotheresearchfindingsand

theproductinspiredbythefindingsarepresentedattheendofthisarticle.

Theimpetusfortheactionresearchprojecttheoreticallyliesinincreasingthequalityand

quantityofteacherreflection(Dewey,1986).Specifically,howemployingdescriptive,

critical,andcomparativereflection(Jay&Johnson,2002),canhelpteachersidentify,

unpack,andovercomeimportantclassroomlearningproblems.Theimpetusfortheproject

morepracticallycomesfromthefirstauthor’sparticipationintheMasterTeacherProgram

sponsoredthestateofOhio.Teacherswereprovidedtimetomeetingroupsthroughoutthe

yeartocreateportfolios.Theportfoliosfocusedonevidencethatexaminedwhatthe

teachersweredoingbefore,during,andafteralessontohelpstudents.Italsofocusedon

evidencethatdemonstratedcommunicationwithstudents,parentsandcommunity

involvement.Thefollowingyear,inattempttocontinuereflectingonteachingand

documentingevidenceforstudentlearning,thefirstauthorbegancreatingblogentries

focusingonthesamethemesfromthemasterteacherprogramthepreviousyear.Eachblog

entryfocusedonexamining,constructing,anddevelopingevidencethatdemonstrated

studentachievementintheclassroom.Anothermajorinfluenceforteacherchangeand

reflectionintheblogwastheauthor’sparticipationintheTargetInquiryProgramatMiami

University,OxfordOhio(TIMU).Thiswasanintensivetwoandahalfyearprogramfunded

bytheNationalScienceFoundation.Itfocusedonchemistryteachersdoingthescientific

processandtranslatingthisintotheirteachingbydevelopingandimplementinginquirylabs

andstudyingtheirimpactsonstudentlearning(http://targetinquirymu.org/).

Methodology

Theresearchquestionthatguidedthisstudywas:Howcanateacherresearcher’s

qualitativeanalysisofblogentriesbeusedtofocusinstructionalimprovementstrategies?

Setting.ThisstudywascarriedoutatasuburbanpublicschoolinsouthwestOhio.Oftheapproximately1800students,73%ofthestudentsareWhite,about8%areBlack,about

10%areAsian/PacificIslander,about3%areHispanicandabout5%aretwoormoreraces.

Theinstructor(firstauthor)hasamaster’sdegreeandisconsidered“HighlyQualified”by

theStateofOhio.Hehas24yearsofteachingexperience,mostlyinchemistry.Overthe

courseofthisstudy(3.5years),theteacherscheduleconsistedoftwoclassesofAccelerated

ChemistryandthreeclassesofAcademicChemistry.AcceleratedChemistrystudentswere

primarilytenthgraderswithafeweleventhgraderswhowereconsideringfuturescience

classesinhighschoolandcollege.TheAcademicChemistryclassesconsistedmainlyof

eleventhgraderswithafewtwelfthgraderswhoarecollegeboundbutnotnecessarily

consideringtakingscienceclassesintheirfuture.



DataCollection.Beginningin2011-12,thefirstauthorbeganwritingaweeklyblogrespondingtothesequestions:

1. WhatamIdoingtohelpkidsachieve?

2. HowdoIknowwhentheyarethere?

3. Whatistheevidence?

Theblogentrieswerepostedonhttp://simpleteach.blogspot.com/andaimedtobemore

thanapersonalreflectionbecausetheyfocusedontheevidenceusedtoreachconclusions

aboutandtheteacher’sroleinstudentlearning.Therewere97entries(averaging329

wordseach)collectedbetweenDecember2012andJune2015.

DataAnalysis.Theblogentrieswereprintedandreadmultipletimes.Withcollaboration

fromafirstyearcollegestudent,entrieswereplacedintocategories.Thefirstauthor

generatedcategorynamesanddescriptionstoserveascodes.Eachentrywascodedsuch

thatithadexactlyonecodeassignedtobemostdescriptive.Approximately10%ofthe

entrieswerecodedbythethirdauthorandfoundthatthedescriptionswerenotdetailed

enoughtoattributeonlycodetoeachblogentry.Assuch,thefirstauthorrevisedthe

descriptionsandselected12randomblogentries.Theseentriesalongwiththecategory

namesanddescriptionsweregiventoanewrater.Tenofthisrater’sassignmentofentries

tocategoriesmatchedwiththefirstauthor’sassignmentsforthissubsetofthedatacorpus.

Thefinalcategorynames,descriptions,andfrequenciesofoccurrenceinthedatasetare

foundinAppendixA.Insubsequentanalyses,wegroupedthecategoriesintothemes.The

emergentthemesdescribingthedataarepresentedintheResults.

ResultsandDiscussion

TheresearchersgroupedthecodesintothemesbasedontheData-DrivenInquiry(DDI)

framework.ThesethemesrepresenttheproductsofthreeofthefourstepsoftheDDI

process:assessment(collectevidence),reflection(makeconclusions),andactivities(take

action).AlthoughgoaldeterminationisacriticalpartoftheDDIprocess,theblogentriesdid

notfocusoncurriculum,whichislikelywhythisthemedidnotemergeinthedataanalysis.

Figure1displaysaconceptmapthatshowstheconnectionsamongtheblogentry

categoriesandquestionsguidingtheblogandhowtheyaresubordinatetomajoremergent

themes.



Figure1:Conceptmapofblogquestions,categoriesfromcoding,andemergentthemes.Amuchmoredetailedconceptmapofblockentriesmaybefoundinthesupplementalinformation.

Afterexamining97blogentriesandthemostfrequentlyoccurringcategories,assessment

emergedasamajortheme.Assuch,thefirstauthorchosethisasafocusforinstructional

improvement.ThischoiceisstronglyalignedwiththeNextGenerationScienceStandards,

whicharebuiltupontheexpectationthatteachersmustrelyonassessmentinformation

thatguidesinstruction(NationalResearchCouncil,2012).Specifically,hewantedtofinda

waytocreateformativeandsummativeassessmentsinthemosteffectiveandefficientway

possible.Thiswasparticularlytimelysinceduringthe2014-15schoolyearstudentswere

subjectedtoasignificantincreaseinstatemandatedassessments.Thechallengeidentified

bythequalitativedataanalysisprocesswastoimproveassessmentwithoutincreasingtime

spentonformalassessmentactivities.Thiswarrantedapplyinganewtheoreticaland

practicallenstothisinitiative,anddatadriveninquiryservedthisneed.

ViewingResultsthroughtheLensofDataDrivenInquiry(DDI).Datafromourownstudents

isoneofthebestsourcesofinformationforusasteacherstomakeinstructionaldecisions.

Thisprocessisfamiliartomostteachers,andinarecentliteraturereviewhasbeen

described(Harshman&Yezierski,inpress).Theprocessgoesbymultiplenamesinthe



literature(U.S.DepartmentofEducation,2008;2011;InstituteofEducationSciences,2009;

CalfeeandMasuda,1997)butwillbereferredtoasDataDrivenInquiry(DDI).Thestepsare:

1. Determinegoals.

2. Collectevidence.

3. Makeconclusions.

4. TakeAction(Harshman&Yezierski,inpress).

Althoughthereseemstobeanabundanceofresearchthatshowsthatteachersshoulduse

datadriveninquiry,thepracticalityofimplementingtheuseofassessmenttoinform

instructiononadaytodaybasishasnotbeenwellstudied(Harshman&Yezierski,2014,

Harshman&Yezierski,inpress).Although20outof97blogentriesdirectlypertainedto

assessment,examinationoftheothercategories(particularlywhatwedidinclass,plansfor

nextyear,andinterestingandcreativeteachingideas)inlightofaDDIframeworkshows

howassessmentwaspervasiveinthedataset.Thisisnotsurprisinguponare-examination

ofthequestionsguidingeveryblogentry:

1. WhatamIdoingtohelpkidsachieve?

2. HowdoIknowwhentheyarethere?

3. Whatistheevidence?

ThesimilaritiesamongtheabovequestionsandthecomponentsofDDIhighlighthowthe

firstauthorhasconvergedonassessmentasameanstoimproveoverallinstruction.With

theDDIprocessinmind,wehavedevelopedatooltoimproveteachingandlearningwithan

assessmentfocus.

UsingResults:DevelopingaTooltoMeetNeedIdentifiedinBlogAnalysis.Thefollowingconsiderations,whichemergedfromtheanalysisoftheblogentries,guidedthe

developmentofatooltoeffectivelyandefficientlycollectandusestudentdata.Thegoal

wastoidentifyandusehighqualityassessmentsdevelopedthroughresearchthatcouldbe

deliveredtostudentsinsuchawaythatscoringwasquickandthecollation/analysisofdata

enabledtheteachertoputresultstouseimmediately.

Assessment:First,theaimwastoselectmultiple-choicechemistryquestionsthatcould

evaluateconceptualknowledge(ahigherlevelthanjustrotememorization).Questionsthat

couldprovideinformationaboutastudent’sdevelopmentallevelwereparticularly

desirable,anditwasessentialthatquestionswerealignedwiththecontentandgoalsofthe

chemistrycurriculumoftheschoolanddistrict.Asmentionedbefore,itwasalsocriticalthat

theassessmentsweredevelopedbasedonsoundeducationaltheoryandevidence.The

threetestsusedweretheLawsonClassroomTestofScientificReasoning(LCTSR)(Lawson,

1978),theChemistry9-12MisconceptionOrientedStandardsBasedResourceforTeachers

(MOSART)fromtheScienceEducationDepartmentoftheHarvard-SmithsonianCenterfor



Astrophysics(Sadler,Coyle,Smith&Miller,2006),andtwo-tieredchemistryquestionsthat

directlyalignedwiththecurriculumtobetaught.

Efficiency:Thenextstepwastodevelopamethodofdelivery.Penandpaper,Scantrons

and“bubblingin”answersistimeconsumingandcumbersome.Afast,effective,

inexpensivemethodthatthatwouldprovidequickandreliablestudentdatawasneeded.

PermissionwasgrantedfromtestauthorstodevelopelectronicversionsoftheLCTSRand

MOSARTtestsaslongastheyweresecured.Allofthequestionsfromthetwotestsandthe

otherquestionsdevelopedwerewrittenintoasingleGoogleForm.Advantagestothis

methodarethelinkcanbeeasilyandsecurelysharedwithstudents,itworksonmultiple

devices,andtheteachercaneasilycollectresponsesintoaGoogleSheet.Another

advantagetotheGoogleFormisthatthereisafreeadd-oncalled“Flubaroo”

(https://chrome.google.com/webstore/detail/flubaroo/mjkbmijfpphoabkogbdmdkolcnaenai

a?hl=en-US).Asstudentssubmittheirdata,theFlubarooadd-ongradesitinstantly.The

gradedsheetbyFlubarooprovidesateacherwithpossiblepoints,averagepoints,andtime

ofsubmission.AnexcerptofstudentresponsesintheGoogleSheet(Figure2)isfollowedby

asampleofthegradedFlubaroosheet(Figure3).

Figure2.RawresponsedatafromtwotestscollectedwithaGoogleFormanddisplayedviaGoogleSheet.



Figure3.StudentresponsesandpercentagecorrectasgradedwithakeybyFluberoointhesameGoogleSheetasFigure2.

TheaforementionedonlinedatacollectionprocessreducestimefillingoutScantrons,

gradingthem,andpreparingtoanalyzethedatabyorganizingitinaspreadsheet.Itis

possibletohavethegradeddatabeforestudentsleavetheclassroom.

Action.Althoughthespeedydatacollectionimprovesefficiencyincollectingthedata,there

neededtobeamethodtosimplifydataanalysis.TwoformattedGoogleSheetswere

developedtoautomatedataanalysis.Rawdataarecut,copiedandpastedinoneofthe

formattedsheetsandthegradedresultsfromFlubaroointheother.Whatmakesthesenew

sheetsdifferentistheconditionalformatting.Figure4showstheRawData(similartothe

informationinFigure2)thathasbeenpastedintothenewformattedsheet.



Figure4.GradedresponsesfromFigure2subjectedtoconditionalformattinginGoogleSheetbasedonthesecriteria:Green=correct;yellow=popularmisconception;white=incorrect.

ThesheetshowninFigure4isconditionallyformattedinawaythatallowstheteacherto

quicklygetasenseofmultipleaspectsofstudentperformancestomaketeachingdecisions

basedonthedata.First,eachquestioncolumnlabelsthetesttype.Thequestionisassigned

tothetypeofassessmentitcamefromsuchas“MOSART,”“LAWSON”or“GenChem.”The

nextcelldownisawordorwordsthatidentifiesthetargetedcontenttopicofthatquestion.

Ifyoulookcarefully,youwillnoticethateachofthesewordsarelinks.Thereisawebsite

(https://sites.google.com/site/simpleinqchem/home)thathasapageforeachtopic.The

pageprovidesinquiryactivitiesthathavebeendevelopedthroughresearchedchemistry

educationprinciplesandteachertestedasdiscussedearlier.Someoftheseactivitiesinclude

TargetInquiryfromGrandValleyStateUniversityandMiamiUniversity(TIMU),Process

OrientedGuidedInquiryLearning(POGIL,2012),ModelingandInquirylabsfromreputable

sources.Thegoalwastoonlyusematerialsthatwerevettedbyeducatorsandresearchers.

AsinFigure4,cellswithstudentanswersautomaticallyfillwithcolorsbasedonparticular

criteria.Ifitiswhite,theansweriswrong.Ifitisyellow,thisanswerhasbeenidentifiedby

theauthorsoftheMOSARTtesttobeapopularmisconception.Iftheanswerisgreen,the

responseiscorrect.

Tocompileindividualstudents’resultstoexaminequestionperformanceacrossanentire

class,aFlubarooformattedsheetwasconstructedusingconditionalformattingtocolor

codelevelsofperformancebyquestion.TheFlubarooformattedgradedsheet(Figure5)has

thequestiontype,content,andlinkatthetopofeachquestioncolumn.Italsohasacolor-

codedpercentagecorrectatthebottom.If75%orgreaterofthestudentsintheclass

correctlyansweredtheitem,thepercentagecellisgreen.If40to74%ofthestudentsscore

correctlyontheitem,thecellwiththepercentageisyellow,andanythingbelow40%isred.



Figure5.Conditionalformattingbyitemtoexaminewhole-classperformanceonMOSARTitems.

Implications

Theassessmenttoolwasimplementedinthefirstauthor’sclassinfall2015.Studentstooktheassessmentatthebeginningoftheyear.Sincestudentshaveexperiencedvisitinga

doctor,thisanalogywasusefulinconveyingthemeaningofa“diagnostic”tool.Thedoctorornursefirsttakestheirtemperature,bloodpressure,height,andweight.These

measurementsarenotnecessarilyseenasatest.Onecannotreally“pass”or“fail”anyofthesemeasurements,buttheyserveasdiagnostictoolstoguidethedoctorinprovidingthe

bestcarepossible.Fortheassessment,ifeveryonedoeswellonthequestionsaboutthe

periodictable,theinstructorshouldrespectthestudentsandspendlesstimeaddressing

whattheyalreadyknow.Settingthepropertoneandculturehelpedstudentstakethe

assessmentseriously.

Oncestudentstooktheassessment,itwasgradedwithFlubarooandpastedintoformatted

googlesheets.TheDDIframeworkguidedhowformattedFlubarooresultswereused.First,

datafromtheLawsontestwereexamined.Heterogeneousstudentgroupswereestablished

fromtheirLawsonscores.Ithasbeenshownthatonceastudentreachesadevelopmental

milestone,thechancesofthemchanginggreatlywithinaschoolyearareslim(Lawson,

1978).However,constantexposuretotheotherlevelsofreasoningmightbehelpful.

Next,theinstructorfocusedondatafromtheMOSARTandtheGenChemquestions.He

spentlimitedclasstimeontopicsinwhich75%ormoreoftheclassalreadydemonstrated

mastery.Moreinstructionaltimewasspentontheyellowquestions,onesforwhich40-74%

ofthestudentsearnedcorrectanswers.Thesecutoffsaresomewhatarbitrary;however,the

greenthreshold(75%)reflectsthepercentageofstudentswhodemonstratecompetency

whenthefirstauthortypicallymovesoninthecurriculum.Foranythinginred,whereless

than40%oftheclassscoredcorrectanswers,theinstructorassumesthatstudentsknow

littletonothingandstartswithbasicideas.Itisencouragedthatshouldanyotherteacher

usethistool,s/heconsiderpercentagesthatbestmeettheneedsofhis/herstudents.

Anexampleistheinstructorresponsetoquestion9onthepreassessment.Itisclearfrom

theinitialrawdatathatthemajorityofstudentsdonotknowbasicinformationabout



isotopes.Onlythirteenpercentofstudentsfromallthreeclassesansweredcorrectly.The

analysisshowsthatabouthalfhadnoideaandhalfhadpopularmisconceptions.This

indicatedtotheinstructorthatthecontent,includingcommonmisconceptions,neededto

beaddressed.Aspartofthelessononisotopes,theinstructorusedaPOGILactivity.

Studentshadtonotonlyanswerthequestionsbuthadverballyrespondtoinstructor

“checkpoints.”

Nexttheformattedrawdata,particularlytheMOSARTquestions,wereexaminedtousethe

incorrectanswerchoicestoguideinstruction.Popularideasalignedwithmisconceptions

(formattedinyellow)shouldbetreateddifferentlythanincorrectones(formattedinwhite).

Asanexample,thereisaquestionthatasksstudentsifthedistanceschangebetweenthe

threeatomsofawatermoleculewhenwatergoesfromasolidtoaliquid.Thecorrect

answeristhatthedistancedoesnotchange.Theanswerthatisapopularmisconceptionis

thatthedistancebetweentheatomsinthemoleculegetslarger.Inthecommentarysection

oftheMOSART,testauthorsnotethattherelationshipbetweenmacroscopicchangesand

microscopicparticlesisevaluatedbythisitem(Sadleretal.,2006).Clickingonthelink“MolecularShapes”atthetopofthecolumnforthisquestionlinkstoasitewithseveral

possibleactivitiestoteachmolecularshapes.Theactionoftheinstructorshouldbetopick

anactivitythatstressestheparticulatenatureofmatter.Notonlycantheinstructorclearly

definegoalsandalignthemwiththecurriculum,assessmentscanbeidentifiedandadded

tothelinks.

Whenstudentsarenotmeetingtheobjectivesoftheunit,theinitialassessmentdatacanbe

usedonastudent-by-studentbasis.Forexample,theLawsontestprovidesinformation

aboutastudent’sscientificreasoningability.Ifastudentisstrugglingwithproportionalreasoninginstoichiometry,theinstructormaychoosetonarrowthegoalandaddresssome

basicskilldevelopmentaroundproportionalreasoning.Otherfeaturesofitemsmayprovide

insights.TheMOSARTtest,alongwithsomeoftheauthor-developedgeneralchemistry

items,addressesspecificlevelsofchemicalknowledge(macroscopic,particulate,or

symbolic).Assuch,theresponsestotheinitialassessmentquestionsmayhelptheteacher

identifyproblematicdomainsforthestudents.Sincesomequestionsstressonelevelmore

thananother,theinstructormaywishtoidentifydifferentactivitiesthataddressthesame

chemistrytopicbutemphasizespecificlevels.

TheDDIprocessrequiresacarefulexaminationofevidenceofstudentknowledge,makinga

reasonabledecisionbasedonthatevidence,andthencarryingoutinstructionthatis

supportedbycurrentchemistryeducationaltheorytobeeffective.Thetoolemploysthese

processesandaimstosynthesizehigh-qualityassessmentsandcurriculummaterialsinto

actionsthatrespondtohigh-qualitystudentdata.Wehaveevidencedtheeffectivenessof

thesenoveltoolsbytrackinghowresponsestoavailablequestionschangefrompre(first

semester)topost(secondsemester)administrations.Incorporatingthefeedbackfromhis

ownblogsandnovelassessmenttools,theteacherobservedimprovementswithallbuttwo

ofhisstudents(Figure6,left).Thesedifferenceswerestatisticallysignificant[t=12.3(44),p<0.001,d=1.8]andthechangeinquartilesareshownbelow(Figure6,right).Theseresults



indicatethatstudentsimprovedtheirscoresdramaticallyonaccountoftheinstruction

received,whichcontainedthenoveltoolsdiscussedhere.

Figure6:Preandpostscoresforeachindividualstudent(left)andaggregatedboxplots(right)demonstratethelargegainsmadebystudentsduringtheyearthetoolwasimplemented.

Conclusion

Today’sclassroomteacherfacesanavalancheofactivitytheminutetheywalkintothe

building.Studentsaskforhelp,dailyemails,unexpectedinterruptions,labsthatneedtobe

prepared,meetingswithparents,phonecalls,endlessgradingofpapers,unfundedstate

andfederalmandatesalladduptothepointwhere,literally,itisnownotuncommonto

hearthewords,“Idon’tevenhavetimetousethebathroom.”Inthechorusofchaos,the

simpleactofdocumentingreflectionforcesateachertoprovideasmallsliverof

informationaboutwhatisreallyimportant.Furthermore,systematicandvalidresearch

methodshelptominethebloginformationtohelpfocustheteacheronwhatreallymatters

whentryingtohelpimprovestudentlearning.

Weexaminedalmost100blogentriesbyasingleeducatorthatalwaysfocusedonthesame

threequestionsthatdealtwithstudentlearningandevidenceforit.Althoughseveral

themesemergedfromanalyzingtheblogentries,assessmentwasacentraltheme.Basedon

thesefindingsandaDDIframework,anoveltoolwascreatedtocollectdatatoinformand

guidedailyinstructioninthemostefficientandeffectivewaypossible.Usingthetoolhelps

andrespectsstudents’priorknowledgeandleveragesreadilyavailabletechnology.The

researchandpracticaloutcomespresentedheredemonstratehowaclassroomteacherand

aresearchercaneffectivelycollaboratetocombineteacher’sknowledgeofthestudents,

classroom,andschoolenvironmentduringprofessionaldevelopmenttostudyandimprove

instructionwhilemaintainingresearchqualityandauthenticity.Furtherresearchwouldbe

greatlybeneficialindetailingtheeffectivenessofthisparticularstrategyinspecifictopics.



Asourscopewastointroducetheideaofdevelopingatoolthatcanbeusedtogenerally

informactivities,studiesthatusethisstrategyandmeasureoutcomesonspecifictopics

wouldyieldvaluableinsightsonthepragmaticchallenges,effectiveness,andtipsonhowto

implementatatopic-specificlevel.Asmentionedpreviously,theseday-to-daydetailsare

scarcelyfoundinrelevantliteraturebutcrucialforsuccessfuladaptationofstrategiessuch

astheoneproposedhere.

Ratherthanendingthediscussionhere,weinvitereaderstousesocialmediatoviewthe

ongoingproject,examineitsprogressinrealtime,andconversewiththefirst

author/practitioner.Thefirstauthor’sprogresscanbetrackedonhisblogat

https://www.chemedx.org/blogs/chad-husting.Additionally,inquiriesarewelcomefrom

educatorswhowishtoadoptthecurricularactivitiesembeddedinthetooland/ordiscuss

employingDDIstrategiestheirclassrooms.

Acknowledgments

WethankClaireHustingandSusanHustingforassistancewithcoding,andMOSARTand

Lawsontestauthorsforgrantingpermissionforuse.Thismaterialisbaseduponwork

supportedbytheNationalScienceFoundationunderGrantNo.DRL-1118749.

AbouttheAuthors

ChadHustingteachesinadiversesuburbanhighschoolofabout1800studentsinasuburbofCincinnati,Ohio.Mr.HustingisamemberoftheAmericanAssociationofChemistry

TeachersandhaswrittenforSolutions.Hehasablogyoucanfollowathttps://www.chemedx.org/blogs/chad-husting.Email:[email protected]

JordanHarshman,Ph.D.,isapostdoctoralresearchassociateintheDepartmentof

ChemistryattheUniversityofNebraska–Lincoln.Dr.Harshmanhasworkedwith

professionaldevelopmentforinstructorsbothinhisgraduateexperiencesatMiami

Universityandhiscurrentpostdoctoralposition.

EllenYezierski,Ph.D.,isaprofessorofchemistryatMiamiUniversityanddirectorofthe

NSF-fundedprofessionaldevelopmentandresearchproject,TargetInquiryatMiami

University.Dr.Yezierski’sresearchgroupinvestigateschemistryinstructioninavarietyof

learningenvironments.Email:[email protected].



References

Anderson,G.L.,&Herr,K.(1999).Thenewparadigmwars:Isthereroomforrigorouspractitionerknowledge

inschoolsanduniversities?.Educationalresearcher,28(5),12-40.

Bartlett,S.,&Burton,D.(2006).Practitionerresearchordescriptionsofclassroompractice?Adiscussionof

teachersinvestigatingtheirclassrooms.EducationalActionResearch,14(3),395-405.

Boyer,E.L.(1990).Scholarshiprevisited.Princeton,NJ:CarnegieFoundationfortheAdvancementofTeaching.

Calfee,R.C.,&Masuda,W.V.(1997).Classroomassessmentasinquiry.InPhye,G.D.Handbookofclassroomassessment.Learning,adjustment,andachievement.SanDiego:AcademicPress.

Coppola,B.P.,BanaszakHoll,M.M.,&Karbstein,K.(2007).Closingthegapbetweeninterdisciplinaryresearch

anddisciplinaryteaching.ACSChemicalBiology,2(8),518-520.

Dewey,J.(1933)HowWeThink:ARestatementoftheRelationsofReflectiveThinkingtotheEducative

Process(2ndRevisedEdition),Boston:D.C.Heath.

Dewey,J.(1986,September).Experienceandeducation.InTheEducationalForum(Vol.50,No.3,pp.241-

252).Taylor&FrancisGroup.

Freeman,D.(1998).Doingteacherresearch:Frominquirytounderstanding.PacificGrove:Heinle&Heinle.

Harshman,J.,&Yezierski,E.(2015).Guidingteachingwithassessments:Highschoolchemistryteachers’useof

data-driveninquiry.Chem.Educ.Res.Pract.16,93-103.

Harshman,J.,&Yezierski,E.(inpress).Assessmentdata-driveninquiry:Areviewofhowtouseassessment

resultstoinformchemistryteaching.ScienceEducator.

Huberman,M.(1996).FocusonResearch:MovingMainstream:TakingaCloserLookatTeacher

Research.LanguageArts,73(2),124-140.

InstituteofEducationSciences,Hamilton,L.,Halverson,R.,Jackson,S.,Manidnach,E.,Supovitz,J.,&Wayman,

J.(2009).Usingstudentachievementdatatosupportinstructionaldecisionmaking(NCEE2009-4067).Washington,DC:NationalCenterforEducationalEvaluationandRegionalAssistance,U.S.

DepartmentofEducation.

Jay,J.K.,&Johnson,K.L.(2002).Capturingcomplexity:Atypologyofreflectivepracticeforteacher

education.TeachingandTeacherEducation,18(1),73-85.

Killeavy,M.,&Moloney,A.(2010).Reflectioninasocialspace:Canbloggingsupportreflectivepracticefor

beginningteachers?TeachingandTeacherEducation,26(4),1070-1076.

Lawson,A.E.(1978).JournalofResearchinScienceTeaching,Vol.15,1,11-24.

Loving,C.C.,Schroeder,C.,Kang,R.,Shimek,C.,&Herbert,B.(2007).Blogs:Enhancinglinksinaprofessional

learningcommunityofscienceandmathematicsteachers.ContemporaryIssuesinTechnologyandTeacherEducation,7(3),178-198.

Luik,P.,Voltri,O.,Taimalu,M.,&Kalk,K.(2011).Ontheuseofstudentteacherblogsduringteaching

practice.Procedia-SocialandBehavioralSciences,11,165-169.



Lytle,S.L.&Cochran-Smith,M.(1992).TeacherResearchasaWayofKnowing.HarvardEducationalReview62

(4),HarvardUniversity.

Meijer,P.C.,Oolbekkink,H.W.,Meirink,J.A.,&Lockhorst,D.(2013).Teacherresearchinsecondary

education:Effectsonteachers’professionalandschooldevelopment,andissuesofquality.InternationalJournalofEducationalResearch,57,39-50.

NationalResearchCouncil(2012).AFrameworkforK-12ScienceEducation:Practices,CrosscuttingConcepts,andCoreIdeas.CommitteeonaConceptualFrameworkforNewK-12ScienceEducationStandards.

BoardonScienceEducation,DivisionofBehavioralandSocialSciencesandEducation.Washington,

DC:TheNationalAcademiesPress.

POGILActivitiesforHighSchoolChemistry(2012).Trout,L.,ed.Batavia,IL:FlinnScientific.

Ray,B.B.,&Hocutt,M.M.(2006).Teacher-created,teacher-centeredweblogs:Perceptionsand

practices.JournalofComputinginTeacherEducation,23(1),11-18.

Sadler,P.M.,Coyle,H.P.,Cook-Smith,N.,&Miller,J.L.(2006).MOSART:Misconceptions-orientedstandards-

basedassessmentresourcesforteachers.Cambridge,MA:HarvardCollege.Retrievedfrom

https://www.cfa.harvard.edu/smgphp/mosart/.

Saka,E.(2008,May).Bloggingasaresearchtoolforethnographicfieldwork.InEASAMediaAnthropologyNetworke-seminar.

Tobias,S.,&Baffert,A.(2010).ScienceTeachingasaProfession:WhyitIsn't,HowitCouldBe.Arlington,VA:NSTAPress.

U.S.DepartmentofEducation,Means,B.,Chen,E.,DeBarger,A.,&Padilla,C.(2011).Teachers’abilitytousedatatoinforminstruction:Challengesandsupports.OfficeofPlanning,EvaluationandPolicyDevelopment.Washington,DC.

U.S.DepartmentofEducation,Gallagher,L.,Means,B.,&Padilla,C.(2008).Teachers’useofstudentdatasystemstoimproveinstruction,2005to2007.OfficeofPlanning,Evaluation,andPolicyDevelopment,

PolicyandProgramStudiesServices.Washington,DC.

Wopereis,I.G.,Sloep,P.B.,&Poortman,S.H.(2010).Weblogsasinstrumentsforreflectiononactionin

teachereducation.InteractiveLearningEnvironments,18(3),245-261.

Yost,D.S.,Sentner,S.M.,&Forlenza-Bailey,A.(2000).Anexaminationoftheconstructofcriticalreflection:

Implicationsforteachereducationprogramminginthe21stcentury.JournalofTeacherEducation,51(1),39-39.

Zeichner,K.M.,&Noffke,S.E.(2001).Practitionerresearch.HandbookofResearchonTeaching,4,298-330.



AppendixA:FinalCategoryNames,Descriptions,andFrequenciesoftheOccurrenceintheDataSet

Name Description Frequency

A

Formative

assessment

with

instructions

and

descriptions

Thiscategoryattemptstodescribesometypeofformative

assessmentandthenprovidesinstructionsonhowtocarrythis

outintheclassroom.Oftenthisisdoneatthebeginningofa

unitorassignmentandusuallynotforagrade.Itisthenusedto

informmyinstruction.

18

B

Englishasa

second

language

ESLteachingtechniquescanhelpallstudents.Thesetechniques

useothermethodsbesidesofinstructionandassessment

besideswords.Itishelpfulwithstudentswhoeitherstruggle

withEnglishorhavebeenraisedspeakingaforeignlanguage.

2

C

Whatwedid

inclass

Theseareactuallabsandactivitieswearedoingoraregoingto

trytodoinclass.Mostofthesecenteraroundinquiry.Atheme

withtheselabsisthatmanyareperformanceassessment.

Studentsultimatelyshouldbeabletopredictanoutcomeor

sometypeofendmeasurement.Itisnotsomethingtheycould

lookuponGoogle.

30

D

Plansfornext

year

Afterreflectingandhavingasetofexperiences,theseareplans

and“bigideas”thatIhopetoguidemyinstructioninthefuture

currentyearorfollowingyear.Ihavehadasetofexperiences

andamtryingtolookatguidingprinciplestoguidemyfuture

teaching.

7

E

Reflectionson

outside

influences

Theseentriesarenotaboutclassroomincidences.Theseare

abouteventsthathavehappenedoutsidetheclassroom(classes

Iamtaking,booksIhaveread,talksIhaveheard).Furthermore,

Ihavereflectedontheseeventsandthiscouldinfluencemy

teaching.

30

F

Interesting

andcreative

teaching

ideas:

TheseareideasthatIhavegottenfromoutsidesources.Ihope

touseorhaveusedthatseemlikestudent-centeredcreative

ideas.Theyfocusonhandsonmanipulativesthatstudentscan

doaslabs,projectsorinsomecases,manipulativesthatIcan

useasatypeofassessment.

8

G

Post

assessment

Thisisanassessmentthatisattheendofanactivityandis

similarto,butnotasdetailed,asaformalsummative

assessment.Itisfast,simpleandattheendofanactivity.

2

THE JOURNAL OF TEACHER ACTION RESEARCH 14 · Abstract The analysis of three years of reflecting on...

Documents

Transcript of THE JOURNAL OF TEACHER ACTION RESEARCH 14 · Abstract The analysis of three years of reflecting on...