Effects of E textbook Instructor Learner

RUNNINGHEAD:LEARNINGEFFECTSOFE‐TEXTBOOKANNOTATIONS 1

EffectsofE‐textbookInstructorAnnotationsonLearnerPerformance

AlanR.Dennisa

SerdarAbacib,*

AnastasiaS.Morronec

JoshuaPlaskoffd

KellyO.McNamarab

ConflictofInterest:Theauthorsdeclarethattheyhavenoconflictofinterest.

*Correspondingauthor.E‐mail:[email protected]:+1‐812‐856‐2888Address:2709E.10thStreet,BloomingtonIN47408,USA

aKellySchoolofBusiness,IndianaUniversity,Bloomington,UnitedStates

bUniversityInformationTechnologyServices,IndianaUniversity,Bloomington,United

States

cSchoolofEducation,IndianaUniversity,Indianapolis,UnitedStates

dHighPointGlobal,LLC,Indianapolis,UnitedStates

_________________________________________________________________________________ This is the author's manuscript of the article published in final edited form as:Dennis, A. R., Abaci, S., Morrone, A. S., Plaskoff, J., & McNamara, K. O. (2016). Effects of e-textbook instructor annotations on learner performance. Journal of Computing in Higher Education, 28(2), 221–235. https://doi.org/10.1007/s12528-016-9109-x

https://doi.org/10.1007/s12528-016-9109-x


AlanDennisisProfessorandJohnT.ChambersChairofInternetSystemsintheKelley

SchoolofBusinessatIndianaUniversity.HeisalsotheCo‐FounderofCourseload,Inc.,

whichdevelopedthesoftwareusedinthisstudy.Hecanbecontactedat

[email protected].

SerdarAbaciispostdoctoralfellowatLearningTechnologiesdivisionofUniversity

InformationTechnologyServicesatIndianaUniversity.Hisresearchinterestsinclude

learningtechnologiesinhighereducation,feedback,andonlinelearning.Hecanbe

[email protected].

AnastasiaMorroneisAssociateProfessorofEducationalPsychology,AssociateVice

PresidentforLearningTechnologies,andDeanofITfortheIUPUIcampus.Herresearch

interestscenteraroundinnovativetechnologiesandlearningenvironmentsthatenable

[email protected].

JoshPlaskoffisTQCDirectorofTrainingforHighPointGlobal.Previouslyhewasa

LecturerinManagementandFellowoftheRandallL.TobiasCenterforLeadership

ExcellenceattheKelleySchoolofBusinessatIndianaUniversity.Hecanbecontactedat

[email protected].

KellyMcNamaraisanInstructionalandPedagogicalSpecialistforUniversityInformation

TechnologyServicesatIndianaUniversity.Shecanbecontactedatkmac@indiana.edu.

Acknowledgements:Apreviousversionofthispaperwaspresentedatthe48thAnnual

HawaiiInternationalConferenceonSystemSciences,2015.WewouldliketothankDavid

A.GoodrumandSarahEngelforreviewingandprovidingtheirvaluablecommentsin

preparationofthismanuscript.


EffectsofE‐textbookInstructorAnnotationsonLearnerPerformance

Abstract

Withadditionalfeaturesandincreasingcostadvantages,e‐textbooksarebecomingaviable

alternativetopapertextbooks.Oneimportantfeatureofferedbyenhancede‐textbooks(e‐

textbookswithinteractivefunctionality)istheabilityforinstructorstoannotatepassages

withadditionalinsights.Thispaperdescribesapilotstudythatexaminestheeffectsof

instructore‐textbookannotationsonstudentlearningasmeasuredbymultiple‐choiceand

open‐endedtestitems.Fifty‐twocollegestudentsinabusinesscoursewererandomly

assignedeitherapaperoranelectronicversionofatextbookchapter.Resultsshowthat

thee‐textbookgroupoutperformedthepapertextbookgroupontheopen‐endedtestitem,

whilebothgroupsperformedequallyonthemultiple‐choicesubjecttest.Theseresults

suggestthattheinstructionalaffordancesthataninteractivee‐textbookprovidesmaylead

tohigher‐levellearning.

Keywords:e‐textbook;printtextbook;learning;readingcomprehension;instructor

annotations;experimentalstudy


Introduction

Textbooksarestillamongthemostfrequentlyusedteachingresourcesincollege

education,andcarefulreadingaccountsfor85%ofsuccessfullearning(Richardson,

Morgan,&Fleener,2012;Simpson&Nist,2000).Everyacademicyear,studentspaymore

than$1,000onaveragetobuytextbooks,whichaddstotheever‐increasingcostofa

collegeeducation(Bidwell,2014).Giventhatsomestudentsrefrainfrombuyingtextbooks

duetotheirhighprices,textbooksmayactuallyimpedestudentlearningratherthan

encouragingit(Senack&TheStudentPIRGs,2014).Asaremedy,educationalpublishers

arenowofferingelectronicversionsoftextbooksatalowercost.Theresearchone‐

textbookadoptionhighlightscostbenefitsasasignificantfactorcontributingtocollege

students’preferenceforelectronicversions;however,e‐textbooksmayoffermorethan

costsavings.Theymayfacilitateimprovedteachingandlearningviafeaturesand

interactivitythatarenotreadilyavailableinpapertextbooks.

Thefeaturesofferedbynewere‐readingsoftwareenablestudentsandinstructorsto

interactwithane‐textbookindifferentways(Walling,2014).Forexample,aninstructor

cansharenotesandhighlightsonareadingassignmentdirectlywithinthee‐textbook.One

keyquestioniswhetherthesenewfeaturesaffectstudentlearning.

Themajorityofresearchonthelearningeffectsofe‐textbookshasthusfar

comparedthedifferenceinmediums(i.e.,textbooksdeliveredonpapervs.onscreen)

ratherthanexamininghowthefeaturesofferedbye‐textbooksenabledifferent

instructionalmethods(e.g.,Daniel&Woody,2013;Siebenbruner,2011;Terpend,Gattiker,

&Lowe,2014).Consequently,previousresearchhasnotreportedanysignificantdifference


betweenpaperandelectronictextbooksbecausethemediaalone(paperversusscreen)

havefewdistinctionswhenitcomestolearning(Ackerman&Goldsmith,2011;Connell,

Bayliss,&Farmer,2012).Incontrast,thisstudyinvestigatesthelearningeffectsofone

featureofenhancede‐textbooks,lookingattheimpactofinstructorannotationsonstudent

performance.

Changingnatureofe‐textbooks

Despitebeingaroundforthepastfourdecades,e‐bookshavelackedapersistent

definitionduetotheever‐changingtechnologiesandfeaturesthroughwhichtheyare

deliveredandread(Vassiliou&Rowley,2008).Therefore,VassiliouandRowleysuggesta

two‐partdefinitionthatcapturesboththecharacteristicsofe‐booksanddynamically

changingtechnologiesunderlyingthem.Basedonthisdefinition,ane‐book:

1)Isadigitalobjectwithtextualand/orothercontent,whicharisesasaresultofintegratingthefamiliarconceptofabookwithfeaturesthatcanbeprovidedinanelectronicenvironment,and2)Typically[has]in‐usefeaturessuchassearchandcrossreferencefunctions,hypertextlinks,bookmarks,annotations,highlights,andmultimediaobjectsandinteractivetools(p.363)

Comparedtoe‐books,e‐textbooksarestillrelativelynewinthemarketplace—but

theyaresubjecttothesamedefinition.Oneoftheevolvingfeaturesisinteractivity,which

Walling(2014)describesonacontinuum.Onthelowendoftheinteractivitycontinuum,e‐

textbooksaredigitalimagesofthepaperbook.Movingfromthelowendtothehighend,e‐

booksande‐textbooksalikeoffervarietyofinteractivefunctionssuchashighlighting,

bookmarking,andannotations.Onthehighend,enhancede‐textbookreadingsoftware

offerssocialnetworkingcapabilitiessuchasallowingtheinstructorandstudentstoshare


notes,highlights,andannotations.Thesecapabilitiesdeservespecialattentionbecause

theyhavethepotentialtoconvertatextbookfromaself‐studytoolintoatechnology‐

supportedcollaborativelearningtool,inwhichnotesanddiscussionsareanchoreddirectly

totextbookcontent.Andbecauseofthesecapabilities,e‐textbooksarebecomingmore

prevalentinhighereducationasinstitutionsareincreasinglyadoptingOERandCBE

contentdeliveryplatformsthatincorporatethesetools.Inthenextsection,wesummarize

theresearchrelatedtoe‐textbookswithannotationfeatures.

Readingandlearningfrome‐textbooks

Enhancede‐textbooksaredeliveredthroughanelectronicmedium—smartphones,

e‐readers,tablets,orcomputers—soresearchone‐textbooksdrawsfromthreedifferent

butrelatedknowledgebases:comprehensionfromscreenreading,mediacomparison

studies,andannotationsontextbooks.Belowwesummarizeeachoftheseareasinorderto

establishthetheoreticalframeworkforthecurrentstudy.

Comprehensionfromscreenreading

Withtheadventofdigitalmediaandmoreinformationavailabledigitally,peopleare

increasinglyspendingmoretimereadingfromdigitaldisplaysthanonpaper.Thisdigital

environmentischangingreadingpracticesandbehaviors.Forexample,Liu(2005)found

thatmostscreen‐readingtimeisspentonone‐time,selective,andnon‐linearreading;

browsingandscanning;andkeywordspotting.Thesamestudyalsoreportsthatpeople

stillpreferpaperforin‐depthreading,whichusuallyinvolvesannotatingandhighlighting.

Previousresearchoninformationrecallwhenreadingfromscreencomparedto

papermediahasmixedresults.However,mostofthisresearchusesasmallamountoftext,


whichisnottypicaloftextbooksforstudyingcoursecontent.Textbookscontainnarrative

andexpositorytext,fromwhichstudentsareexpectedtodrawconclusionsbasedon

carefulreading(Margolin,Driscoll,Toland,&Kegler,2013).Recentstudiesexaminingthe

screenreadingofnarrativeandexpositorytextshowthatcollegelevelreaders

comprehendthesamelevelofinformationfromscreenreadingastheydofrompaper

reading(Ackerman&Goldsmith,2011;Connell,Bayliss,&Farmer,2012;Eden&Eshet‐

Alkalai,2013;Green,Perera,Dance,&Myers,2010;IndianaStateUniversity,2013;

Margolinetal.,2013;Niccoli,2015).

Learningfrome‐textbooksasinstructionalmedia

Ase‐textbooksofferanalternativemediumtotextbooks,itisimportanttosituate

theresearchonlearningfrome‐textbooksintheframeworkofmediacomparisonstudies.

Intheseminaldebateregardingtheinfluenceofmediaonlearning,Clark(1983)positsthat

themediumandtheinstructionalmethodaretwodistinctentities.Basedonhisreviewof

theresearch,Clarkconcludesthatmedianeverinfluenceslearningandanysignificant

differenceinlearningshouldbeattributedtoinstructionalmethod.Heviewsmediajustas

avehicletotransferinformationtolearners,whoheviewsaspassivereceiversof

information.Incontrast,Kozma(1991)describestherelationshipbetweenmediaand

methodasreciprocal.Hearguesthat“thecapabilities[attributes]ofaparticularmedium,

inconjunctionwithmethodsthattakeadvantageofthesecapabilities,interactwithand

influencethewayslearnersrepresentandprocessinformationandmayresultinmoreor

differentlearningwhenonemediumiscomparedtoanotherforcertainlearnersandtasks”

(Kozma,1991,p.179).Kozma(1994)alsodiffersfromClarkinthatheviewslearningasan

activeandconstructiveprocess.


AshighlightedbyKozma(1994)andlaterreviewersofthisdebate,theresearchon

learningfrommediashouldfocusonhowmediacanfacilitatelearningwhenusedin

conjunctionwithaparticularinstructionalmethod(Hastings&Tracey,2004;Morrison,

1994;Nathan&Robinson,2001).Inthelast30years,wehaveseendramaticchangesin

computerandinternettechnologies,offeringcapabilitiesthatcansupportcertain

instructionalmethodsinwaysthatweredifficultoreffectivelyimpossiblewithout

technology.Therefore,researchintoearliertechnologiesdoesnothelpusassesshownew

technologysuchase‐textbookscanfacilitatelearningandreadingcomprehension(Kamil&

Chou,2009).Aswedescribeabove,e‐textbookshaveevolvedfromdigitalimagesof

textbookpagestoincludeinteractivitybetweenstudents,content,andtheinstructor.Asa

result,thereisaneedforresearchonhowe‐textbookfeaturescansupportlearningand

readingcomprehension.

Althoughtherehavebeenanumberofstudiesfocusingonthelearningeffectsofe‐

textbooks,mostofthesestudiescompareoverallimpactofe‐textbookstopapertextbooks

(Daniel&Woody,2013;Giacominietal.,2013;Ji,Michaels,&Waterman,2014;Rockinson‐

Szapkiw,Courduff,Carter,&Bennett,2013;Shepperd,Grace,&Koch,2008;Siebenbruner,

2011;Terpend,Gattiker,&Lowe,2014).Thee‐textbooksinthesestudiesaremere

replacementsofthepapertextbooks;thus,noneofthesestudiesemployaninstructional

methodthatcancapitalizeonfeaturesofthee‐textbooks.Consequently,andnot

surprisingly,theydonotreportanysignificantdifferenceinlearningduetothemedia,

regardlessofwhetherlearningismeasuredwithtestscores(Daniel&Woody,2013;

Siebenbruner,2011),coursegrades(Shepperdetal.,2008;Terpendetal.,2014),orself‐


reportedlearninggains(Giacominietal.,2013;Jietal.,2014;Rockinson‐Szapkiwetal.,

2013).

Onlyafewstudiesconsidertheuniquefeaturesofenhancede‐textbooks—suchas

highlightingorannotating—whenexamininglearningdifferences.Annotationsare

comments,notes,orexternalremarksattachedtoadocument(Yang,Zhang,Su,&Tsai,

2011).Addingannotationstoatextbookcanmakeasignificantcontributiontoboth

cognitiveandmetacognitiveaspectsoflearning.Underliningandhighlightingcanalso

assistinrecall(Flavell,1981;Lee,Lim,&Grabowski,2010).Weisberg(2011)examined

studentattitudesandbehaviorstowardse‐textbooksovertwoyearsusingfivedifferente‐

readerdevicesandonepapertextbookgroup.Therewerenolearningdifferencesbetween

differente‐readerdevicesorbetweenthesedevicesandthepapertextbookgroup.

Althoughthesee‐readersincludedhighlightingandnote‐takingfeatures,thestudydidnot

directlyexaminetheuseorimpactofhighlightingandnotetaking.

Inanexperimentalstudy,Taylor(2011)treatedstudentengagementwitha

textbook(cleanvs.annotatedreading)asoneofthemaineffectsforbothpaperande‐

textbookgroups.Thecleanengagementgroupinthisstudywastoldnottomakeany

annotations,whereastheannotatedengagementgroupwasencouragedtoannotatethe

assignedtextbookwhilereading.Thestudyreportsnosignificantdifferencesdueto

engagement.However,theextentofannotationisunclear,andthestudydoesnotnote

whetherthepaperande‐textbookgroupswerecombinedfortheanalysis.

Instructorannotationsonstudents'e‐textbooks

AccordingtoYangetal.(2011),annotationsonadocumentmayhelpthereaderin

fourimportantareas:attention,organization,indexing,anddiscussion.Inparticular,


instructorannotationsavailabledirectlywithine‐textbooksmayimprovestudentlearning

intwoways.First,annotationenablesinstructorstogobeyondtextbookcontentwith

additionalonlinemultimediaresources.Basedonextensiveresearchintomultimedia

learning,Mayer(2009)concludesthatstudentslearnbetterfromacombinationofwords

andvisualssuchaspictures,animations,andvideosthanwordsalone—andspokenwords

coupledwithvisualshavedeeperimpactonlearningthanwrittenwordscombinedwith

visuals.Richardsonetal.(2012)alsosuggestthattextbookinformationshouldbe

complementedwithotherresources.Second,instructorannotationsone‐textbookscreate

newopportunitiesforinstructor‐student‐contentinteraction.Withannotations,instructors

canofferstudentsinsightsintotheirinterpretationofandperspectivesonthetextbook

content,therebymakingiteasierforstudentstounderstandandinterpretthematerial.

Thissupportcanresultinlearninggains(Gee&Rakow,1990).Specifically,theseinsights

providescaffoldingbeyondformalclasstime,guidingstudenteffortstograspthemost

criticalcontent.Annotationsalsoenableinstructorstomodelexpertpracticesbymaking

thosepracticesvisibletostudents(Linn&Eylon,2011).

Whilelittleresearchhasdelvedintotheeffectsofinstructorannotationsonlearning

orstudentengagementwithe‐textbooks,Dennis(2011)foundthat84%ofcollegestudents

reportedthatinstructorannotationsonane‐textbookwereusefulintheirlearning.

Similarly,studentsreportedthattheyreadmoreandlearnedmorewithane‐textbook

comparedtoapapertextbookwhentheirinstructorsharedannotationsandhighlightson

thee‐textbook(Abaci,Morrone,andDennis,2015).Unfortunately,neitherofthesestudies

examinesthecontentofinstructorannotations,whichmaybeakeyfactorinstudent

assessmentsofhelpfulness.Abaci,Morrone,andDennis(2015)alsoconductedinterviews


withinstructors,whousede‐textbookannotationsforvaryingreasons:toprovide

additionalrelevantcontent,elaborateonaparticulartopic,clarifyterminology,orprovide

theirperspectiveonthetextbookcontent.Theirresponsessuggestthatstudentswere

moreengagedwithtextbookcontentwheninstructorsguidedstudentreadingthrough

sharedannotationsandhighlights.Nevertheless,theactualimpactofinstructor

annotationsonstudentlearningorperformanceremainstobeinvestigatedsincethesetwo

studieswerebasedonself‐reportedsurveyandinterviewdata.

Inanattempttoanswertheimpactofinstructornotesonstudentlearning,Murray

andPérez(2011)conductedaquasi‐experimentalstudy.Theyassignedonesectionofan

onlineITliteracycourseane‐textbook,whichincludedsupplementarylinksandshort

videoclips.Thecontrolgroupreceivedtheprintedtextbookwithoutthelinksorshort

videoclips.Thestudydidnotfindanydifferencebetweenthetwosectionsregarding

studentlearning,whichwasmeasuredbytwoopen‐bookmultiple‐choiceexams.The

annotationsinthestudy—hyperlinksandvideoclips—offeredadditionalinformation,but

didnotguidestudentsonhowtousetheinformation.Therefore,moreresearchisneeded

toexaminewhetherothertypesofinstructorannotationsimprovestudentlearning.

Purposeofthestudy

Ase‐textbookadoptioninhighereducationgrowsandenhancedfeaturesenable

moreinteractionbetweenthestudents,thecontent,andtheinstructor,e‐textbookshave

thepotentialtooffermorethanjuststaticimagesofpagesonascreen.Today’senhancede‐

textbooksoffersupportingfeaturesforinstructionalstrategiesthatwouldnototherwisebe

possible.However,currentapproachestostudyingthelearningeffectsofe‐textbooksthat


solelycomparepaperanddigitalmediamayoverlookthepotentialeffectsofthesefeatures

onlearning.Therefore,theresearchdesignshouldtakeintoaccountthespecificfeatures

andsupportinginstructionalstrategies.Ashighlightedintheprevioussection,theresearch

intotheeffectsofe‐textbookswithsharedannotationshasbeenverylimited.Inanattempt

tofillthevoid,thepurposeofthisstudyistoexaminetheeffectsofinstructorannotations

onstudentlearningasmeasuredbyaknowledgetestcomposedofmultiple‐choiceand

open‐endedquestions.Specifically,weaimtoanswertworesearchquestions:

1. Dostudentsusingane‐textbookwithinstructorannotationsperformdifferently

onmultiple‐choiceexamquestionsthanstudentsreadingfromapaper

textbook?

2. Dostudentsusingane‐textbookwithinstructorannotationsperformdifferently

onanopen‐endedexamquestionthanstudentsreadingfromapapertextbook?

Methods

Thispilotstudyusesasinglefactorexperimentaldesignwithtwolevels:apaper

textbook(control)groupandane‐textbook(experimental)group(Creswell,2009).The

studycollecteddatafromcollegestudentsonapaperquizandanalyzeditinSPSSv.21.

Participants

Participantsinthisstudyweresecond‐yearundergraduatestudentsfromthe

businessschoolofalargeMidwestpublicuniversity.Fifty‐twostudents(22malesand30

females)takingacomputertechnologycoursewererandomlyassignedtooneofthetwo

studyconditions:e‐textbookgroup(n=27),orpaper‐textbookgroup(n=25).


Materials

ThefirstchapterofaDataCommunicationsandNetworkingtextbookwasused.The

paperversionwasapaperphotocopyofthechapteranddidnotcontainanyinstructor

annotations.Thee‐textbookversionwaspresentedonacomputerscreenusingane‐reader

softwarethatincludesinteractivefeaturessuchashighlighting,bookmarkingand

annotating,andallowsbothstudentsandinstructortosharetheirhighlightsand

annotationswithothers.Inthepresentstudy,studentscouldcreatebutnotshare

annotationswithotherparticipants.

Thetextbookcontentforthisstudyisusedinathird‐yearcomputernetworking

course.Wesharedthe13annotationsthecourseinstructorusedintheprevioussemester

withtheparticipantsinthee‐textbookgroup.Someoftheannotations(n=6)provide

guidanceregardingwheretofocus,suchas“Readthissectionlightly”or“Thissectionis

veryimportant.Makesureyouknowitwell!”Otherannotations(n=7)areintendedfor

elaborationsuchas“Standardsarekeytonetworking.Withoutstandards,wecouldn'thave

theInternetbecauseeverycompany'snetworkcouldoperatealittledifferently.”Oneofthe

elaboratedannotationsalsoincludesalinktoafour–minutevideoanimationexplaining

howthefivelayersoftheInternetNetworkingModelworktogethertomovemessages

acrosstheInternet.Figure1presentsanexampleoftheannotatede‐textbookpage,where

theinstructorhighlightedthesectiontitleandleftanotetostudentsexplainingthe

importanceofthissectionandprovidingasupplementaryvideolink.

[INSERTFIGURE1]


Measures

Learningperformancewasmeasuredwitha25‐itemquiz,administered

immediatelyafterthereadingassignment.Alloftheitemsinthequizcamefromthe

Instructor’sManualthataccompaniesthetextbook.Oftheseitems,24weremultiple‐choice

questionsworthonepointeach.Thus,themaximumscoreformultiple‐choicequestions

was24points.

Therewasoneopen‐endedquestion,designedtotestdeeperunderstandingand

worththreepoints.Itaskedstudentsto“useadiagramtoshowhowthefivelayersinthe

Internetmodelworktogethertosendamessagefromaclienttoaserver.”Theanswerkey

allowedforpartialpoints;therefore,studentscouldearnbetween0.0and3.0pointswith

0.5increments.Inordertoestablishinter‐rateragreement,twooftheresearchersgraded

thisquestionbasedonaninstructor‐generatedanswerkey.

Procedure

Uponarrivalattheresearchlab,participantsgaveconsentandwererandomly

assignedtoeitherthepaperore‐textbookgroup.Averagereadingtimeforthechapterwas

previouslyfoundtobeapproximately35minutes.Therefore,participantsinbothgroups

weregiven35minutestoreadtheassignedchaptertoprepareforaquiz.Thee‐textbook

groupreceivedoneminuteofadditionalpreparatoryinstructiononusingthesoftware's

basicnavigationtoreadthechapterandreviewannotations.Afterreadingthechapter,all

participantshad15minutestocompleteapaperquiz.Duringthequiz,neithergrouphad

accesstothetextbook.


Results

Intheirpreliminaryanalysis,theresearcherscheckedtheinternalreliabilityofthe

multiple‐choicequestionsaswellastheinter‐raterreliabilityoftheopen‐endedquestion

betweenthetwograders.Cronbach’salphaforthemultiple‐choiceitemswas.724,

demonstratinggoodinternalconsistency(Nunnally,1978).Intraclasscorrelationbetween

thetwogradersoftheopen‐endedquestionwas.964(p<.001),indicatingexcellent

agreementbetweenthegraders.

Theresearchersconductedindependentsamplest‐teststoanswertheresearch

questions.Theytestedthedependentvariablesmultiple‐choicetotalscoreandopen‐ended

itemscoreagainsttheassumptionsoft‐test:normalityandequalityofvariances.

Descriptiveanalysisindicatedthatmultiple‐choicetotalscoreswerenormallydistributed.

Furthermore,Levene’stestofequalityofvariancewasnotsignificant(p=.471);thus,

equalityofvariancewasassumed.Foropen‐endeditemscores,descriptiveanalysis

indicatednon‐normaldistribution.However,thesamplesizeforeachgroupwaslarge

enoughtoproceedwiththeanalysis.Levene’stestforequalityofvariancewassignificant

(p<.001);therefore,equalityofvariancewasnotassumed,andcorrespondingtestresults

werereportedfromtheSPSSoutput.

AsTable1shows,theindependentsamplest‐test(df=50)didnotyieldsignificant

resultsforthemultiple‐choicetotalscore.Thatis,studentsreadingthee‐textbookchapter

didnotperformdifferentlyonthemultiple‐choiceexamthanstudentsreadingfromthe

paperchapter.Incontrast,thee‐textbookgroupperformedbetterthanthepapertextbook

groupontheopen‐endeditem.Thisdifferencehadalargeeffectsize(d=.89)accordingto

Cohen’sdefinition(1988).


Table1

Means(StandardDeviations‐SD),t‐values,andsignificancevaluesforcomparisongroups

MeasurePaper(n=25)

M(SD)E‐textbook(n=27)

M(SD)Teststatistic

(t)Significance

(p)

Multiple‐choicetest 12.41(3.68) 14.04(4.26) 1.48 .145

Open‐endeditem .56(.67) 1.40(1.15) 3.19 .003*

*Significantatα=.05Discussion

Researchintoe‐textbooks'impactonlearningisstillemergingbecausenewfeatures

continuetoaffordinstructorsandlearnersnewwaystointeractwiththetextandwith

eachother.Thispilotstudyisamongthefirsttoexaminetheeffectsofinstructor‐annotated

e‐textbookcontentonstudentlearning.Ourresultsindicatethatinstructorannotationsdid

notaffectstudentlearningasmeasuredbymultiple‐choiceknowledgequestions.This

outcomeisconsistentwiththepriorliteratureonthelearningeffectsofe‐textbooks.

However,aswehighlightedinourreviewoftheexistingresearch,onlyonestudytoour

knowledgewasconductedatthefeaturelevel(Murray&Pérez,2011).Thepresentstudyis

differentinthatitincludesinstructorannotationsintendedtofocusstudents’attentionand

elaborateonkeypointsinthetext.

Thisstudyalsomeasuredstudentlearningonanopen‐endedfiguralresponseitem,

whichaimedtoevaluatedeeperunderstandingoftheInternetmessagetransferprocess.

Open‐endedessayquestionsingeneralofferaneffectivewaytoassesshigher‐learning

objectives(Reiner,Bothell,Sudweeks,&Wood,2002).Figuralresponsesaremoredifficult

butslightlymorediscriminatingandreliablethantheirmultiple‐choicecounterparts

(Martinez,1991).Oneoftheannotationsinthee‐textbookcontainedalinktoavideo

demonstrationthatwasspecificallyassociatedwiththeopen‐endedquestion.Whilethee‐


readersoftwaredoesnottrackwhetherstudentswatchedthisvideo,studentsinthee‐

textbookgroupperformedsignificantlybetterthanthepapertextbookgroupontheopen‐

endeditem.

Althoughthisstudyfocusedonlyoninstructorannotations,thee‐readersoftware

usedinthisstudyaffordsstudentstheopportunitytomaketheirownannotationsand

sharethesewithothersintheclass.Makingannotationsisconsideredausefulstudy

practice(Leeetal.,2010;Marshall,1997).Annotatingbehaviorisalsoassociatedwithin‐

depthreading,whichisimportantforcollegecourses.Bycreatingannotations,students

canrevisetheirpriorknowledgeastheyencounternewideasandinformationandasthey

testtheircurrentschema(Bransford,Brown,&Cocking,1999;Sawyer,2006).More

complexannotationstrategies—suchassummarizing,paraphrasing,findingexamples,and

generatingquestions—contributetometacognitivemonitoringandfeedback.These

strategiescanenhancelearners’self‐regulation,recall,andcomprehension(Flavell,1981;

Leeetal.,2010;Leutner,Leopold,&DenElzen‐Rump,2007),andimprovelearningwhen

usedinreview(Kiewraetal.,1991).

Despitethebenefitsofannotatingforreadingcomprehension,peopleannotateless

onelectronicdocumentsthanpaperdocumentsbecauseitfeelslessnaturalormore

distractingtoreading(Liu,2005).Incontrast,Abacietal.(2015)foundthatstudents

annotatemoreonane‐textbookthanapapertextbookwhentheirinstructoralso

annotatesinthee‐textbook.Thus,instructorannotationsmayencouragestudentsto

annotatemore.AsDobler(2015)claims,students'skillwithenhancedfeaturesofe‐

textbookscannotbeassumedbasedonappearancesofbeinggoodatusingdigitaldevices


andsocialnetworkingtools.Ultimately,moreresearchisneededtostudytheimpactof

instructorannotationsonstudents’annotatingbehavior.

Finally,sharingannotationsine‐textbookscanenablecollaborativelearningas

studentsengagewithcontent.Studentsmayfeelmoreconceptualcontrolwhentheyshare

annotationswitheachother(Greeno,2006).Comparedtotraditionalforumdiscussionsin

aLearningManagementSystem(LMS),vanderPol,Admiraal,andSimons(2006)found

anchoreddiscussion,anothertypeofsharedannotation,moreefficientand“to‐the‐point.”

Otherresearchalsoindicatesthatcollaborativeannotationispositivelyassociatedwith

learningandreadingcomprehension(Chen&Chen,2014;Nokelainen,Miettinen,Kurhila,

Floréen,&Tirri,2005).Researchontheeffectsofsharedstudentannotationoninteraction

andlearningisjustemerging.InGiacominietal.(2013),collegestudentsreportedno

changetotheirinteractionwiththeinstructororwithotherstudentsdespitethe

collaborativeannotationfeaturesofe‐textbooksusedinthestudy.However,studentsrated

collaborativefeatureswiththelowestpriorityamongotherfeaturesofe‐textbooks.More

recently,Hwangandcolleaguesfoundthateffectivein‐classtextannotationbysix‐grade

studentsimprovelearningachievement(Hwang,Liu,Chen,Huang,&Li,2015).Inaddition,

high‐achievinglearnerscreatemorein‐classtextannotationsandmoreafter‐classvoice

andtextannotationsthanlow‐achievingstudents.Additionalresearchwithstudentsusing

collaborativeannotationisneededtounderstandtheeffectsofsharedstudentannotations

atthecollegelevel.


Limitations

Althoughexperimentaldesignisapowerfulresearchmethodtoconfidently

attributeoutcomestotreatmentfactorsratherthanextraneousfactors,itisoftendifficult

toestablishtrueexperimentsinnaturaleducationalsettingsoverthecourseofasemester.

Inthispilotstudy,wewereabletoassignstudentstoexperimentalandcontrolgroups.

Nevertheless,ourrelativelysmallsamplesizemayhaveinfluencedourresults.Alarger

samplesizewouldhaveallowedustocapturesignificantdifferencesevenwithsmaller

effectsizes.Inaddition,thee‐readersoftware,atthetimeofthisstudy,wasnotableto

capturewhetherstudentsreadtheinstructornotesorwatchedtheexternalvideolink.

Havingtheabilitytotracestudentbehaviorwithannotations,wecouldhaveclaimedmore

confidentlythattwogroupsequallyperformedonmultiple‐choiceevenwhene‐textbook

groupattendedtoinstructornotes.Similarly,wecouldhaveidentifiedwhetherthevideo

linkcausedthesignificantdifferenceinopen‐endeditemperformance.Therefore,future

replicationsofthisstudyshouldconsider(a)alargersamplesize,(b)tracingstudents’

behaviorswithinstructornotesandexternallinks,and(c)measuringthedifferencesover

thecourseofasemester.Fromanexternalvaliditystandpoint,ourresultscanonlybe

generalizedtostudentsintechnologycoursesinbusinessschool.Futurereplications

shouldextendthisstudytootherdisciplines.

IdeasforFutureResearchonE‐TextbookFeatures

Webelievethatthereismerittodiscussingresearchideasone‐textbookfeatures

beyondthosethataddressthelimitationsofthisstudybecauseresearchinthisareais

relativelynew.Thescopeofthisstudywaslimitedtotheexistenceof(orlackthereof)


instructorannotations.Aswenotedearlier,contentoftheinstructornotesandhowan

instructorintegratestheseannotationsintohis/herteachingcouldbeanimportant

determinantofstudentengagementwithannotations.Futurestudiesshouldinvestigate

howstudentoutcomesvarybasedonthetype(e.g.,studyguide,elaboration,andlinksto

multimediaresources)anduseofinstructorannotations.

Today’sonlinee‐readersoftwarecancaptureallinstructorandstudentbehavior

withe‐textbooks,whichcanlendtodata‐rich,unobtrusiveresearchonreadingwithe‐

textbooks.Itisencouragingtoseethatresearchusinge‐textbookusagedatahasalready

started.JuncoandClem(2015),examininge‐textbookusagemetricssuchasnumberof

pagesread,numberofmark‐ups(i.e.,highlights,bookmarks,andnotes)andtimespent

reading,foundthatamountoftimespentonreadingisastrongerpredictorofcourse

outcomesthanpreviousacademicachievement.Similarly,VanHorne,Russell,andSchuh

(2016)studiedtheadoptionofmark‐uptoolsbystudentsandreportedthatstudentsare

stillintheearlyadoptionphasewithe‐textbookmarkuptools;highlightingistheonlytool

usedbymorethanhalfofthestudentsinthestudy.Moreinterestingly,theyfoundthatthe

interactionbetweenbookmarkusageandamountofreadingwaspositivelycorrelatedwith

coursegrades.Theauthorsalsosuggestthatstudentsmayneedmorescaffoldingbytheir

instructorsfortheadoptionofinteractivee‐textbooktools.Therefore,futurestudies

shouldexaminestudentusageofe‐textbookfeaturesinconjunctionwithinstructorusage

forapossibleinteractionbetweenthetwo.Furthermore,ase‐textbookreadersallowfor

sharingofthemark‐upsamongstudents,studentinteractionswithe‐textbookscanbe

studiedfromacollaborativelearningtheoryperspective.


Allinall,studiescomparinge‐textbookstopapertextbooks(includingthecurrent

study)haveshownthatlearningisnothinderedbye‐textbooksevenwhene‐textbooks

havelimitedinteractivefeaturesorfeaturesareusedonalimitedbasis.Futureresearch

shouldfocusonhowe‐textbookmarkuptoolsareusedbothbyinstructorandstudents

ratherthancomparingreadingbetweenpaperandelectronicmediums.

Conclusions

Alongwithcostsavings,enhancede‐textbooksofferfeaturesthatenablethe

instructorandstudentstointeractwitheachotherthroughnotesshareddirectlywithin

thetextbook.Mostoftheexistingresearchonlearningwithe‐textbookshascomparedonly

thedifferencebetweenmediums,whileoverlookingfeaturesthatcansupportinstructional

methodsnotpossiblewithpapertextbooks.AsJabr(2013)posits,perhapsweshouldnot

trytomimicreadingonpaperwhiledigitaltechnologiescanturnscreen‐basedreadinginto

anentirelydifferentexperienceutilizinginteractivefeatures.Thefindingsfromthisstudy

suggestthatinstructorannotationsone‐textbooks—guidingandelaborativenotesaswell

aslinkstoadditionalresources—mayimprovehigher‐levellearning.Althoughthisstudy

onlyexaminedinstructorannotations,theaffordancesthatinteractivee‐textbooksprovide

cancreateacollaborativeenvironmentconducivetolearning.

Acknowledgments

Anearlierversionofthispaperwaspresentedatthe48thHawaiiInternationalConference

onSystemSciences(HICSS).


References

Abaci,S.,Morrone,A.,&Dennis,A.(2015).Instructorengagementwithe‐texts.EducauseReview,50(1).Retrievedfromhttp://www.educause.edu/ero/article/instructor‐engagement‐e‐texts

Ackerman,R.,&Goldsmith,M.(2011).Metacognitiveregulationoftextlearning:Onscreenversusonpaper.JournalofExperimentalPsychology:Applied,17(1),18‐32.doi:10.1037/a0022086

Bidwell,A.(2014).Report:HighTextbookPricesHaveCollegeStudentsStruggling.U.S.News&WorldReport.Retrievedfromhttp://www.usnews.com/news/articles/2014/01/28/report‐high‐textbook‐prices‐have‐college‐students‐struggling

Bransford,J.,Brown,A.L.,&Cocking,R.R.(Eds.).(1999).Howpeoplelearn:Brain,mind,experience,andschool.Washington,D.C.:NationalAcademyPress.

Chen,C.‐M.,&Chen,F.‐Y.(2014).Enhancingdigitalreadingperformancewithacollaborativereadingannotationsystem.Computers&Education,77(0),67‐81.doi:10.1016/j.compedu.2014.04.010

Clark,R.E.(1983).Reconsideringresearchonlearningfrommedia.Reviewofeducationalresearch,53(4),445‐459.doi:10.2307/1170217

Cohen,J.(1988).Statisticalpoweranalysisforthebehavioralsciences(2nded.).Hillsdale,NewJersey:LawrenceErlbaumAssociates.

Connell,C.,Bayliss,L.,&Farmer,W.(2012).EffectsofeBookreadersandtabletcomputersonreadingcomprehension.InternationalJournalofInstructionalMedia,39(2),131‐140.

Creswell,J.(2009).Researchdesign:Qualitative,quantitative,andmixedmethodsapproaches(3rded.).ThousandsOaks,CA:SAGEPublications.

Daniel,D.B.,&Woody,W.D.(2013).E‐textbooksatwhatcost?Performanceanduseofelectronicv.printtexts.Computers&Education,62(0),18‐23.doi:http://dx.doi.org/10.1016/j.compedu.2012.10.016


Dennis,A.R.(2011).E‐textbooksatIndianaUniversity:Asummaryoftwoyearsofresearch.fromhttp://etexts.iu.edu/files/eText%20Pilot%20Data%202010‐2011.pdf

Dobler,E.(2015).e‐Textbooks:Apersonalizedlearningexperienceoradigitaldistraction?JournalofAdolescent&AdultLiteracy,58(6),482‐491.doi:10.1002/jaal.391

Eden,S.,&Eshet‐Alkalai,Y.(2013).Theeffectofformatonperformance:Editingtextinprintversusdigitalformats.BritishJournalofEducationalTechnology,44(5),846‐856.doi:10.1111/j.1467‐8535.2012.01332.x

Flavell,J.H.(1981).Cognitivemonitoring.InW.P.Dickson(Ed.),Children'soralcommunicationskill(pp.35‐60).NewYork:AcademicPress.

Gee,T.C.,&Rakow,S.J.(1990).Guidingreadingcomprehension:TechniquesEnglishteachersvalue.TheClearingHouse:AJournalofEducationalStrategies,Issues,andIdeas,63(8),341‐344.doi:10.1080/00098655.1990.10114123

Giacomini,C.,Wallis,P.,Lyle,H.,Haaland,W.,Davis,K.,&Comden,D.(August,2013).ExploringetextbooksattheUniversityofWashington:Whatwelearnedandwhatisnext.Retrievedfromhttps://www.washington.edu/itconnect/wp‐content/uploads/2013/10/UWeTextCampusReport.pdf

Green,T.D.,Perera,R.A.,Dance,L.A.,&Myers,E.A.(2010).ImpactofPresentationModeonRecallofWrittenTextandNumericalInformation:HardCopyVersusElectronic.NorthAmericanJournalofPsychology,12(2),233‐242.

Greeno,J.G.(2006).Learninginactivity.InR.K.Sawyer(Ed.),TheCambridgehandbookofthelearningsciences(pp.79‐96).NewYork:CambridgeUniversityPress.

Hastings,N.,&Tracey,M.(2004).Doesmediaaffectlearning:wherearewenow?TechTrends,49(2),28‐30.doi:10.1007/BF02773968

Hwang,W.‐Y.,Liu,Y.‐F.,Chen,H.‐R.,Huang,J.‐W.,&Li,J.‐Y.(2015).Roleofparentsandannotationsharinginchildren’slearningbehaviorandachievementusinge‐readers.EducationalTechnology&Society,18(1),292–307.


IndianaStateUniversity.(2013,May24).Studentsperformwellregardlessofreadingprintordigitalbooks.ScienceDaily.RetrievedDecember14,2015fromwww.sciencedaily.com/releases/2013/05/130524160710.htm

Jabr,F.(2013,April11).Thereadingbraininthedigitalage:Thescienceofpaperversusscreens.ScientificAmerican.RetrievedDecember15,2015fromhttp://www.scientificamerican.com/article/reading‐paper‐screens/.

Ji,S.W.,Michaels,S.,&Waterman,D.(2014).Printvs.electronicreadingsincollegecourses:Cost‐efficiencyandperceivedlearning.TheInternetandHigherEducation,21(0),17‐24.doi:10.1016/j.iheduc.2013.10.004

Junco,R.,&Clem,C.(2015).Predictingcourseoutcomeswithdigitaltextbookusagedata.TheInternetandHigherEducation,27,54‐63.doi:10.1016/j.iheduc.2015.06.001

Kamil,M.L.,&Chou,H.K.(2009).Comprehensionandcomputertechnology:Pastresults,currentknowledge,andfuturepromises.InS.E.Israel&G.G.Duffy(Eds.),HandbookofResearchonReadingComprehension(pp.289‐304).NewYork:Routledge.

Kiewra,K.A.,DuBois,N.F.,Christian,D.,McShane,A.,Meyerhoffer,M.,&Roskelley,D.(1991).Note‐takingfunctionsandtechniques.Journalofeducationalpsychology,83(2),240‐245.doi:10.1037/0022‐0663.83.2.240

Kozma,R.B.(1991).LearningwithMedia.Reviewofeducationalresearch,61(2),179‐211.doi:10.2307/1170534

Kozma,R.B.(1994).Willmediainfluencelearning?Reframingthedebate.EducationalTechnologyResearchandDevelopment,42(2),7‐19.doi:10.2307/30218683

Lee,H.,Lim,K.,&Grabowski,B.(2010).Improvingself‐regulation,learningstrategyuse,andachievementwithmetacognitivefeedback.EducationalTechnologyResearch&Development,58(6),629‐648.doi:10.1007/s11423‐010‐9153‐6

Leutner,D.,Leopold,C.,&DenElzen‐Rump,V.(2007).Self‐regulatedlearningwithatext‐highlightingstrategy:Atrainingexperiment.ZeitschriftfürPsychologie/JournalofPsychology,215(3),174‐182.


Linn,M.C.,&Eylon,B.(2011).Sciencelearningandinstruction:Takingadvantageoftechnologytopromoteknowledgeintegration.NewYork:Routledge.

Liu,Z.(2005).Readingbehaviorinthedigitalenvironment.JournalofDocumentation,61(6),700‐712.doi:10.1108/00220410510632040

Margolin,S.J.,Driscoll,C.,Toland,M.J.,&Kegler,J.L.(2013).E‐readers,ComputerScreens,orPaper:DoesReadingComprehensionChangeAcrossMediaPlatforms?AppliedCognitivePsychology,27(4),512‐519.doi:10.1002/acp.2930

Marshall,C.C.(1997).Annotation:Frompaperbookstothedigitallibrary.Paperpresentedatthe2ndACMInternationalConferenceonDigitalLibraries,Philadelphia,PA.

Martinez,M.E.(1991).AComparisonofMultiple‐ChoiceandConstructedFiguralResponseItems.JournalofEducationalMeasurement,28(2),131‐145.doi:10.2307/1434795

Mayer,R.E.(2009).Multimedialearning(2nded.).NewYork:CambridgeUniversityPress.

Morrison,G.R.(1994).TheMediaEffectsQuestion:"Unresolvable"orAskingtheRightQuestion.EducationalTechnologyResearchandDevelopment,42(2),41‐44.doi:10.2307/30218686

Murray,M.C.,&Pérez,J.(2011).E‐textbooksarecoming:Areweready?IssuesinInformingScienceandInformationTechnology,8,49‐60.

Nathan,M.,&Robinson,C.(2001).ConsiderationsofLearningandLearningResearch:Revisitingthe“MediaEffects”Debate.JournalofInteractiveLearningResearch,12(1),69‐88.

Niccoli,A.(2015).Paperortablet?Readingrecallandcomprehension.EducauseReview,50(5).Retrievedfromhttp://er.educause.edu/articles/2015/9/paper‐or‐tablet‐reading‐recall‐and‐comprehension.

Nokelainen,P.,Miettinen,M.,Kurhila,J.,Floréen,P.,&Tirri,H.(2005).Ashareddocument‐basedannotationtooltosupportlearner‐centredcollaborativelearning.BritishJournalofEducationalTechnology,36(5),757‐770.doi:10.1111/j.1467‐8535.2005.00474.x


Nunnally,J.C.(1978).Psychometrictheory(2nded.).NewYork,NY:McGraw‐Hill.

Reiner,C.M.,Bothell,T.W.,Sudweeks,R.R.,&Wood,B.(2002).Preparingeffectiveessayquestions:Aself‐directedworkbookforeducators:NewForumsPress.

Richardson,J.S.,Morgan,R.F.,&Fleener,C.E.(2012).Readingtolearninthecontentareas(8thed.).Belmont,CA:Wadsworth.

Rockinson‐Szapkiw,A.J.,Courduff,J.,Carter,K.,&Bennett,D.(2013).Electronicversustraditionalprinttextbooks:Acomparisonstudyontheinfluenceofuniversitystudents'learning.Computers&Education,63(0),259‐266.doi:10.1016/j.compedu.2012.11.022

Sawyer,R.K.(2006).Thenewscienceoflearning.InR.K.Sawyer(Ed.),TheCambridgehandbookofthelearningsciences(pp.1‐16).NewYork:CambridgeUniversityPress.

Senack,E.,&TheStudentPIRGs.(2014).Fixingthebrokentextbookmarket:Howstudentsrespondtohightextbookcostsanddemandalternatives.Retrievedfromhttp://uspirg.org/reports/usp/fixing‐broken‐textbook‐market

Shepperd,J.A.,Grace,J.L.,&Koch,E.J.(2008).Evaluatingtheelectronictextbook:Isittimetodispensewiththepapertext?TeachingofPsychology,35(1),2‐5.doi:10.1080/00986280701818532

Siebenbruner,J.(2011).Electronicversustraditionaltextbooks:Acomparisonofcollegetextbookformats.JournalonExcellenceinCollegeTeaching,22(3),75‐92.

Simpson,M.L.,&Nist,S.L.(2000).AnupdateonStrategicLearning:It'sMorethanTextbookReadingStrategies.JournalofAdolescent&AdultLiteracy,43(6),528‐541.doi:10.2307/40016831

Taylor,A.K.(2011).Studentslearnequallywellfromdigitalasfrompaperboundtexts.TeachingofPsychology,38(4),278‐281.doi:10.1177/0098628311421330

Terpend,R.,Gattiker,T.F.,&Lowe,S.E.(2014).ElectronicTextbooks:AntecedentsofStudents’AdoptionandLearningOutcomes.DecisionSciencesJournalofInnovativeEducation,12(2),149‐173.doi:10.1111/dsji.12031


vanderPol,J.,Admiraal,W.,&Simons,P.R.J.(2006).Theaffordanceofanchoreddiscussionforthecollaborativeprocessingofacademictexts.InternationalJournalofComputer‐SupportedCollaborativeLearning,1(3),339‐357.doi:10.1007/s11412‐006‐9657‐6

VanHorne,S.,Russell,J.‐e.,&Schuh,K.L.(2016).Theadoptionofmark‐uptoolsinaninteractivee‐textbookreader.EducationalTechnologyResearchandDevelopment,1‐27.doi:10.1007/s11423‐016‐9425‐x

Vassiliou,M.,&Rowley,J.(2008).Progressingthedefinitionof“e‐book”.LibraryHiTech,26(3),355‐368.doi:10.1108/07378830810903292

Walling,D.R.(2014).Designinglearningfortabletclassrooms:Innovationsininstruction.Switzerland:SpringerInternationalPublishing.

Weisberg,M.(2011).Studentattitudesandbehaviorstowardsdigitaltextbooks.PublishingResearchQuarterly,27(2),188‐196.doi:10.1007/s12109‐011‐9217‐4

Yang,S.J.H.,Zhang,J.,Su,A.Y.S.,&Tsai,J.J.P.(2011).AcollaborativemultimediaannotationtoolforenhancingknowledgesharinginCSCL.InteractiveLearningEnvironments,19(1),45‐62.doi:10.1080/10494820.2011.528881


Figure 1 Sample of an e-textbook page with instructor annotation

Effects of E textbook Instructor Learner

Documents

Transcript of Effects of E textbook Instructor Learner