Orchestrating learning: survey

6/8/2010 1

CHAIR FOR EMPIRICAL EDUCATION AND EDUCATIONAL PSYCHOLOGY

„Orchestrating Learning“ from an Educational Psychology Perspective

-Conceptual, theoretical, and

empirical considerations

Ingo KollarLudwig-Maximilians-University Munich, Germany

06/08/2010Stellar Summer School on Technology-Enhanced Learning

6/8/2010 2


Early promises and hopes of (educationaland psychological) research on TEL:

• Computers enable individuals and groups to become self-regulated learners

• Computers can make teachers superfluous

• Distance education will replace schools and universities

6/8/2010 3


But…

• Learners‘ problems with self-regulated learning seem to be amplified when learning with computers („lost in hyperspace“; Dillon & Gabbard, 1998)

• Teachers are still there

• Schools and universities are still there

• (However, distance learning is also there, I admit…)

6/8/2010 4


Viewing the role of computer technologiesfrom an educational psychology perspective

• „Media will never influence learning“ (Clark, 1994)

• Computer technologies should…– …be appropriately embedded into existing learning situations,– …help to appropriately structure learning activities,– …leave the role of the orchestrator to the teacher and support her

in this job.

• No primacy of computer technologies, but instead of scientific knowledge about learning and instruction!

6/8/2010 5


So what is it that needs to be„orchestrated“?

• Learning processes occuring on different socialplanes

• Scaffolds aiming at structuring/supportinglearning processes on different social planes

• Technology-supported and face-to-face learningprocesses

• Self-regulated and externally regulated learningprocesses

(see Dillenbourg, Järvelä & Fischer, 2009)

6/8/2010 6


Promises of computer technologiesfor „orchestrating learning“

• Computer technologies may…– …help distribute learning activities over the different

social planes (Dillenbourg & Jermann, 2007),– …help scaffold learning activities on single social

planes („synergistic scaffolding“; Tabak, 2004),– …support the transition from more externally

regulated to more self-regulated forms of learning(e.g., through adaptivity and fading; Kollar & Fischer, 2006).

6/8/2010 7


Indi

vidu

alSm

all g

roup

Plen

ary

Activities

6/8/2010 8


Indi

vidu

alSm

all g

roup

Plen

ary

Scaffolds

Teachermodelling

Givingexplanations

Collaborationscripts

Shared ext. representations

Worked out examples

Reflectionprompts

6/8/2010 9


So how should activities and scaffolds beorchestrated over the social planes?

• Suggestions can be derived from different instructional approaches, e.g.:– Reciprocal Teaching (Palincsar & Brown,

1984)

– Problem-based Learning (Hmelo, 2004)

– Learning by Design (Kolodner, 2007)

– ArgueGraph (Dillenbourg & Jermann, 2007)

6/8/2010 10


So how should activities and scaffolds beorchestrated over the social planes?

• Suggestions can be derived from different instructional approaches, e.g.:– Reciprocal Teaching (Palincsar & Brown,

1984)

– Problem-based Learning (Hmelo, 2004)

– Learning by Design (Kolodner, 2007)

– ArgueGraph (Dillenbourg & Jermann, 2007)

6/8/2010 11


Reciprocal Teaching(Palincsar & Brown, 1984)

6/8/2010 12


Reciprocal Teaching

• RT as one of the most prominent approaches to structuring collaboration

• Originally developed as instructional interventionfor primary school students‘ acquisition of readingcompetence

• Meanwhile: Adaptation of RT in many contextsand age groups

6/8/2010 13


Reciprocal Teaching

• Procedure:1. Teacher models four text comprehension strategies

(Questioning, Summarizing, Clarifying, Predicting) in front of whole class

2. Individual reading phase3. Formation of small groups (4 learners):

– Students take over and switch teacher role among them, helping each other in the correct application of thestrategies

– Teacher gradually withdraws her support as students gainmore competence („fading“)

6/8/2010 14


Indi

vidu

alSm

all g

roup

Plen

ary

6/8/2010 15


Reciprocal Teaching

• Overwhelming evidence for the effectiveness of Reciprocal Teaching compared to traditional classroom learning and other programs(Rosenshine & Meister, 1994)

• Strong effects in particular for low-achievers

• Positive effects on all grade levels from grade 3 – adult education

6/8/2010 16


ArgueGraph(Dillenbourg & Jermann, 2007)

6/8/2010 17


ArgueGraph

• Procedure:1. Individuals answer pre-defined questionnaire on a

controversial topic2. Individual answers are presented and discussed in the plenary3. Heterogeneous dyads based on the individual answers are

formed4. Dyads answer questionnaire again and must agree on their

answers5. Dyadic answers are presented and discussed and opinion

changes from individual to dyadic phase are made visible6. Individuals summarize discussion outcomes on each single

question

6/8/2010 18


Indi

vidu

alSm

all g

roup

Plen

ary

Lernen ermöglicht dem Menschen, sich andynamische Umwelten anzupassen, darinsinnvoll zu handeln und seine kulturellenErrungenschaften an nachfolgendeGenerationen weiterzugeben. …

6/8/2010 19


ArgueGraph

• Instructional goals:– (a) activation of learners‘ prior knowledge/opinions on a

controversial debate– (b) production of socio-cognitive conflicts as a motor for

collaborative learning in small groups and on the plenary level– (c) realization of a „macro script“ that distributes learning

activities over the three social planes of the classroom

• Empirical evidence for positive effects of ArgueGraphcompared to other instructional approaches is scarce

6/8/2010 20


In-between summary• Instructional psychology offers approaches that

orchestrate activities and scaffolds over the different social planes of the classroom – with and without TEL!

• Empirical evidence indicates positive effects of distributing activities and scaffolds over the social planes compared to other forms of classroom instruction

• But: little research that systematically disentagles theeffects of learning activities and scaffolds located at different social planes and their combination

6/8/2010 21


Research project „IKS“(Internet-Kompetenz an Schulen)

6/8/2010 22


Project information

• Located at LMU Munich• Funded by Deutsche Forschungsgemeinschaft (DFG)• Project investigators: Christof Wecker, Frank Fischer,

Ingo Kollar• Main goal: Investigating effects of (computer-

supported) small group collaboration scripts and (non-computer-supported) classroom scripts on acquisitionof Internet search competence in a 9th grade inquirycurriculum

6/8/2010 23


Theoretical background• Internet search competence as educational goal that is

an essential part of scientific literacy• Scientific literacy as ability to participate in science-

related societal debates (e.g., Laugksch, 2000)• Internet search competence comprises the ability to…

– …find relevant, scientifically valid and credible information thatmatches a pre-defined evidence scheme

– …form well-grounded arguments based on this evidence– …use these arguments in science-related debates.

• However, students‘ Internet search competence often issuboptimal (e.g., Ikpeze & Boyd, 2007; Luconi & Tabatabai, 1999)

6/8/2010 24


Theoretical background

• Two promising ways to support students‘acquisition of Internet search competence:

– Small group collaboration scripts that structurelearning activities on a small group level

– Classroom scripts that distribute learning activitiesover the different social planes

6/8/2010 25


Theoretical background• Small group collaboration scripts…

– …specify and sequence learning activities and rolesamong the members of a small group (Kollar et al., 2006),

– …may effectively support learning processes (e.g., Stegmann, Weinberger, & Fischer, 2007,

– …typically help learners acquire domain-generalskills (e.g., argumentation competence; Kollar, Fischer & Slotta, 2007),

– …have mostly been researched under lab conditionsemploying short learning periods.

6/8/2010 26


Theoretical background

• An example of a small group collaboration script(O‘Donnell & Dansereau, 1992):

1. Two learners individually read a text paragraph2. Learners put text aside3. Learner A („recaller“) is supposed to recall all text information4. Learner B („listener“) is supposed to listen to A‘s summary and

to identify omissions and misconceptions5. Both learners jointly discuss how to make the text more

memorable6. Both learners read the next paragraph and then switch roles

…

6/8/2010 27


Theoretical background• Classroom scripts as instructional interventions that

specify, sequence and distribute learning activities overthe different social planes of the classroom devicesfor „orchestrating learning“

• Distributing learning activities over different socialplanes as part of a number of instructional approaches(Reciprocal Teaching, PBL…)

• Empirical evidence: distribution of learning activitiesover the social planes of the classroom may lead to more positive individual learning outcomes than otherkinds of classroom instruction (Rosenshine & Meister, 1994)

6/8/2010 28


Research questions1. Can small group collaboration scripts be

designed to support high school students‘acquisition of online search competence?

2. Do positive effects on domain-generalcompetences (e.g., Internet searchcompetence) also occur under field conditionsemploying longer learning phases?

3. Can the effects of small-group collaborationscripts be amplified by a classroom script thatdistributes learning activities over the plenaryand the small group level?

6/8/2010 29


Method• Participants: 174 9th grade students from five urban high schools• Design:

N = 51 students(2 classes)


With



WithoutSmall groupcollaboration

script

Alterations btw. small group and

plenary level

Small grouplevel only

Type of classroom script

6/8/2010 30


Method

• 4.5 weeks curriculum on Genetic Engineering• Each student was equipped with a laptop computer• 3 cycles on different aspects of the topic:

– Economical aspects– Ecological aspects– Health-related aspects

• Each cycle consisted of 3 steps each:– Gathering domain-specific information on genetics and genetic

engineering (Online project library)– Collaborative online search– Plenary discussion

6/8/2010 31


Posttest 18

2-3 weeks later:

Online search7

Plenary discussion

Posttest 29

Gathering background information (online project library)6

Plenary discussion

Online search5

Gathering background information (online project library)

Plenary discussion4

Online search3

Gathering background information (online project library)2

Introduction and gathering background information (online project library)1

Pretest0

PhaseLesson

6/8/2010 32


Method• Both independent variables were realized

during the three online search phases

• Learners engaged in online searchprocesses basically as dyads

• Laptop computers were connected by a software called S-COL (Wecker et al., accepted)

6/8/2010 33


Method• Small group script split online search up into 5 phases:

1. Sketching initial argument and required information2. Entering search terms3. Evaluating hit list4. Finding information on selected web page5. Revision of initial argument

• Script prompts to distribute complementary activities and roles among the members of a dyad:– A: Executive role (e.g., „Make a suggestion what link to click on“)– B: Metacognitive role (e.g., „Listen to your partner‘s suggestion

and estimate whether the proposed link is relevant and credible“)

6/8/2010 34


6/8/2010 35


Revision of initial argument

tSketch of initial argumentEntering search terms

Evaluating hit list

Discussion

Plenary

Small group

Classroom script with online search processes on smallgroup level only

Finding information on selected web site

... ...

Individual

6/8/2010 36


Revision of initial argument

tSketch of initial argumentEntering search terms

Evaluating hit list

Discussion

Plenary

Small group

Classroom script with alterations between small group and plenary level

Finding information on selected web site

... ...

Individual

6/8/2010 37


Method

• Dependent variable: Online search competence (post test)

• Operationalization: Test that asked students(individually) to describe how they would proceed whenbeing asked to use the Internet for forming a positionon a different topic 2 components:

• Description of steps• Description of quality criteria

• Sufficient intra-class correlation for overall scale and both sub-scales (ICC > .67)

6/8/2010 38


Results

0

1

2

3

4

5

without smallgroup script

with smallgroup script

Classroomscript "smallgroup levelonly"

Classroomscript"alterationplenary-smallgroup level"M

ean

post

test

sco

res

onlin

e se

arch

com

pete

nce Significant

interaction effect: (F(1,165)=12.41; p<.01; partial Eta²=.07)

**

6/8/2010 39


Discussion1. Can small group collaboration scripts be

designed to support learners‘ acquisition of online search competence?

2. Do positive effects on domain-generalcompetences (e.g., Internet searchcompetence) also occur under field conditionsemploying longer learning phases?

3. Can the effects of small-group collaborationscripts be amplified by a classroom script thatdistributes learning activities over the plenaryand the small group level?

Yes, they can!

Yes, they do!

Obviously not!

6/8/2010 40


Discussion

• Scripting on the small group level can beneglected when activities are well distributedover the plenary and the small group level

• When activities are exclusively realized on thesmall group level, they need to be scripted witha small group collaboration script

• no „synergistic scaffolding“ (Tabak, 2004) through the combination of the two scaffolds

6/8/2010 41


Open questions

• What learning processes contributed to theresult patterns?

• What if other skills than online searchcompetence are aimed at?

• What if an individual phase would be added?

6/8/2010 42


Summary and overall conclusions• Computer technologies may help orchestrate learning

processes and scaffolds over the different social planes • Development of technological tools for orchestrating

learning should be informed by instructional approaches• Sometimes, successful orchestration means to not play

out the full potential of distributing and scaffoldinglearning activities!

• More empirical research on the effects of distributing and scaffolding activities over the social planes of theclassroom is neccessary

• Importance of „orchestrating learning“ also informal and purely technology-based learning settings

6/8/2010 43


Thank you for your attention!

Contact information:

[email protected]

http://www.psy.lmu.de/ffp_en/persons/ag-fischer/kollar-ingo/index.html

Orchestrating learning: survey

Education

Transcript of Orchestrating learning: survey