Daniel Schwartz Stanford University

Daniel Schwartz

Stanford University

Interactivity and Learning

I don’t mean to be vain, but does this really look like me?


Some types of learning do not seem to be the sort of thing that the representations and interactivity of the computer will be particularly helpful for.

Artificial Intelligence in Education Two main capacities of interest:

Social Interactivity AI techniques create socially inspired interactions

Learning in Formal Domains Math, science, and other things that can be well modeled.

Soon, there will be sufficient models of social interaction that we can help students learn valued social behaviors. (Many attempts are underway.)

Goal of the talk Present some history of research on valued social

interactions – which is largely about motivation and attitude change.

Describe two dimensions of valued social interactions – which are largely about learning.

Present some suggestive research that my colleagues have done with teachable agents.

Discuss some academic history as to what counts as valuable learning.

Describe how modeling social interactions can yield new possibilities for valuable learning.


Goals of social interaction research Enhancing social interaction for learning Teachable Agents Relevant Evidence using the Agents Goals of learning research Sweet Spot of Social Interaction and Learning

Early research on social interaction For some, the goal of studying or controlling

interaction is to improve human interaction per se.

This includes the research on

cooperation that spawned

cooperative learning.

Interactivity

Valued Interactivity

How to Create Productive Interactions?

Research in response to WWII The goal was conflict resolution and cooperation

Morton Deutsch, 1973 “I started my graduate career not long after Hiroshima

and Nagasaki, and my work in social psychology has been shadowed by the atomic cloud ever since. The efforts reported in this book reflect my continuing interest in contributing to the understanding of how to prevent destructive conflict and initiate cooperation.”

Led to Educational Applications About motivation and attitude.

Content learning not the target so much as “attitude change” and “motivation management.”

Still, led to applications for learning math, science, reading.

Begins from assumption of potential conflict or withdrawal. Not such a bad assumption for many school settings (in U.S.).

Applications to learning. If students work cooperatively, they might improve their learning.

Two key conditions from cooperation research: Mutual Interdependence Individual Accountability

Slavin’s (1996) meta-analysis on cooperative learning: MI or IA = +.07 effect size MI & IA = +.32 effect size

Unfortunately, only 25% of teachers who are trained implement both. (Antil et al., 1998).

Shift to collaboration More recently, interest in collaboration.

Collaboration is cooperation in the absence of serious conflict.

Collaboration is a valued form of interaction. But, it does not necessarily mean people will learn well.

Barron et al. (1998) study…

In summary The goal of much research is to promote valued

interactions per se. Exploration of motivations for positive interactions.

When borrowed by education, leads to a model: Motivation Valued Interactions ( Learning) The motivations are for interactions, and not learning

Need to understand valued interactions that directly “motivate” learning.

OUTLINE HERE

Valued Interactionsfor Learning

The umbrella of valued interactions (Deutsch) “A cooperative process is characterized by open

and honest communication of relevant information among participants. Each is interested in informing, and being informed by, the other.”

What conditions turn this into learning?

Two dimensions

Incorporation of Ideas The degree to which participants’ ideas are taken up

by one another.

Initiative in Action The degree to which all participants’ can initiate

actions.

High motivation when achieved. Take the example of conversation:

People like to talk.

People like to make the effort after shared meaning.

The effort to produce and share meaning. We want others to incorporate the ideas we initiate. We want to incorporate the ideas that others initiate (even if

just to disagree or elaborate).

Positive Instances Think of an animated conversation…

The world drops away. Try to persuade others to understand and incorporate your

ideas. Listen to how they uptake your ideas and reflect them back. Listen to their ideas and reflect them back, combined with

your own ideas.

A talk is a slow version (stylized turn taking). Writing a journal article is a really slow version. Gossip is a favorite version.

Negative Instances

No chance to take the initiative blocked from entering conversation told what to do or say listening with no prospect of action

No chance to incorporate your ideas talking to people who do not understand talking to people who cannot respond ignored during conversation

Interactivity Space for Novice Learningin Motivating Collaborations

Self OtherMixed

Initiative(actions)

Self

Other

Merged

Inco

rpora

tion

(ideas)

Showing

WatchingCopying

Being Imitated

OptimalLearning

ForNovices

Not just low motivation…also, low learning.

No mutual incorporation of ideas: Neither teacher nor student learns.

No sharing of the initiative: Neither teacher nor student learns.

Examples of optimal region

Self OtherMixed

Initiative(actions)

Self

Other

Merged

Inco

rpora

tion

(ideas)

OptimalLearning

ForNovices

Computer Technologies

Present some research relevant to the two dimensions of interactivity.

I’ll look at teaching as the example As last movie clip showed, not all teaching yields

ideal social interaction and learning. We use computers to help optimize the balance of

incorporation and initiative.

Teachable Agents Learning By Teaching

Common wisdom people “really” learn when they teach.

Empirical findings Students who prepare to teach learn more than students who

prepare to take a test. (Bargh & Schul,1980; Biswas, et al., 2001)

Built computer agents that students teach A natural social interaction students know well

Teach – Test – Remediate

Basic Teaching Interaction

Not machine induction; students must explicitly teach.

Students teach agent. Students uses visual representations to teach the agent.

Agent performs based on teaching. Generic AI algorithms draw inferences based on student teaching.

Students revise agent to do better. Based on agent performance student revises agent and own

knowledge.

Betty: A Teachable Agent

Extensions to TA paradigm

Students know they are not real people. We are more interested in enabling social

interactions that facilitate learning. The well-known teaching schema works well. Plus, once the basic interaction is developed,

they can be extended in numerous ways.

Videogames(Kristen Blair)

Students teach agent to perform in game. Besides motivation and game leveling,

it enables a number of learning resources

On-Line Homework Game Show(Paula Wellings)

Students can log on, chat, and do homework with whomever is on-line.

Teach agent, who performs in a gameshow.

Front of the Class System(Joan Davis)

Students create models that can answer all questions, instead of memorizing a few answers to select problems.

Present results at front of class.

Gain Scores in Correct Answers for Concept Maps after Feedback and Revision

0%

5%

10%

15%

20%

25%

One (n=26) Two (n=16) Three-Four(n=18)

Seven-Eight(n=8)

Links Involved in Question

Incr

ease

in %

Cor

rect

Front-of-Class

Solitary

Feedback Condition

A Suite of Homely Teachable Agent “Engines”

Betty Qualitative Reasoning

Orbo Reasoning by Assumption

Milo Reasoning by Model

Moby Hypothetico-Deductive Reasoning

J-Mole Reasoning by Discrepancy

Also, a suite of homely collaborators

In order of homeliness: Gautam Biswas John Bransford Krittaya Leelawong, Thomas Katzleberger, Ying Bin Joan Davis, Kristen Blair, Paula Wellings, George Chang Girija Mittagunta, Elliot Castillo, Anh Huynh, Nancy Vye

Return to proposal about valued interactions that promote learning.

Self OtherMixed

Initiative(actions)

Self

Other

Merged

Inco

rpora

tion

(ideas)

OptimalLearning

ForNovices

Incorporation Agents, by design, merge ideas with students.

Students provide facts of the matter. Agent provides spatial representation and reasoning.

Not just learning the brute facts, learning how the “expert” thinks with those facts. Literally making thinking visible.

Hope is that merging with Betty’s representations and reasoning will lead student to learn and adopt those representations.

Interactivity Space for Novice Learning

Student AgentMixed

Initiative(actions)

Student

Agent

Merged

Inco

rpora

tion

(ideas) Would students learn causal structure

when ideas get merged with agent?

Merging Ideas

Undergraduates read exercise physiology text.

8 Taught Betty on cell metabolism. 8 Wrote Summary on cell metabolism.

Would students adopt Betty’s knowledge structure?

Direction of Causality

During activity:

Betty students discovered

they had confused

causation and correlation.Mitochondria <-> ATP synthesis

Summary students tended to

focus on outlining0%

25%

50%

75%

100%

Betty Summary

Multiple Causality

0

1

2

3

4

Betty Summary

Learning TaskC

ompl

ex L

inks

Given a metabolism word, list entities related to it.

Simple Link: Mitochondria increase ATP synthesis.

Complex Link: Mitochondria with glycogen or free fatty acids increase ATP synthesis.

Dynamic aspect of thinking Merging is not just of representations, but also of

reasoning. Wanted to examine if the AI-reasoning component was

important for merging ideas ideas. 4th-grade students learned about pond ecology over

three days. Animation condition:

Taught Betty and she could answer their questions. No Animation condition:

Created concept map using Betty (reasoning turned off)

Adoption of Causal Structure

0

5

10

15

20

Day 1 Day 2 Day 3

Nu

mb

er

of

Ca

us

al L

ink

s

No Animation

Animation

Opportunities to commingle thoughts with agent helped students learn/adopt causal structure.

How about initiative?

Interactivity Space for Novice Learning

Student AgentMixed

Initiative(actions)

Student

Agent

Merged

Inco

rpora

tion

(ideas)

Does mixing initiative with agent improve student learning?

Independent Performance Conversation is often taken as model of social interaction

Mixed-initiative involves shared lead. A broader view extends interaction over time.

Mixed-initiative can include independent performance. Teaching and then watching one’s student perform.

Student incorporates the teacher’s ideas, but also has to have abilities to do an independent performance.

Swedish dissertation defenses? A very powerful form of mixed initiative.

Motivating Excellent opportunity for teacher to learn.

Examine value of mixed-initiative as independent performance with a second agent.

Moby(Anh Huynh)

To teach science content using hypothetico-deductive reasoning.

Green and Not Pink are Necessary for a Flower(Shade and ~ Sun are Necessary for a Flower)

Significance of Independent Performance Study with 100+ high school seniors

Control Never used game

Play Played game themselves without teaching feature But did fill in “teaching rule” after each “win”

Teach Used game, filled in “teaching rule,” watched Moby play

Students reached same level in same time. Posttest of inductive-deductive reasoning

0%

25%

50%

75%

100%

Induce Imply Translate

Reasoning Task

Acc

urac

yControl

Play

Teach

0%

25%

50%

75%

100%


Reasoning Task

Acc

urac

yControl

Play

Teach

0%

25%

50%

75%

100%


Reasoning Task

Acc

urac

yControl

Play

Teach

Summary

Incorporation seems to help. Mixed-Initiative (as far as we went) helps.

Next is the question of what is valuable learning.

Problem 1: Narrow definitions

Detterman from Transfer on Trial. “…most studies fail to find transfer …and those

studies claiming transfer can only be said to have found transfer by the most generous of criteria and would not meet the classical definition of transfer.”

Classic “stimulus generalization” view – efficient replication of old behavior in a new situation.

(Bransford & Schwartz, 1999, Review of Research in Education)

The History of the Transfer Problem

Much of the psychological literature on learning has emphasized efficiency Faster and more accurate retrieval and

application of previously learned behaviors.

Efficiency’s long, dominant history in psychology and the USA…

Efficiency is important

99.9% = failure for orchestral musician.

Improved efficiency frees up cognitive resources.

Important for routine tasks.

Most learning assessments are about efficiency Speed, accuracy, consistency, 1st-try positive transfer

Issues with Efficiency Businesses worry that too much emphasis on efficiency

reduces innovation.

For novel learning, efficiency can interfere Assimilate to efficient schemas and miss what is new.

Novick finding of expert “letting go” at transfer.

Assessments of efficiency can miss early forms of knowledge that prepare students to learn.

Harvard students and seasons

Innovation Innovation involves generation of new ideas

Rather than refinement of pre-existing ones.

Efficiency & innovation often seen as opposites. Myth of creative person versus drudge.

Adaptive experts are presumably high on both. A strong set of efficient schemas to draw upon.

Ericsson’s 10-year latency to innovation. Ability to “let go,” adapt, and learn new ideas when needed.

Efficiency & Innovation

Efficiency

Innovation

Novice

AdaptiveExpert

Routine Expert

EternalNovice?

(Schwartz, Bransford, & Sears, in press)

Optimal Trajectory for Learning?In

nova

tion

Efficiency

AdaptiveExpertise

Problem 3: Restricted methods for assessing transfer

Sequestered problem solving assessments (SPS) Harvard students on the seasons.

Harvard(Treatment A)

High School(Treatment B)

Sequestered Transfer Assessments(Seasons)

Teaching Method A(discovery-based)

Teaching Method B(direct instruction)

Target Transfer Problem

Learning Resource in Test(worked example)

CorrectSolutions

67%

33%

Hypothesize that both are necessary and there is an optimal balance.

Inno

vati

on

Efficiency

AdaptiveExpertise

NoviceRoutineExpertise

FrustratedNovice?

Optimal A

daptabilit

y Corri

dor

- Help students learn to use resources for learning.

- Put Betty in larger instructional model.

- Develop intelligent tutor and coach.

Value of Teaching Agents for Future Learning

The mentor provided tips on:(i) How to be a better teacher,(ii) How to be a better learner, (iii) Help on domain concepts.

The Mentor Agent

Here, the mentor responds to a student query about decomposition. Rather than give a direct answer, the mentor suggests what material the student should look up.

Preparation for Future Learning 5th-graders

Taught Betty about the Oxygen Cycle. Tutored how to create Oxygen Cycle.

On posttest on Oxygen Cycle, the groups looked the same.

Returned 7 weeks later. Asked children to learn about Nitrogen Cycle and create a concept map.

Quality of Learning about Nitrogen Cycle,Weeks after Completing Intervention

0

1

2

3

4

5

6

7

Good Nodes Good Links

Teach

Tutored

A more nuanced view

Interactivity Learning

Valued Interactivity

Valued Learning

OUTLINE HERE

Limitations Limited value in feedback:

Betty simply answers. Need some way to evaluate whether an answer makes sense.

Limited resources for learning: Typically text resources on the side. Outside resources, not directly linked to agent interaction

Limited expressiveness: Betty does not use conditional logic, temporal cycles

Most motivation theories do not entail learning with understanding

Extrinsic Reward External reinforcement

Achievement motivation Motivates any sort of change

Intrinsic Reward “Desirable” internal state

Flow Motivates to maintain state

Motivation to Learn

Want a more specific motivation -- to continue learning with understanding

Some common proposals: Curiosity

Depends too much on environment Disequilibrium

Not something many people strive for

An Example of Motivation to Learn with Understanding

w/ Jay Pfaffman (at UT Knoxville)

Surveyed 100’s of hobbyists Motor cycle racers, beer brewers, musicians, etc.

People self-select into hobbies so motivating

They rated importance of different aspects of their hobby.

Five classes of questions (with 5 items each)

Sample Survey Item Types

How important is X for why you do your hobby?

Extrinsic Motivators – increasing success

Intrinsic Motivators – losing a sense of time

Social Motivators – belonging to a group

Learning Motivators – learning the history of my hobby

Production Motivators – being creative

Three Lowest Motivations

(1 is low, 7 is high)

2.7 – To Increase Success

2.8 – To Improve Stature

3.2 – To Be Liked

Highest Motivations

6.0 – See the “fruits” of their labor6.0 – Learn new methods to create products5.7 – Just manageable challenges5.6 – End in itself5.5 – Share artifacts/performances (products)

Asked high school students about their favorite classes and found similar results.

Learning is “built-in” to the hobby motivation

Personal feedback from the fruits of their labor. Reflect on externalized products and improve.

Social feedback and new possibilities through sharing Other people’s responses, variations, and products.

Consult resources to make new artifacts/performances. What we typically think of as learning behaviors.

Increasing challenges create more needs for learning

When do we choose collaboration to learn?

Education often misses this question. Students are told to collaborate We orchestrate various structures:

Rewards Scripts and norms Interdependency

Consider when collaborative learning intent naturally appears.

Outline Where learning is integral to motivation.

An example from hobbyists Is learning integral to collaborative motivations?

A production by efficiency task space.

Sample of research in the task space.

Addressing the assigned questions.


Is learning integral to collaborative motivations? When people make the effort to share meaning.




Motivation to share ideas something like hobbies

See the products of their labor See own ideas appear in other person

Sharing Learn from other’s responses and additions

Consult resources to make new products Recruit evidence (sometimes) to make a point

Increasing challenges Reply of other person can create a learning challenge


Is learning integral to collaborative motivations?

A production by efficiency task space. Tasks with productive agency lead to sharing.



Motivations to Collaborate:Production Productive Agency: The opportunity to produce own ideas

plus share meaning with those ideas. Hobby productions often in material form Social productions often in verbal form

Not a welfare model of learning. Not about just taking in the wealth of knowledge from an expert.

About the means of production. Do learners get to produce ideas (not just construct)? Do they have material and intellectual resources?

Plus opportunity to share the productions (not ownership). Do ideas get taken up and reflected back with variation?

Motivations to Collaborate:Efficient Performance Standard answers often involve efficiency.

Too much for one person. Error checking. More ideas to sample from.

Research often examines efficiency. Are two heads better (more efficient) than one? Measures of the efficiency of information exchange.

School often emphasizes efficiency motivations Proximal task: solve a problem without error and for a grade. Learning is a side effect, not a motivation.

Over emphasis on efficiency may interfere with effort to share meaning because there is little original production.

Collaborative Task Space

Efficiency Tasks(speed, accuracy)

Production Tasks(invention, authoring)

Brain Storming

Invent/DesignSolutions

ImplementProcedure

InsightProblems

Which task regions naturally lead people to want to share meaning and learn through collaboration?

Individuals & Dyads on Production Task

Received Passage:

When either the Spotted Halluck or the Black Froling live in a lake, the lake has weeds. However, the Spotted Halluck needs a lake with trees around it, while the Black Froling needs a lake with minnows and a sandy bottom….

Told to invent a representation that would help answer questions like:

What fish need weeds and trees?

A Surprising Outcome of Effort to Share Meaningin a Production Task

Individual rate of abstract production = 6%

Actual rate of dyad abstract production = 67%

Emergence of abstraction replicated for several production tasks. Abstraction not the goal of collaboration – it naturally emerged. Shared meanings yield an abstraction of common structure.

MOST COMPETENT MEMBER MODEL A & B: .06 x .06 = .0036 A & ~B: .06 x .94 = .0564~A & B: .94 x .06 = .0564 Dyad solution odds = .1164 = 11.6%

Hard to Imagine for an Efficiency-only Task

65432100

100

200

300

400

500

600

Ideal

Effects of Collaboration on Weight Pulling

Number of People Pulling Together

Po

un

ds

Pu

lled

65432100

100

200

300

400

500

600

FoundIdeal

Effects of Collaboration on Weight Pulling

Number of People Jointly Pulling

Po

un

ds

Pu

lled

65432100

100

200

300

400

500

600

Number of People Jointly Pulling

+100 (each person)

-20 (each link)

Collaborative Task Space



Brain Storming


ImplementProcedure

InsightProblems


Is learning integral to collaborative motivations?


Sample of research in the task space. Efficiency vs. sharing interactions: Effects on learning.


What Conditions Promote aCollaborative Learning Intent?

Requires both efficiency and production. Production and sharing are motivating Need some efficiency of knowledge and constraint to

avoid floundering. People rated “learning new methods” highly in hobby study.

A Hypothesis Degree a task emphasizes efficiency and/or production

determines collaborative intent towards learning.

An initial example of what research might look like.



Brain Storming


ImplementProcedure

InsightProblems

versus

Telling differences in collaborative interactions.

Looked at how pairs used resources

Looked at differences in interactions Dedicated to efficiency

Discussions about how to organize work Partitioning of labor

Dedicated to sharing meaning Evaluating one another’s ideas Constructing shared, “content-based” beliefs

A Production Pair – Sharing Jointly evaluated each entry into the concept map

“That’s true what you just said, arterial plaque increases risk of heart disease.”

Even jointly challenged experimenter over a specific relation

Started finishing one another’s sentences

Repeatedly consulted passage and compared to map.

Relatively few discussions about roles in task.

An Efficiency Pair – Partitioning Negotiated separate roles

S1: “Why don’t I write down the links?”S2: “As long as you don’t mind being the scribe and don’t

mind me being the speaker.”

Read out – Copy strategy

When studying in remaining time Worked separately at first Then split passage and quizzed each other

“Shared” meaning typically occurred outside of task

Our hypothesis for future research Different types of collaborative learning

interactions for production and efficiency tasks. Production tasks generate effort to share meaning.

Good if the goal is new learning. Production pairs seemed to learn better.

Maybe not so good if goal is efficiency. Production pairs took longer.

Efficiency tasks lead to partitioning of work. Good if the goal is increasing efficiency. Maybe not so good if the goal is new learning.


An example from hobbyists Is learning integral to collaborative motivations?

When people make the effort to share meaning. A production by efficiency task space.

Tasks with productive agency lead to sharing. Sample of research in the task space.

Efficiency vs. sharing interactions: Effects on learning. Addressing the assigned questions.

Assigned Questions What has been under studied?

Why people choose to collaborate. How different motivations affect self-organization of learning.

What sorts of research designs? Manipulations of task space (or natural observations). Longer lasting studies that permit mobility among groups.

What sorts of measures? Choice of whether to collaborate throughout. Do different process markers predict successful collaboration?

Effort to share meaning in productive tasks. Partitioning of workload in efficiency tasks.

Subsequent effects on future collaboration and learning.

[email protected]@Stanford.Edu

Productive Task Space Works for Instruction? Pfaffman tried.

Create a simulation Change values Watch simulation

Measured Learning resource use Spontaneous sharing Overall learning

Results If he recovers from study…

Level of Production

inventiontasks

rhetoricaltasks

Different forms of collaboration Procedure

Dyads read passage on fat metabolism and heart attack. Received nodes that represent key elements. Made a concept map of relations between elements.

(Olive oil increases HDL; HDL decreases LDL; etc.) Complete a posttest working alone.

Conditions Implement dyads: Received written list of pairwise relations. Produce dyads: Created own way to connect nodes.

Effects of different collaborative conditions on interactions

Approximately the same number of interactions. Implement dyads partitioned labor

“Handed-off” knowledge to one another Produce dyads discussed ideas.

0102030

405060708090

100110120

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17+

# of Turns Taken on a Causal Relation

Fre

qu

ency Implement

Produce

Effects of different collaborative conditionson learning

Sears mapped each moment-to-moment interaction with its learning outcome as measured by a posttest.

Examined mutual learning (different from mutual understanding).

Mutual Learning as a Function of Turns Taken

0%

25%

50%

75%

1-3 4-6 7-9

Number of Turns Taken per Item During Learning

Bo

th M

em

be

rs C

orr

ec

t o

n P

os

tte

st

Ite

m

Produce

Implement

Daniel Schwartz Stanford University

Documents

Transcript of Daniel Schwartz Stanford University