Hiroyuki Matsuda (Yokohama Nat’l Univ ) Makino M, Yumoto T, Sato T
SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning...
-
Upload
sophie-kelley -
Category
Documents
-
view
213 -
download
1
Transcript of SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning...
![Page 1: SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning Noboru Matsuda Human-Computer Interaction Institute Carnegie.](https://reader031.fdocuments.in/reader031/viewer/2022032806/56649efd5503460f94c10b97/html5/thumbnails/1.jpg)
SimStudent:A Computational Model of Learning as a Research Toolbox for the Sciences of Learning
Noboru Matsuda
Human-Computer Interaction Institute
Carnegie Mellon University
![Page 2: SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning Noboru Matsuda Human-Computer Interaction Institute Carnegie.](https://reader031.fdocuments.in/reader031/viewer/2022032806/56649efd5503460f94c10b97/html5/thumbnails/2.jpg)
Sim
Stu
den
t Pro
ject
PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU)
Research Questions
• Building a cognitive model is hard. Can machine-learning techniques help non-expert authors build a Cognitive Tutor?
• Would like to simulate students’ learning. Can machine-learning techniques help us build a computational model of learning with a cognitive fidelity?
• I heard that students learn by teaching others. Can we use the computational model of learning to study the theory of learning by teaching?
2
![Page 3: SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning Noboru Matsuda Human-Computer Interaction Institute Carnegie.](https://reader031.fdocuments.in/reader031/viewer/2022032806/56649efd5503460f94c10b97/html5/thumbnails/3.jpg)
Sim
Stu
den
t Pro
ject
PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU) 3
Solution: SimStudent
• Machine learning agent
– Learns procedural skills, by
– Observing model solutions & solving problems
• Fundamental technology
– Programming by Demonstration
– Inductive Logic Programming
• Knowledge representation
– Production rules (Jess)
Lau & Weld (1998). Blessing (1997).
![Page 4: SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning Noboru Matsuda Human-Computer Interaction Institute Carnegie.](https://reader031.fdocuments.in/reader031/viewer/2022032806/56649efd5503460f94c10b97/html5/thumbnails/4.jpg)
Sim
Stu
den
t Pro
ject
PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU)
SimStudent Projects
• Intelligent Authoring– Building a Cognitive Tutor as a CTAT Plug-in
• Student Modeling and Simulation– Controlled educational studies
– Error formation study
– Prerequisite conceptual knowledge study
• Teachable Peer Learner– Learning by teaching
4
![Page 5: SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning Noboru Matsuda Human-Computer Interaction Institute Carnegie.](https://reader031.fdocuments.in/reader031/viewer/2022032806/56649efd5503460f94c10b97/html5/thumbnails/5.jpg)
Sim
Stu
den
t Pro
ject
PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU) 5
Authoring a Cognitive Tutor
• Example-Tracing Tutor
– Little programming
– A cognitive model specific to a particular problem
• Limited generalization by editing a behavior graph
• Model-Tracing Tutor
– Powerful student model– Cognitive task analysis is hard– Writing production rules is even more challenging
• Performing a task is relatively easy…
![Page 6: SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning Noboru Matsuda Human-Computer Interaction Institute Carnegie.](https://reader031.fdocuments.in/reader031/viewer/2022032806/56649efd5503460f94c10b97/html5/thumbnails/6.jpg)
Sim
Stu
den
t Pro
ject
PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU) 6
Next Generation AuthoringBuild a tutor GUI
Teaching a solution
SimSt. learning
Rule simplify-LHS:
IF is-equation( Eq ),
is-lhs( Eq, Lhs ),
polynomial( Lhs ),
all-var-terms( Lhs )
Then simplify( Lhs, S-lhs ),
enter( S-lhs )
Production Rules
Rule simplify-LHS:
IF is-equation( Eq ),
is-lhs( Eq, Lhs ),
polynomial( Lhs ),
all-var-terms( Lhs )
Then simplify( Lhs, S-lhs ),
enter( S-lhs )
Rule simplify-LHS:
IF is-equation( Eq ),
is-lhs( Eq, Lhs ),
polynomial( Lhs ),
all-var-terms( Lhs )
Then simplify( Lhs, S-lhs ),
enter( S-lhs )
![Page 7: SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning Noboru Matsuda Human-Computer Interaction Institute Carnegie.](https://reader031.fdocuments.in/reader031/viewer/2022032806/56649efd5503460f94c10b97/html5/thumbnails/7.jpg)
Sim
Stu
den
t Pro
ject
PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU)
Demo
7
Authoring by Tutoring
![Page 8: SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning Noboru Matsuda Human-Computer Interaction Institute Carnegie.](https://reader031.fdocuments.in/reader031/viewer/2022032806/56649efd5503460f94c10b97/html5/thumbnails/8.jpg)
Example: Learning to subtract a constant term
PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU) 8
Learning to subtract a constant numberFirst example
subtract 1
Subtract the last term on the left-hand side…Subtract the last term on the left-hand side…
Subtract the coefficient of X…
Subtract the coefficient of X…
Subtract the difference between 4 and 3…
Subtract the difference between 4 and 3…
I see 3x, 1, x, and 4 in the equation.I wonder where the ‘1’ came from…
I see 3x, 1, x, and 4 in the equation.I wonder where the ‘1’ came from…
![Page 9: SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning Noboru Matsuda Human-Computer Interaction Institute Carnegie.](https://reader031.fdocuments.in/reader031/viewer/2022032806/56649efd5503460f94c10b97/html5/thumbnails/9.jpg)
Example: Learning to subtract a constant termLearning to subtract a constant number
First example
subtract 1
Subtract the last term on the left-hand side…
Subtract the last term on the left-hand side…
Subtract the coefficient of X…Subtract the coefficient of X…
Subtract the difference between 4 and 3…Subtract the difference between 4 and 3…
I see 3x, 1, x, and 4 in the equation.I wonder where the ‘1’ came from…
I see 3x, 1, x, and 4 in the equation.I wonder where the ‘1’ came from…
PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU) 9
![Page 10: SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning Noboru Matsuda Human-Computer Interaction Institute Carnegie.](https://reader031.fdocuments.in/reader031/viewer/2022032806/56649efd5503460f94c10b97/html5/thumbnails/10.jpg)
Prior Knowledge• Feature predicates
– 18 predicates– isFractionTerm(X), isConstant(X), isPolynomial(X),…
• Operators– 42 operators– add(X,Y), coefficient(X), getFrstNumber(X),
…
10PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU)
![Page 11: SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning Noboru Matsuda Human-Computer Interaction Institute Carnegie.](https://reader031.fdocuments.in/reader031/viewer/2022032806/56649efd5503460f94c10b97/html5/thumbnails/11.jpg)
Sim
Stu
den
t Pro
ject
PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU) 11
Example: Stoichiometry Tutor
![Page 12: SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning Noboru Matsuda Human-Computer Interaction Institute Carnegie.](https://reader031.fdocuments.in/reader031/viewer/2022032806/56649efd5503460f94c10b97/html5/thumbnails/12.jpg)
Sim
Stu
den
t Pro
ject
PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU)
SimStudent Projects
• Intelligent Authoring– Building a Cognitive Tutor as a CTAT Plug-in
• Student Modeling and Simulation– Controlled educational studies
– Error formation study
– Prerequisite conceptual knowledge study
• Teachable Peer Learner– Learning by teaching
12
![Page 13: SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning Noboru Matsuda Human-Computer Interaction Institute Carnegie.](https://reader031.fdocuments.in/reader031/viewer/2022032806/56649efd5503460f94c10b97/html5/thumbnails/13.jpg)
Sim
Stu
den
t Pro
ject
PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU)
Model of Incorrect Learning
• Identify errors students commonly make
• Weaken SimStudent’s background knowledge
• Let SimStudent make an induction error
13
![Page 14: SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning Noboru Matsuda Human-Computer Interaction Institute Carnegie.](https://reader031.fdocuments.in/reader031/viewer/2022032806/56649efd5503460f94c10b97/html5/thumbnails/14.jpg)
Sim
Stu
den
t Pro
ject
PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU)
Weak Prior Knowledge Hypothesis
• Multiple ways to make sense of examples
14
3x=5 “divide by 3”3x=5 “divide by 3”
Get a coefficient and divideGet a coefficient and divide
Get a number and divideGet a number and divide
Get a denominator and multiplyGet a denominator and multiply
4/x=5
“divide by 4”
“multiply by x ”
![Page 15: SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning Noboru Matsuda Human-Computer Interaction Institute Carnegie.](https://reader031.fdocuments.in/reader031/viewer/2022032806/56649efd5503460f94c10b97/html5/thumbnails/15.jpg)
Sim
Stu
den
t Pro
ject
PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU)
Results: Learning Rate
Steps Score = 0 (if there is no rule applicable)# correct rule applications / # all rule applications
# training problems
Step Score
![Page 16: SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning Noboru Matsuda Human-Computer Interaction Institute Carnegie.](https://reader031.fdocuments.in/reader031/viewer/2022032806/56649efd5503460f94c10b97/html5/thumbnails/16.jpg)
Student Model A set of knowledge components (KCs) Encoded in intelligent tutors to model how
students solve problems E.g. How to proceed given problems of the form Nv=N
One of the key factors that affects automated tutoring systems in making instructional decisions
Previous Approach: Require expert input Highly subjective
Proposed Approach: Use a machine-learning agent, SimStudent, to acquire
knowledge 1 Skill 1 KC Skill application for that step KC for each step
(Li et al. EDM2011) A Machine Learning Approach for Automatic Student Model Discovery
![Page 17: SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning Noboru Matsuda Human-Computer Interaction Institute Carnegie.](https://reader031.fdocuments.in/reader031/viewer/2022032806/56649efd5503460f94c10b97/html5/thumbnails/17.jpg)
Human-generated vs SimStudent KCsHuman-generated Model
SimStudent
Comment
Total # of KCs 12 21
# of Basic Arithmetic Operation KCs
4 13 Split into finer grain sizes based on different problem forms
# of Typein KCs 4 4 Approximately the same
# of Other Transformation Operation KCs (e.g. combine like terms)
4 4 Approximately the same
4x = 20 vs.
–x = 5
![Page 18: SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning Noboru Matsuda Human-Computer Interaction Institute Carnegie.](https://reader031.fdocuments.in/reader031/viewer/2022032806/56649efd5503460f94c10b97/html5/thumbnails/18.jpg)
Results
Human-generated Model
SimStudent
AIC 6529 6448
3-Fold Cross Validation RMSE
0.4034 0.3997
Significance Test SimStudent outperforms the human-
generated model in 4260 out of 6494 steps p < 0.001
SimStudent outperforms the human-generated model across 20 runs of cross validation
p < 0.001
![Page 19: SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning Noboru Matsuda Human-Computer Interaction Institute Carnegie.](https://reader031.fdocuments.in/reader031/viewer/2022032806/56649efd5503460f94c10b97/html5/thumbnails/19.jpg)
Sim
Stu
den
t Pro
ject
PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU)
SimStudent Projects
• Intelligent Authoring– Building a Cognitive Tutor as a CTAT Plug-in
• Student Modeling and Simulation– Controlled educational studies
– Error formation study
– Prerequisite conceptual knowledge study
• Teachable Peer Learner– Learning by teaching
19
![Page 20: SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning Noboru Matsuda Human-Computer Interaction Institute Carnegie.](https://reader031.fdocuments.in/reader031/viewer/2022032806/56649efd5503460f94c10b97/html5/thumbnails/20.jpg)
Sim
Stu
den
t Pro
ject
PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU)
Learning by Teaching SimStudent
![Page 21: SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning Noboru Matsuda Human-Computer Interaction Institute Carnegie.](https://reader031.fdocuments.in/reader031/viewer/2022032806/56649efd5503460f94c10b97/html5/thumbnails/21.jpg)
Sim
Stu
den
t Pro
ject
PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU)
Learn more about SimStudents
• Project Web– www.SimStudent.org
• Contact us– Noboru Matsuda
21