Author: Fang Wei, Glenn BlankAuthor: Fang Wei, Glenn Blank

Department of Computer Department of Computer ScienceScience

Lehigh UniversityLehigh University

July 10, 2007July 10, 2007

A Student Model A Student Model for an Intelligent Tutoring System for an Intelligent Tutoring System

Helping Novices LearnHelping Novices LearnObject Oriented DesignObject Oriented Design

Intelligent Tutoring System Intelligent Tutoring System (ITS)(ITS)

A computer-based instructional system A computer-based instructional system has knowledge bases for instructional content has knowledge bases for instructional content

and teaching strategiesand teaching strategies uses a student’s level of mastery of topics to uses a student’s level of mastery of topics to

adapt instruction dynamically adapt instruction dynamically A cost-effective means of one-on-one A cost-effective means of one-on-one

tutoring to provide novices with tutoring to provide novices with individualized attentionindividualized attention

Computer Assisted Instruction (CAI) system Computer Assisted Instruction (CAI) system does not model what a student is learning does not model what a student is learning and cannot adapt to studentand cannot adapt to student CAI provides same instruction, problems and CAI provides same instruction, problems and

feedback to every studentfeedback to every student

Intelligent Tutoring SystemIntelligent Tutoring System

Typically contains three main Typically contains three main components: components: An expert evaluator that observes a An expert evaluator that observes a

student’s work and identifies errors in student’s work and identifies errors in his/her solution his/her solution

A student model that diagnoses gap in A student model that diagnoses gap in student’s knowledge student’s knowledge

A pedagogical advisor that provides A pedagogical advisor that provides feedback to studentfeedback to student

Student ModelStudent Model Maintains a model of students’ current Maintains a model of students’ current

knowledge state by rknowledge state by representing and epresenting and updatingupdating

Provides information for intelligent Provides information for intelligent pedagogical decisions and actions including:pedagogical decisions and actions including: curriculum sequencingcurriculum sequencing interactive problem solving supportinteractive problem solving support pedagogical tutoring customized to each pedagogical tutoring customized to each

individual student’s learning state individual student’s learning state

AuthorsAuthors System System ContextContext

Consider Consider historyhistory

Diagnose Diagnose ConceptConcept

Pre-Pre-requisitesrequisites

Real Real TimeTime

Murray (1998)Murray (1998) Desktop Desktop AssociateAssociate

skillsskills √√VanLehn et al.VanLehn et al.(2001, 2005)(2001, 2005)

Solve physics Solve physics problemsproblems

rules, not rules, not conceptsconcepts √√

Butz et al. Butz et al. (2004)(2004)

C++ C++ programming programming √√ No No

evaluationevaluation

Millan et al.Millan et al.(2002, 2005)(2002, 2005)

CAT for mathCAT for math √√ √√ Post-Post-processprocess

Reye(1996, Reye(1996, 1998, 2004)1998, 2004)

Theoretical Theoretical

analysisanalysis √√ √√

Wei&Blank Wei&Blank (2006,2007)(2006,2007)

OO Design OO Design (UML)(UML) √√ √√ √√ √√

Student Model in Wei & Blank (2006,2007)compared with other BN Student Models

Layers of Student Layers of Student KnowledgeKnowledge

(Self 1994)(Self 1994) Domain knowledge layerDomain knowledge layer explain all explain all vocabularyvocabulary for discussing or solving for discussing or solving

problems problems

Reasoning knowledge layerReasoning knowledge layer contain reasoning relationships between propositions contain reasoning relationships between propositions

in domain knowledge in domain knowledge

Monitoring knowledge layerMonitoring knowledge layer specify how to solve a problem using reasoning specify how to solve a problem using reasoning

knowledge and domain knowledge knowledge and domain knowledge

Reflective knowledge layerReflective knowledge layer specify appropriate strategies students should have specify appropriate strategies students should have

in a learning environment in a learning environment

Three Layered ArchitectureThree Layered Architecture

• CM recognizes cognitive strategies that a student is using

•HM simulates students’ hierarchical knowledge in a history

•PDM simulates current students’ hierarchical knowledge

actor

actor_object

object

object_class

class

class_attribute

attribute

attribute_constructor

constructor

doubleint

numeric datatype

datatype

string

datatype_variable

variable

variable_parameter

parameter

variable_returntype

returntype

pass in only

class_method

method

method_constructor

class_constructor

object_constructor

method_parameter

variable_attribute

object_attribute

object_method

double_int

int_string

double_string

method_returntype

datatype_returntype

attribute_method

attribute_parameter

actor_method

A is prerequisite of B A B

Curriculum Information NetworkCurriculum Information Network

Two kinds of concepts Two kinds of concepts

UniqueUnique concept, such as attribute or concept, such as attribute or parameterparameter

RelationshipRelationship concepts, such as concepts, such as attribute_parameterattribute_parameter

Relationships emerge because of student’s Relationships emerge because of student’s confusions between conceptsconfusions between concepts

E.g., student defines E.g., student defines movieTitlemovieTitle as a as a parameter when he has already defined parameter when he has already defined movieTitlemovieTitle as an attribute as an attribute

Prerequisite relationshipsPrerequisite relationships

Prerequisite is relationship between concepts:Prerequisite is relationship between concepts: The concepts a learner needs to understand The concepts a learner needs to understand

before understanding a conceptbefore understanding a concept E.g., one needs to understand int and double E.g., one needs to understand int and double

in order to understand numericDatatypein order to understand numericDatatype

Relationship concepts are prerequisites of Relationship concepts are prerequisites of unique concepts and vice versaunique concepts and vice versa

E.g., class_constructor -> constructorE.g., class_constructor -> constructor Understanding constructor doesn’t imply Understanding constructor doesn’t imply

understanding of class, just how to define a understanding of class, just how to define a constructor for a classconstructor for a class

Connecting Knowledge with Connecting Knowledge with PerformancePerformance

Student action unit and knowledge unit Student action unit and knowledge unit make a pair(make a pair(KUKU,,AUAU)) Infer understanding of a concept (KU) Infer understanding of a concept (KU)

from a student solution step (AU)from a student solution step (AU) Action unit (AU): Action unit (AU):

A single action or step in a student’s A single action or step in a student’s solutionsolution

E.g., add an attribute to a classE.g., add an attribute to a class Knowledge unit (KU) – concept a student Knowledge unit (KU) – concept a student

need to learnneed to learn KU directly causes a student action unitKU directly causes a student action unit KU is a concept in Curriculum Information KU is a concept in Curriculum Information

Network (CIN)Network (CIN)

au

ku

……

au

ku

d-prereq(ku)1 d-prereq(ku)2d-prereq(ku)N

Atomic Bayesian Network (ABN)

Noisy-andgeneralizeslogical-and

Students must understand all direct prerequisites of the concept ku in order to understand ku

How to generate an ABNHow to generate an ABN

Student model generates an ABN in Student model generates an ABN in response to a student solution stepresponse to a student solution step

First, define the structure of an ABN, First, define the structure of an ABN, i.e., the causal relationship between i.e., the causal relationship between KU and AU, and the direct-KU and AU, and the direct-prerequisites of KUprerequisites of KU

Second, determine conditional Second, determine conditional probability tables for this ABNprobability tables for this ABN

…

au

ku

d-p(ku)1

d-p(ku)2

d-p(ku)N

…

au

ku

d-p(ku)1

d-p(ku)2

d-p(ku)N

0

0

0

0

0

1

1

1

1

1

Atomic Dynamic Bayesian Network (ADBN) for HM layer

How to generate an ADBNHow to generate an ADBN

Student model generates an ADBN in Student model generates an ADBN in response to a student solution stepresponse to a student solution step

First, look for the ABN in response to First, look for the ABN in response to previous student solution step previous student solution step

Second, generate an ABN in response Second, generate an ABN in response to current student solution stepto current student solution step

Third, determine conditional Third, determine conditional probability tables for the ADBNprobability tables for the ADBN

Concrete ExampleConcrete Example

Student defined Student defined movieTitle movieTitle as a as a parameter for method parameter for method displayMovieTitledisplayMovieTitle after she has already defined after she has already defined movieTitlemovieTitle as an attribute to a class as an attribute to a class TicketMachineTicketMachine

EE determines that EE determines that movieTitle movieTitle should should not be a parameter not be a parameter

SM determines that the center concept SM determines that the center concept of an ABN is of an ABN is attribute_parameterattribute_parameter, and, and finds all direct prerequisites, finds all direct prerequisites, attributeattribute and and parameterparameter, from CIN , from CIN

Concrete ExampleConcrete Example

attributeattribute’s prior can be found from the database ’s prior can be found from the database parameterparameter’s prior is 0.5, students’ knowledge ’s prior is 0.5, students’ knowledge

state is assessed based on the difference state is assessed based on the difference between prior and posterior probabilities between prior and posterior probabilities (VanLehn (VanLehn et al.et al. 1998, Millán & Pérez-de-la-Cruz 1998, Millán & Pérez-de-la-Cruz 2002)2002)

SM determines: SM determines: student has good understanding of student has good understanding of classclass, , attribute,attribute,

methodsmethods, and , and parameterparameter but low understanding of but low understanding of attribute_parameterattribute_parameter

the tutoring need is: explanation of the tutoring need is: explanation of attribute_parameterattribute_parameter

Concrete ExampleConcrete Examplefeedbackfeedback

““Since you have added Since you have added movieTitlemovieTitle as as an attribute to the class an attribute to the class TicketMachineTicketMachine, , you shouldn’t also make it a parameter you shouldn’t also make it a parameter to the method to the method displayMovieTitledisplayMovieTitle. To . To decide whether movieTitle should be decide whether movieTitle should be an attribute or a parameter, an attribute or a parameter, remember: attributes are accessible remember: attributes are accessible anywhere within the scope of a class, anywhere within the scope of a class, while parameters are accessible only while parameters are accessible only within the scope of a method”within the scope of a method”

ConclusionsConclusions Student models with ADBNs can Student models with ADBNs can

diagnose student knowledge states diagnose student knowledge states accurately in real-timeaccurately in real-time

Accuracy of ADBN-based student Accuracy of ADBN-based student model is significantly higher than ABN model is significantly higher than ABN student modelstudent model

Future workFuture work Implement cognitive model to simulate Implement cognitive model to simulate

monitoring knowledge and reflective monitoring knowledge and reflective knowledgeknowledge

Consider students learning gain from Consider students learning gain from reviewing feedbackreviewing feedback how do we determine the conditional probability how do we determine the conditional probability

table for the ADBN so as to simulate the real table for the ADBN so as to simulate the real student learning? student learning?

how do we update the new ADBN? how do we update the new ADBN? how do we convey how do we convey empirical studies with empirical studies with

simulated students and human subjects?simulated students and human subjects? Diagnose students’ learning state in other Diagnose students’ learning state in other

domains, such as object-oriented domains, such as object-oriented programmingprogramming