Author: Fang WeiAuthor: Fang Wei

Advisor: Prof. BlankAdvisor: Prof. Blank

Department of Computer Department of Computer ScienceScience

Lehigh UniversityLehigh University

May 10, 2007May 10, 2007

A Student Model A Student Model for an Intelligent Tutoring System for an Intelligent Tutoring System

Helping Novices LearnHelping Novices LearnObject Oriented DesignObject Oriented Design

Presentation OutlinePresentation Outline

PublicationsPublications BackgroundBackground Research questionsResearch questions MethodologyMethodology Evaluation Evaluation Conclusions and future workConclusions and future work

PublicationsPublications Wei, F. & Blank, G.D. (2007) Wei, F. & Blank, G.D. (2007) Atomic Dynamic Bayesian Atomic Dynamic Bayesian

Networks for a Responsive Student ModelNetworks for a Responsive Student Model, Proceedings of , Proceedings of the 13th International Conference on Artificial Intelligence the 13th International Conference on Artificial Intelligence in Education, AIED 2007 in Education, AIED 2007

Wei, F. & Blank, G.D. (2006) Wei, F. & Blank, G.D. (2006) Student Modeling with Atomic Student Modeling with Atomic Bayesian NetworksBayesian Networks, Proceedings of the 8th International , Proceedings of the 8th International Conference on Intelligent Tutoring Systems, ITS 2006, pp. Conference on Intelligent Tutoring Systems, ITS 2006, pp. 491-502 491-502

Wei, F., Moritz, S., Parvez, S., & Blank, G.D. (2005) Wei, F., Moritz, S., Parvez, S., & Blank, G.D. (2005) A A Student Model for Object-Oriented Design and Student Model for Object-Oriented Design and ProgrammingProgramming. The Journal of Computing Sciences in . The Journal of Computing Sciences in Colleges (CCSC), Vol. 20, pp. 260-273. "Best Paper" AwardColleges (CCSC), Vol. 20, pp. 260-273. "Best Paper" Award

Blank, G. D., Parvez, S., Wei, F., Moritz, S (2005) Blank, G. D., Parvez, S., Wei, F., Moritz, S (2005) A web-A web-based ITS for object-oriented design. Poster for 12th based ITS for object-oriented design. Poster for 12th International Conference on Artificial Intelligence in International Conference on Artificial Intelligence in EducationEducation, Workshop of Adaptive Systems for Web-Based , Workshop of Adaptive Systems for Web-Based Education: Tools and reusability, Amsterdam, The Education: Tools and reusability, Amsterdam, The Netherlands, JuneNetherlands, June

Moritz, S., Wei, F., Parvez, S., & Blank, G. D. (2005), Moritz, S., Wei, F., Parvez, S., & Blank, G. D. (2005), From From objects-first to design-first with multimedia and intelligent objects-first to design-first with multimedia and intelligent tutoringtutoring. The Tenth Annual Conference on Innovation and . The Tenth Annual Conference on Innovation and Technology in Computer Science Education (ITiCSE), Monte Technology in Computer Science Education (ITiCSE), Monte da Caparica, Portugal, June. da Caparica, Portugal, June.

Presentation OutlinePresentation Outline

PublicationsPublications BackgroundBackground Research questionsResearch questions MethodologyMethodology Evaluation Evaluation Conclusions and future workConclusions and future work

Intelligent Tutoring System Intelligent Tutoring System (ITS)(ITS)

A computer-based instructional system A computer-based instructional system has knowledge bases for instructional content has knowledge bases for instructional content

and teaching strategiesand teaching strategies uses a student’s level of mastery of topics to uses a student’s level of mastery of topics to

adapt instruction dynamically adapt instruction dynamically A cost-effective means of one-on-one A cost-effective means of one-on-one

tutoring to provide novices with tutoring to provide novices with individualized attentionindividualized attention

Computer Assisted Instruction (CAI) system Computer Assisted Instruction (CAI) system does not model what a student is learning does not model what a student is learning and cannot adapt to studentand cannot adapt to student CAI provides same instruction, problems and CAI provides same instruction, problems and

feedback to every studentfeedback to every student

Intelligent Tutoring SystemIntelligent Tutoring System

Typically contains three main Typically contains three main components: components: An expert evaluator that observes a An expert evaluator that observes a

student’s work and identifies errors in student’s work and identifies errors in his/her solution his/her solution

A student model that diagnoses gap in A student model that diagnoses gap in student’s knowledge student’s knowledge

A pedagogical advisor that provides A pedagogical advisor that provides feedback to studentfeedback to student

Student ModelStudent Model Maintains a model of students’ current Maintains a model of students’ current

knowledge state by rknowledge state by representing and epresenting and updatingupdating

Provides information for intelligent Provides information for intelligent pedagogical decisions and actions including:pedagogical decisions and actions including: curriculum sequencingcurriculum sequencing interactive problem solving supportinteractive problem solving support pedagogical tutoring customized to each pedagogical tutoring customized to each

individual student’s learning state individual student’s learning state

MotivationMotivation Novices in high school and college have Novices in high school and college have

much difficulty in object oriented designmuch difficulty in object oriented design ITSs can aid learners with complex ITSs can aid learners with complex

problem-solvingproblem-solving DesignFirst-ITS developed to help novices DesignFirst-ITS developed to help novices

learn object-oriented design learn object-oriented design (Blank (Blank et al.et al. 2005, 2005, Moritz & Blank 2005)Moritz & Blank 2005)

Research has shown that an ITS that Research has shown that an ITS that adapts to accurate student knowledge adapts to accurate student knowledge enhances learning enhances learning (Corbett (Corbett et al.et al. 2000) 2000)

Corbett’s ITS focuses on adaptive problem Corbett’s ITS focuses on adaptive problem sequencing rather than adaptive feedbacksequencing rather than adaptive feedback

Our ITS focuses on adaptive feedbackOur ITS focuses on adaptive feedback

AuthorsAuthors System System ContextContext

Consider Consider historyhistory

Diagnose Diagnose ConceptConcept

Pre-Pre-requisitesrequisites

Real Real TimeTime

Murray (1998)Murray (1998) Desktop Desktop AssociateAssociate

skillsskills √√VanLehn et al.VanLehn et al.(2001, 2005)(2001, 2005)

Solve physics Solve physics problemsproblems

rules, not rules, not conceptsconcepts √√

Butz et al. Butz et al. (2004)(2004)

C++ C++ programming programming √√ No No

evaluationevaluation

Millan et al.Millan et al.(2002, 2005)(2002, 2005)

CAT for mathCAT for math √√ √√ Post-Post-processprocess

Reye(1996, Reye(1996, 1998, 2004)1998, 2004)

Theoretical Theoretical

analysisanalysis √√ √√

Wei&Blank Wei&Blank (2006,2007)(2006,2007)

OO Design OO Design (UML)(UML) √√ √√ √√ √√

Student Model in Wei & Blank (2006,2007)compared with other BN Student Models

Layers of Student Layers of Student KnowledgeKnowledge

(Self 1994)(Self 1994) Domain knowledge layerDomain knowledge layer explain all explain all vocabularyvocabulary for discussing or solving for discussing or solving

problems problems

Reasoning knowledge layerReasoning knowledge layer contain reasoning relationships between propositions contain reasoning relationships between propositions

in domain knowledge in domain knowledge

Monitoring knowledge layerMonitoring knowledge layer specify how to solve a problem using reasoning specify how to solve a problem using reasoning

knowledge and domain knowledge knowledge and domain knowledge

Reflective knowledge layerReflective knowledge layer specify appropriate strategies students should have specify appropriate strategies students should have

in a learning environment in a learning environment

Common Problems with Student Common Problems with Student Models Models

Do not consider relationship between Do not consider relationship between individual concepts individual concepts

Do not represent layered knowledge (Self Do not represent layered knowledge (Self 1994)1994)

Do not simulate students’ knowledge history Do not simulate students’ knowledge history Separate the inferred students’ knowledge Separate the inferred students’ knowledge

from closed- and open-ended exercisesfrom closed- and open-ended exercises Do not consider students’ cognitive strategies Do not consider students’ cognitive strategies

including general and domain-specificincluding general and domain-specific Bayesian student models require exponential Bayesian student models require exponential

updating time and hence cannot provide real-updating time and hence cannot provide real-time tutoring adaptive to individual student’s time tutoring adaptive to individual student’s learning statelearning state

Presentation OutlinePresentation Outline

PublicationPublication BackgroundBackground ResearchResearch questionsquestions MethodologyMethodology Evaluation Evaluation Conclusions and future workConclusions and future work

Research Questions (1 of 2)Research Questions (1 of 2) Can this student model provide information for Can this student model provide information for

pedagogical decisions? pedagogical decisions?

How should this student model represent a How should this student model represent a student’s current knowledge state and the student’s current knowledge state and the student’s knowledge structure? student’s knowledge structure?

How will the student model track students’ How will the student model track students’ knowledge state over time? Under this research knowledge state over time? Under this research question there are two sub questions:question there are two sub questions:

Would tracking a history of students’ knowledge state be Would tracking a history of students’ knowledge state be useful for pedagogical decisions?useful for pedagogical decisions?

Can a history be maintained efficiently enough to be Can a history be maintained efficiently enough to be responsive in real-time?responsive in real-time?

Research Questions (2 of 2)Research Questions (2 of 2)

How to synthesize information from How to synthesize information from two different sources, open-ended two different sources, open-ended problem solving (object-oriented class problem solving (object-oriented class diagram design) and closed-ended diagram design) and closed-ended exercises (multiple choice quizzes or exercises (multiple choice quizzes or drag-and-drop exercises)?drag-and-drop exercises)?

What cognitive strategies should the What cognitive strategies should the student model consider and how to student model consider and how to consider them?consider them?

Presentation OutlinePresentation Outline

PublicationPublication BackgroundBackground Research questionsResearch questions MethodologyMethodology Evaluation Evaluation Conclusions and future workConclusions and future work

Three Layered ArchitectureThree Layered Architecture

• CM recognizes cognitive strategies that a student is using

•HM simulates students’ hierarchical knowledge in a history

•PDM simulates current students’ hierarchical knowledge

actor

actor_object

object

object_class

class

class_attribute

attribute

attribute_constructor

constructor

doubleint

numeric datatype

datatype

string

datatype_variable

variable

variable_parameter

parameter

variable_returntype

returntype

pass in only

class_method

method

method_constructor

class_constructor

object_constructor

method_parameter

variable_attribute

object_attribute

object_method

double_int

int_string

double_string

method_returntype

datatype_returntype

attribute_method

attribute_parameter

actor_method

A is prerequisite of B A B

Curriculum Information NetworkCurriculum Information Network

Two kinds of concepts Two kinds of concepts

UniqueUnique concept, such as attribute or concept, such as attribute or parameterparameter

RelationshipRelationship concepts, such as concepts, such as attribute_parameterattribute_parameter

Relationships emerge because of student’s Relationships emerge because of student’s confusions between conceptsconfusions between concepts

E.g., student defines E.g., student defines movieTitlemovieTitle as a as a parameter when he has already defined parameter when he has already defined movieTitlemovieTitle as an attribute as an attribute

Prerequisite relationshipsPrerequisite relationships

Prerequisite is relationship between concepts:Prerequisite is relationship between concepts: The concepts a learner needs to understand The concepts a learner needs to understand

before understanding a conceptbefore understanding a concept E.g., one needs to understand int and double E.g., one needs to understand int and double

in order to understand numericDatatypein order to understand numericDatatype

Relationship concepts are prerequisites of Relationship concepts are prerequisites of unique concepts and vice versaunique concepts and vice versa

E.g., class_constructor -> constructorE.g., class_constructor -> constructor Understanding constructor doesn’t imply Understanding constructor doesn’t imply

understanding of class, just how to define a understanding of class, just how to define a constructor for a classconstructor for a class

Connecting Knowledge with Connecting Knowledge with PerformancePerformance

Student action unit and knowledge unit Student action unit and knowledge unit make a pair(make a pair(KUKU,,AUAU)) Infer understanding of a concept (KU) Infer understanding of a concept (KU)

from a student solution step (AU)from a student solution step (AU) Action unit (AU): Action unit (AU):

A single action or step in a student’s A single action or step in a student’s solutionsolution

E.g., add an attribute to a classE.g., add an attribute to a class Knowledge unit (KU) – concept a student Knowledge unit (KU) – concept a student

need to learnneed to learn KU directly causes a student action unitKU directly causes a student action unit KU is a concept in Curriculum Information KU is a concept in Curriculum Information

Network (CIN)Network (CIN)

au

ku

……

au

ku

d-prereq(ku)1 d-prereq(ku)2d-prereq(ku)N

Atomic Bayesian Network (ABN)

Noisy-andgeneralizeslogical-and

Students must understand all direct prerequisites of the concept ku in order to understand ku

How to generate an ABNHow to generate an ABN

Student model generates an ABN in Student model generates an ABN in response to a student solution stepresponse to a student solution step

First, define the structure of an ABN, First, define the structure of an ABN, i.e., the causal relationship between i.e., the causal relationship between KU and AU, and the direct-KU and AU, and the direct-prerequisites of KUprerequisites of KU

Second, determine conditional Second, determine conditional probability tables for this ABNprobability tables for this ABN

…

au

ku

d-p(ku)1

d-p(ku)2

d-p(ku)N

…

au

ku

d-p(ku)1

d-p(ku)2

d-p(ku)N

0

0

0

0

0

1

1

1

1

1

Atomic Dynamic Bayesian Network (ADBN) for HM layer

How to generate an ADBNHow to generate an ADBN

Student model generates an ADBN in Student model generates an ADBN in response to a student solution stepresponse to a student solution step

First, look for the ABN in response to First, look for the ABN in response to previous student solution step previous student solution step

Second, generate an ABN in response Second, generate an ABN in response to current student solution stepto current student solution step

Third, determine conditional Third, determine conditional probability tables for the ADBNprobability tables for the ADBN

Concrete ExampleConcrete Example

Student defined Student defined movieTitle movieTitle as a as a parameter for method parameter for method displayMovieTitledisplayMovieTitle after she has already defined after she has already defined movieTitlemovieTitle as an attribute to a class as an attribute to a class TicketMachineTicketMachine

EE determines that EE determines that movieTitle movieTitle should should not be a parameter not be a parameter

SM determines that the center concept SM determines that the center concept of an ABN is of an ABN is attribute_parameterattribute_parameter, and, and finds all direct prerequisites, finds all direct prerequisites, attributeattribute and and parameterparameter, from CIN , from CIN

Concrete ExampleConcrete Example

attributeattribute’s prior can be found from the database ’s prior can be found from the database parameterparameter’s prior is 0.5, students’ knowledge ’s prior is 0.5, students’ knowledge

state is assessed based on the difference state is assessed based on the difference between prior and posterior probabilities between prior and posterior probabilities (VanLehn (VanLehn et al.et al. 1998, Millán & Pérez-de-la-Cruz 1998, Millán & Pérez-de-la-Cruz 2002)2002)

SM determines: SM determines: student has good understanding of student has good understanding of classclass, , attribute,attribute,

methodsmethods, and , and parameterparameter but low understanding of but low understanding of attribute_parameterattribute_parameter

the tutoring need is: explanation of the tutoring need is: explanation of attribute_parameterattribute_parameter

Concrete ExampleConcrete Examplefeedbackfeedback

““Since you have added Since you have added movieTitlemovieTitle as as an attribute to the class an attribute to the class TicketMachineTicketMachine, , you shouldn’t also make it a parameter you shouldn’t also make it a parameter to the method to the method displayMovieTitledisplayMovieTitle. To . To decide whether movieTitle should be decide whether movieTitle should be an attribute or a parameter, an attribute or a parameter, remember: attributes are accessible remember: attributes are accessible anywhere within the scope of a class, anywhere within the scope of a class, while parameters are accessible only while parameters are accessible only within the scope of a method”within the scope of a method”

Presentation OutlinePresentation Outline

PublicationPublication BackgroundBackground Research questionsResearch questions MethodologyMethodology EvaluationEvaluation Conclusions and future workConclusions and future work

Evaluation of ABNs with simulated Evaluation of ABNs with simulated studentsstudents

Hypotheses:Hypotheses: Pre-setting slip and guess values will lead Pre-setting slip and guess values will lead

to a reliable student modelto a reliable student model Varying slip and guess values will affect Varying slip and guess values will affect

the accuracy of the student modelthe accuracy of the student model

0.2 0.1 0.01 0.001

0.2 84.76% 81.31% 80.40% 79.08%

0.1 84.43% 93.18% 86.22% 83.51%

0.01 86.01% 88.85% 94.28% 91.15%

0.001 86.04% 88.88% 89.1% 94.31%

slipp, guessp

slipe, guesse

Pre-setting slip and guess values to same (relatively small e.g. Pre-setting slip and guess values to same (relatively small e.g. <=0.1) values produces accuracy of at least 93%, confirming <=0.1) values produces accuracy of at least 93%, confirming the first hypothesisthe first hypothesis

Changing of presetting slip and guess causes accuracy to Changing of presetting slip and guess causes accuracy to

change from 79.1% to 94.3%, confirming the second hypothesischange from 79.1% to 94.3%, confirming the second hypothesis Correct diagnostic rates are higher when Correct diagnostic rates are higher when slipslippp, , guessguesspp and and slipslipee, ,

guessguessee take same value take same value No significant difference when slip and guess take a same small No significant difference when slip and guess take a same small

value (<=0.1) value (<=0.1)

Evaluation of ADBNs with Evaluation of ADBNs with simulated studentssimulated students

Hypotheses:Hypotheses: Pre-setting slip and guess values can lead to Pre-setting slip and guess values can lead to

a reliable student model a reliable student model Modeling learning history with ADBNs will Modeling learning history with ADBNs will

enhance the accuracy of the student modelenhance the accuracy of the student model

ABN ADBN Increase Average increase

0.1 93.18% 97.90% 5.1%

0.01 94.28% 99.04% 5.0%

0.001 94.31% 99.36% 5.4%

5.2%

slipe/p, guesse/p

ABN ADBN

Mean 87.0% 93.4%

SD 8.7% 4.8%

The significant difference between correct The significant difference between correct diagnostic rates using ABNs versus using diagnostic rates using ABNs versus using ADBNs demonstrates that ADBNs enhance the ADBNs demonstrates that ADBNs enhance the accuracy of the student model, confirming the accuracy of the student model, confirming the second hypothesissecond hypothesis

Pre-setting Pre-setting slipslipp/ep/e, , guessguessp/ep/e, to relatively small , to relatively small (e.g. <=0.1) values produces accuracy of at (e.g. <=0.1) values produces accuracy of at least 97%, confirming the first hypothesis. least 97%, confirming the first hypothesis.

The accuracy is not sensitive to change of The accuracy is not sensitive to change of slip and guess values so along as slip and guess values so along as the values the values are relatively small (<=0.1) are relatively small (<=0.1)

0.2 0.1 0.01 0.001

0.2 67.40% 85.71% 83.32% 86.27%

0.1 96.13% 97.66% 97.90% 97.83%

0.01 98.38% 99.04% 98.58% 98.47%

0.001 98.41% 99.09% 99.36% 98.60%

slipt, guesst

slipe/p, guesse/p

Evaluation of student model for Evaluation of student model for CIMEL multimedia with real CIMEL multimedia with real

studentsstudents Hypotheses:Hypotheses:

Pre-setting slip, guess will lead to a reliable Pre-setting slip, guess will lead to a reliable student model student model

Pre-setting slip and guess to various values Pre-setting slip and guess to various values will affect the accuracy of the student modelwill affect the accuracy of the student model

The student model will perform in real-time, The student model will perform in real-time, i.e., it will support responses as students are i.e., it will support responses as students are working on exercisesworking on exercises

Results:Results: The average response time of the student model The average response time of the student model

after a students enters the solution step is 0.24 after a students enters the solution step is 0.24 secondsseconds

Presetting slip and guess with relatively small Presetting slip and guess with relatively small values (<=0.1) can produce accuracy of up to values (<=0.1) can produce accuracy of up to 80.1%. 80.1%.

Changing the presetting slip and guess causes Changing the presetting slip and guess causes the accuracy to change from 71.7% to 80.1%the accuracy to change from 71.7% to 80.1%

slipe guess slipt, guesst slipp, guessp correct prediction rate

0.001 0.4 0.001 0.001 71.7%

0.1 0.04 0.01 0.1 80.1%

Has a higher value of Has a higher value of rr than student than student model by Corbett model by Corbett et al.et al. (2000) (2000)

Our student model with highest correct prediction rate

Student model by Corbett et al.(2000)

Net improvement

r 0.85 (p=0.001) 0.81 (p=0.001) 4.9%

MAE* 0.08 0.15 47% *MAE=mean absolute error proportion in accuracy prediction

(Mean absolute prediction errors/total number of exercises in post-test)

Hypotheses:Hypotheses: Pre-setting slip, guess will lead to a Pre-setting slip, guess will lead to a

reliable student model reliable student model The student model will perform in real-The student model will perform in real-

time, i.e., it will support responses as time, i.e., it will support responses as students are working on problem-solving students are working on problem-solving stepssteps

Results:Results: The average response time of the The average response time of the

student model after a student enters a student model after a student enters a solution step is 0.63 secondssolution step is 0.63 seconds

Evaluation of student model for Evaluation of student model for DesignFirst-ITS with real studentsDesignFirst-ITS with real students

Presetting slip and guess with relatively small Presetting slip and guess with relatively small values (<=0.1) can produce accuracy of up to values (<=0.1) can produce accuracy of up to 81.8% 81.8%

Varying the slip and guess value does not Varying the slip and guess value does not affect the accuracy of the student model, so affect the accuracy of the student model, so long as slip and guess values are relatively long as slip and guess values are relatively small (<=0.1)small (<=0.1)

0.1 0.01 0.001

0.1 79.1% 79.8% 79.1%

0.01 81.8% 81.5% 80.5%

0.001 81.5% 81.5% 81.5%

slipt, guesst

slipe/p, guesse/p

Has a higher value of Has a higher value of rr than student model than student model by Corbett by Corbett et al.et al. (2000) (2000)

Our student model with highest correct prediction rate

The student model of Corbett et al.(2000)

Net improvement

r 0.89 (p=0.001) 0.81 (p=0.001) 9.9%

MAE* 0.09 0.15 40% *MAE=mean absolute error proportion in accuracy prediction

(Mean absolute prediction errors/total number of exercises in post-test)

Evaluation of diagnoses integrationEvaluation of diagnoses integration(multimedia and DesignFirst-ITS) (multimedia and DesignFirst-ITS)

with real studentswith real students

Hypothesis:Hypothesis: Integrating diagnoses from closed-ended Integrating diagnoses from closed-ended

questions will enhance the accuracy of questions will enhance the accuracy of diagnoses of student model for open-diagnoses of student model for open-ended questions ended questions

Accuracy increases 7.7% when adding Accuracy increases 7.7% when adding diagnoses from closed-ended exercises in diagnoses from closed-ended exercises in multimedia, confirming hypothesismultimedia, confirming hypothesis

highest correct prediction rate without prior knowledge from multimedia

correct prediction rate with prior knowledge from multimedia

increase rate

81.8% 88.1% 7.7%

Comparing with non-advanced-Comparing with non-advanced-numerical student models numerical student models

Non-advanced-numerical techniques Non-advanced-numerical techniques include match, include match, summation and summation and subtractionsubtraction

Advanced-numerical techniques Advanced-numerical techniques include Bayesian networks include Bayesian networks

Hypotheses:Hypotheses: Straightforward algorithm will not lead to a Straightforward algorithm will not lead to a

reliable student model reliable student model ADBNs will perform better than “match” ADBNs will perform better than “match”

student modelstudent model

From same set of evidences, ADBNs From same set of evidences, ADBNs perform more than two times better than perform more than two times better than “match” student models“match” student models

“Match” student models’ correct prediction rate

ADBNs’ correct prediction rate

diagnose closed-ended exercises

35.5% 80.1%

diagnose open-ended exercises

28.6% 81.8%

considering prior knowledge state from CIMEL multimedia

41.1% 88.1%

increased correct prediction rate after considering priors

12.5% 6.3%

Presentation OutlinePresentation Outline

PublicationsPublications BackgroundBackground Related ResearchRelated Research Research questions Research questions MethodologyMethodology Evaluation Evaluation ConclusionsConclusions andand futurefuture workwork

ConclusionsConclusions Student models with ADBNs can diagnose Student models with ADBNs can diagnose

student knowledge states accurately in real-student knowledge states accurately in real-timetime

Accuracy of ADBN-based student model is Accuracy of ADBN-based student model is significantly higher than ABN student modelsignificantly higher than ABN student model

Integrating diagnoses from closed- and Integrating diagnoses from closed- and open-ended exercises is an effective way to open-ended exercises is an effective way to increase accuracy of student modelsincrease accuracy of student models

Student models using ADBNs perform much Student models using ADBNs perform much better than the student models that use better than the student models that use straightforward algorithmstraightforward algorithm

Future workFuture work Implement cognitive model to simulate Implement cognitive model to simulate

monitoring knowledge and reflective monitoring knowledge and reflective knowledgeknowledge

Consider students learning gain from Consider students learning gain from reviewing feedbackreviewing feedback how do we determine the conditional probability how do we determine the conditional probability

table for the ADBN so as to simulate the real table for the ADBN so as to simulate the real student learning? student learning?

how do we update the new ADBN? how do we update the new ADBN? how do we convey how do we convey empirical studies with empirical studies with

simulated students and human subjects?simulated students and human subjects? Diagnose students’ learning state in other Diagnose students’ learning state in other

domains, such as object-oriented domains, such as object-oriented programmingprogramming

…

au

ku

p1(ku)2

p1(ku)N

…

au

ku

p1(ku)1

p1(ku)2 0

0

0

0

0

2

2

2

2

2

p1(ku)1

p1(ku)N

ku 1

feedback

An ADBN considers feedback

Contributions (1 of 2)Contributions (1 of 2) A novel way to represent students’ knowledge A novel way to represent students’ knowledge

structure, where both concepts and relationship structure, where both concepts and relationship between concepts are knowledge units that between concepts are knowledge units that students need to learnstudents need to learn

A novel three-layered architecture which can be A novel three-layered architecture which can be standardized in modeling various stratums of standardized in modeling various stratums of students’ knowledgestudents’ knowledge

ABN – a novel Atomic Bayesian network that ABN – a novel Atomic Bayesian network that provides a refined representation of prerequisite provides a refined representation of prerequisite relationships, diagnoses student’s knowledge relationships, diagnoses student’s knowledge structure, and guarantees real-time responsivenessstructure, and guarantees real-time responsiveness

Contributions (2 of 2)Contributions (2 of 2) ADBN – an innovative dynamic Bayesian ADBN – an innovative dynamic Bayesian

network that represents refined network that represents refined representation of prerequisite relationships representation of prerequisite relationships and diagnoses students’ knowledge and diagnoses students’ knowledge structure in real-time considering learning structure in real-time considering learning historyhistory

A unique student model that integrates A unique student model that integrates knowledge from open-ended problem knowledge from open-ended problem solving (object-oriented class diagram solving (object-oriented class diagram design) and closed-ended exercisesdesign) and closed-ended exercises

A general approach for student models that A general approach for student models that help students learn complex problem help students learn complex problem solving in real timesolving in real time

Questions?Questions?