Csse 2014 hmm presentation_ta_ed
-
Upload
saad-chahine -
Category
Data & Analytics
-
view
56 -
download
0
description
Transcript of Csse 2014 hmm presentation_ta_ed
Machine Learning Algorithms: Applications to Educational Data
Saad Chahine, PhD May 26, 2014
Machine Learning “Field of study that gives computers the ability to learn without being explicitly programmed.” (Arthur Samuel, 1959)
“A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.” ( Tom Mitchel (1997), Machine Learning, McGraw Hill | Web Page http://www.cs.cmu.edu/~tom/ )
Two Main Types of Algorithms
Supervised learning:• What we are commonly used to in educational research• We know the data and outputs • We have an idea of the kids of analysis we plan to run (e.g., Linear
Regression)
Unsupervised learning:• Used less often in educational research • We try to find a hidden structure to data that may not be labeled • We have more of an intuition of what we are trying to find (e.g. K-
Means Cluster)
My Interest in Machine Learning
Q: Can we begin to build software programs that learn who we are and can then provide individual learning supports through the use of assessment and feedback?
Markov Models “The future is independent of the past, given the present.” (translation Andrey Markov, 1856-1922)
Limitation – Only takes into account current state and the most recent prior state
Hidden Markov Models - A method(s) of finding a hidden
(latent) structure with a sequential data set
Ghahramani, Z.(2001) An introduction to hidden Markov models and Bayesian networks. International Journal of Pattern Recognition and Artificial Intelligence, 15(1): 9-42.
Piaget Developmental Data
• Visser & SpeekenBrink (2010). depmixS4: An R package for hidden Markov Models. International Journal of Statistical Software, 36(7). http://dare.uva.nl/document/361939
• Data from: Jansen, B.R.J., & van der Maas, H.L.J. (2002). The development of children’s rule use on the balance scale task. Journal of Experimental Child Psychology, 81(4), 383–416.
• Siegler, R.S. (1981). Developmental sequences within and between concepts. Number 46 in Monographs of the Society for Research in Child Development. SRCD.
depmixS4 – Balance Data > data(balance)- 779 participants - Ages from 5-19 years - 4 distance items
CAPP - OSCE• Clinical Assessment for Practice Program
(CAPP)• A program of the College of Physicians
and Surgeons of Nova Scotia (CPSNS)• Objective Structured Clinical Exam (OSCE)• Multiple stations with sequences &
competencies
CAPP OSCE Dataset • 434 observations • 31 participants • 14 stations • 9 measures of competency (Coded
P/F) • 13 different case IDs
My Learning • I conducted the balance data
analysis first• Then I began to examine the OSCE
data• The next slides compare the two as
preliminary analysis
Balance • “Used Age as a
covariate on class membership”
• 3 State Model best• Converged in 77
iterations • loglink = -917.50• AIC = 1867• BIC = 1942
OSCE• Used CASE ID as a
covariate on class membership
• 2 state model best• Converged in 55
iterations • loglink = -1757.81• AIC = 3555• BIC = 3637
Balance • Probabilities at
zero values of the covariates
• 0.001, 0.988, 0.009
OSCE• Probabilities at
zero values of the covariates
• 0.606, 0.394
Balance
OSCE
Balance
OSCE
Balance
OSCE
What's Next • OSCE data did not fit as well as the
Balance data – More Years may help • Learning HMM further and potential
application to performance assessments
• Experiment with different covariates in the datasets
Acknowledgements • Acknowledge the use code by Visser
& SpeekenBrink (2010) in depmixS4 package
• Thank you to CAPP & Bruce Holmes
for the use of OSCE data
Thank You