Post on 15-Jan-2017
Experience API and Machine Learning for Patients and Learners
JESSIE CHUANGCLASSROOM AID INC.
VISCA ANALYTICS
XAPI CHINESE COP
About Us
AcrossX VocabularyWiki.visualcatch.org
Background and Motivation (1/2)Fact Mobile Health(mHealth) flourished. (Saxon, 2016.)
150Health60%
Other40%
Background and Motivation (1/2)
Related
Monitoring physiological
value
Health Information(Text, Picture, Video)
FactMobile Health(mHealth) flourished. (Saxon, 2016.)
Mostly focused on providing information and monitoring physiological value. (Vashist, et al, 2014. Saxon, 2016.)
Background and Motivation (1/2)
1. We need to monitor patients daily activities and motions of upper limb. (Subbarao, et al, 1995.)Gap
92% 50 - 70%
Fact
Related
Mostly focused on providing information and monitoring physiological correlation value. (Vashist, et al, 2014. Saxon, 2016.)
Mobile Health(mHealth) flourished. (Saxon, 2016.)
Background and Motivation (1/2)
2. Applications provided for Spinal cord injury patients only taught health information, but lack of monitoring and management.
Solution
1. Develop an intelligent assistance system - SmartChair APP
2. Proposed Context Awareness Suggestion Engine (iCASE)
FactMobile Health(mHealth) flourished. (Saxon, 2016.)
Mostly focused on providing information and monitoring physiological correlation value in Health APPs. (Vashist, et al, 2014. Saxon, 2016.)
Gap
Related
1. The need for patients daily activities and rehabilitation of upper limb motion should be monitored. (Subbarao, et al, 1995.)
Background and Motivation (2/2)
SmartChair APP(Dept Engineering Science, NTU)
Motor power wheelchair(Dept Mechanical Engineering, NTU) Physician, occupational therapist
(Dept Occupational Therapy, NTU)
Spinal cord injury(Taoyuan Potential Development Center, National Taiwan University Hospital)
Data
Prescription
Objective
Identify problems, dynamic correction, improve the system.
System DeveloperUserIntegration
Implement a mHealth APP for Patients with SCI.
Purpose
Objective
Click : 1 Click : 2 Click : 3 Click : 4System DeveloperUserRelated Staff
Implement a mHealth APP for Patients with SCI.
Issue Frequent clicks will result in chronic injuries.
Purpose
Objective
Click : 1 Click : 2System DeveloperUserIntegration
Implement a mHealth APP for Patients with SCI.
Issue Frequent click will result in chronic injuries.
Purpose
Solution
Machine builds context-awarenessfrom user history, prescription, & other contexts and prompts recommendations dynamically.
Objective
Solution
Using Experience API (xAPI) , data can be transferred between different services
System DeveloperUserIntegration
Implement a mHealth APP for Patients with SCI.
Purpose
IssueIssue Collect data from different services
System DeveloperUserIntegration
Objective
System
System’
Collection Analysis
FeedbackRevision
System
System’
Collection Analysis
Feedback
Revision
Stable?Y
N
Implement a mHealth APP for Patients with SCI.
Purpose
System revision is usually inefficient and time-consuming.
Issue
Objective
Solution
Add "Recommended Interface” to the current architecture to get feedback. (A/B testing)
System DeveloperUserIntegration
Implement a mHealth APP for Patients with SCI.
Purpose
System rebuilding is usually inefficient and time-consuming.
Issue
Methodology
Collection
Analysis
Technique ServerLevel
Client Level
The actual usage can not be completely recorded.
Server LogAccess easily to information, but can not collect JavaScript event. (Srivastava, 2000.)
MethodologyTechnique Server
Level
Client Level
Access easily to information, but can not collect JavaScript event. (Srivastava, 2000.)
Server Log
Direct Access
Intermediary Server-
Side
There will exist the problem of grammar incompatibility in migration. (Corbi and Burgos, 2014.)
Leverage Escrow Services to avoid grammar migration issue.
Software / APIs
Collection
Analysis
Methodology
Technique ServerLevel
Client Level
Server Log
Direct Access
Intermediary Server-
Side
xAPI
Since the Experience API (xAPI) is an open standard, so it is used as Intermediary Server.
Collection
Analysis
Access easily to information, but can not collect JavaScript event. (Srivastava, 2000.)There will exist the problem of grammar incompatibility in migration. (Corbi and Burgos, 2014.)
Leverage Escrow Services to avoid grammar migration issue.
MethodologyTechniqu
e
xAPI
Client Level
Through Escrow Services to avoid incompatibility problem with the grammar migration.
Intermediary Server-
SidexAPI
Cross-platform
Use the Activity Streams to record user experience.
Actor(Who)
Verb(How)
Object(What)
Collect and transfer data between heterogeneous platforms through Learning Record Store (LRS).
Context descriptio
n
Parameters for different situations can be recorded as context data.
Collection
Analysis
Data integrity
Methodology
Context-Awarene
ss
Collect user behavior through xAPIDefinitio
n
Categories
Dynamic
a person, place or object. (Dey, et al. 2001.)Static
Actor behaviors (Actor, Verb, Object) (G. Chen and D. Kotz, 2000.)
i.e. user profile, location(G. Chen and D. Kotz, 2000.)
Collection
Analysis
Computing context
Usercontext
Physical context
Timecontext
i.e. network connectivity
i.e. time of a day, week
i.e. lighting, noise level
Methodology
Methods
Statistics
Sequential Pattern
Statistical inference (frequency, average, etc.) is the most popular.
Investigate the probability that when an event appears, another event also appears.
Classify old data, and then predict the future data.
Cluster the data by property similarity.
Analyze data pattern on timeline.Context-Awarene
ss
Collect user behavior through xAPICollection
Analysis
Association Rule
Clustering
Classification
System DeveloperUserIntegration
System Architecture
Filter Model
Context-Awareness
Model
Behavior Model
Expert Knowledge
Cost-Benefit Analysis
iCASE
xAPI
Therapist
SystemDeveloperRecommended
Interface
iCASE systemData Flow
Filter Model
Context-Awareness
Model
Expert Knowledg
e
Cost-Benefit Analysis
iCASE
xAPI
Therapist
SystemDeveloperRecommende
d Interface
iCASE systemData Flow
System BlocksContext-Awareness Model
xAPI
Behavior ModelxAPI
format Mapping
Interface Segmentation
Block Naming
Behavior Library(LRS)
Behavior Model
Context-Awareness Model
xAPI
Behavior ModelxAPI
format Mapping
Block Naming
Behavior Library(LRS)
Behavior Model: Collecting User Data through xAPI
VIPS algorithm (Microsoft, 2003.)
Interface Segmentation
Context-Awareness Model
xAPI
Behavior ModelxAPI
format Mapping
Interface Segmentation
Behavior Library(LRS)
Behavior Model: Collecting User Data through xAPI
DiscomfortRecord
ActivityRecord
RouteRecord
Exercise
ExerciseTime
Block Naming
Context-Awareness Model
xAPI
Behavior Model
Interface Segmentation
Block Naming
Behavior Library(LRS)
Behavior Model: Collecting User Data through xAPI
xAPI Verb xAPI Object
viewedexperiencedmodifiedrecorded
Discomfort RecordActivity RecordExercise…
xAPI format
Mapping
DiscomfortRecord
ActivityRecord
RouteRecord
Exercise
ExerciseTime
xAPI Verb xAPI Object
viewedexperiencedmodifiedrecorded
Discomfort RecordActivity RecordExercise…
Context-Awareness Model
Behavior ModelxAPI
format Mapping
Interface Segmentation
Block Naming
Behavior Model: Collecting User Data through xAPI
Event: { “Actor” :” John Lee”, “Verb” :” recorded”, “Object” :” Discomfort Record” }
Behavior Library(LRS)
…
xAPI
Filter Model Expert Knowledge
Cost-Benefit Analysis
iCASE
xAPI
Therapist
SystemDeveloperRecommended
Interface
iCASE systemData Flow
Behavior Model
Context-Awareness ModelFilter Model
Behavior Model
Context Analysis
TimeContext
UserContext ……
Context Definition
Context-Awareness Model
Context-Awareness
Model
Behavior Model
Context-Awareness
Model
Expert Knowledge
Cost-Benefit Analysis
iCASE
xAPI
Therapist
SystemDeveloperRecommende
d Interface
iCASE systemData Flow
Filter Model
Sort Results Rule filtering engine
Filter Model
Context-Awareness
Model Behavior Model
Filter Model
Behavior Model
xAPIRecommended
Interface
Sort Results Rule filtering engine
Filter Model
Context-Awareness Model
Filter Model + Expert Advice
# System functions
Therapist prescription
135347 Route Distance 5 - 10 km
116170
Frequency of Wheelchair
Repair
At least once a month
99242 Rehabilitation Three times a week
Hash Table
Expert Knowledge Therapist
# System functions Therapist prescription
135347 Route Distance 5 - 10 km
116170 Frequency of Wheelchair Repair
At least once a month
99242 Rehabilitation Three times a week
Sort Results Rule filtering engine
Filter Model
Filter ModelAdjust the weight to improve the bad habits.
Time
Frequency
Therapist prescription(Threshold)
Forcibly removed
Time
FrequencyTherapist prescription
(Upper limit)Forcibly removed
Therapist prescription(Lower limit)
1) User habits2) Increase the weight
Reduce the weight
Expert Knowledge Therapist
Behavior Model
xAPI
Sort Results Rule filtering engine
Filter Model
Context-Awareness Model
L0.5L1
L0Recommend
L2
L3
Login
function1
Index
function1.1
function1.3
function2
function3
function2.1function2.2
function3.1function3.2function3.3
function1.2
Ln : Level n ; n : clicks required to access the function
1) Sequentially outputs.2) Until satisfy the size of
the recommended list.
Recommendation Interface
Recommended Interface
Physiological sensors : blood pressure; blood glucose level; temperature; blood oxygen level; and the signals related to ECG, EEG, and EMG.
Biokinetic sensors : to measure the acceleration and the angular rate of rotation that results from body movements. Ambient sensors : to measure environmental factors such as temperature, humidity, light, and the sound pressure level.
Self-reporting : alarm, habit, discomfort recording, survey, check-list, request help.
Patient-centered “Sensor Network”
XAPI records rich CONTEXT information, which is crucial for medical data.
Serve Humanity ASAPXAPI data are highly structured
in a pre-designed way, can be integrated meaningfully as soon as collected, data can be put to use right away. for human to read, for machine to compute &
respond (less guess), services can talk to each
other & work together in real time !! (If … then … )
If data talk in different languages, we can NOT make sense out of them or use them UNTIL the time and computing power are committed to integrate and interpret them.
Related Works
LRS
Behavior Model
PrescriptionLearning
Plan
iCASE brainxAPI Applicatio
ns
Human-in-the-loop Machine Learning : machine is human’s collaborator.
Food Control for Cancer PatientsTraining and Learning
Development Strategy
Support instructors & learners with workflow and data flow, connect with their brains with effective visualizations.
Put data into use in real time for data-driven actions / automation.
Computer learns from human’s actions to build learner model, adaptive recommendations, and iterate from human’s feedback continuously.
Image credit: LACEproject
Design Thinking w/I xAPI
From content-oriented to experience-oriented design Data + Design = Behavior Engineering Return data to learners first, help them understand their own
data, give them agency and ownership of learning. Process matters, from fix mindset to growth mindset Learner as co-designer in their learning journey
40
Instructor and machine as collaboratorsto help learner navigate through learning journey, but shall give them agency
TakeawaysXAPI is a very effective tool in enabling Apps to serve humanity ASAP, because it connects heterogeneous data immediately.
XAPI is about people working together. xAPI projects are really across domains collaboration.
XAPI is about connecting current technologies, instead of re-inventing wheels.(API’s power)
@classroomaidinc
Jessie@classroomaid.org
Citationjia-Ru Ho, Yun Yen Chuang, Ray-I Chang, “SmartChair APP - Mobile Technologies for Supporting Patients with Spinal Cord Injury,” The 11th E-Learning and Information Technology Symposium, 2016.
Jessie Chuang is co-founder of Classroom Aid Inc., lead of ADL xAPI Chinese Community of Practice, and consultant of Visca – xAPI visual analytics service. She has provided consulting services and courses in OER (Open Educational Resources), mobile learning design, learning standards, educational technology product/solution design and visualization design for educators, researchers and vendors. Recently she is passionate about xAPI implementation design and analysis, data-driven learning design and how analytics & machine learning work in different industries. She often connects ideas from different domains, in her past career in high tech. R&D she had obtained more than 20 patents for new inventions.
Bio.