Post on 24-May-2020
Jecomputerbesturenmetjeogenenjehersenen
JaapHam
19-06-18 1MAMEMpresenta<ve–Duchenne,Luxembourg,September16th,2016
Introduc<ons
• JaapHam
• j.r.c.ham@tue.nl
• SocialPsychology• Persuasivetechnology• SocialRobo<cs
2
“Youshouldnothaveanotherdrink!”
AssociateeditoroftheInterna<onalJournalonSocialRobo<csProgramChairPersuasive2018,April18-19,Waterloo,Canada
JaapHamj.r.c.ham@tue.nlResearchGroup:HumanTechnologyInterac<onEindhovenUniversityofTechnologyPhDstudents:• SofiaFountoukidou:Personaliza<onofPersuasiveTechnology
• AimiGhazali:PersuasiveSocialRobots• HanneSpelt:PsychophysiologyandPT• CaixiaLiu:FacialExpressionIden<fica<on
19-06-18 5
MAMEM:Mul<mediaAuthoringandManagementusingyourEyesandMind(EU--H2020)www.mamem.eu
MAMEMVisie
19-06-18 6
Patient&Clinician Academia&Research Industry&Society
Brainelectricalsignals
Eyemovements
Perspiration&HeartRate
WP2 WP3 WP4 WP5 WP6 WP7
WP1(ProjectManagement)&WP8(Dissemination,Exploitation&FosteringInnovation)
andothers…
Author
andothers…
Share
vvvPandothers…
Manage
PersuasiveDesign
+Ψ =
MAMEMVisie
19-06-18 7
Patient&Clinician Academia&Research Industry&Society
Brainelectricalsignals
Eyemovements
Perspiration&HeartRate
WP2 WP3 WP4 WP5 WP6 WP7
WP1(ProjectManagement)&WP8(Dissemination,Exploitation&FosteringInnovation)
andothers…
Author
andothers…
Share
vvvPandothers…
Manage
PersuasiveDesign
+Ψ =
MAMEMVisie
19-06-18 8
Patient&Clinician Academia&Research Industry&Society
Brainelectricalsignals
Eyemovements
Perspiration&HeartRate
WP2 WP3 WP4 WP5 WP6 WP7
WP1(ProjectManagement)&WP8(Dissemination,Exploitation&FosteringInnovation)
andothers…
Author
andothers…
Share
vvvPandothers…
Manage
PersuasiveDesign
+Ψ =
Consor<um
Eye-tracking&Integra<on
EEGandsensorsop<miza<on
EEGAnalysis&interac<onparadigms
Eye-gazeanalysis&novelinterfaces
19-06-18 9
Psychology&persuasivedesign
Spinalcordinjury,trials&evalua<on
Neuro-musculardisorders,trials&
evalua<on
Parkinsondisease,trials&evalua<on
VanEindhoventotTelAviv
19-06-18 MAMEMReview,Luxembourg,September16th,2016 10
• MAMEMbouwen– Technisch– Gebruikersinbeeldbrengen– S<mulerentotgebruik
• Veldtest1:Gebruikersinhetlab• MAMEMaanpassen
– Socialeinclusie– S<mulerentotsocialeinclusie
• Veldtest2:Bijgebruikersthuis
• MAMEMbouwen– Technisch– Gebruikersinbeeldbrengen– S<mulerentotgebruik
• Veldtest1:Gebruikersinhetlab• MAMEMaanpassen
– Socialeinclusie– S<mulerentotsocialeinclusie
• Veldtest2:Bijgebruikersthuis
19-06-18 14
GazeTheWebbrowser
19-06-18 15
Devicesforsignalcapturing&recording
19-06-18 16
Eye-tracking• Heavy-weight(SMIREDnScien<fic)
• Light-weight(MyGazeAssis<ve-2eyetracker
EEGrecording• Heavy-weight(EBNBePlusLTMEEG
amplifier)
• Light-weight(EPOCEEGheadset)
19-06-18 16
GalvanicSkinResponse
HeartRate Shimmer3GSR+
MAMEMReview,Luxembourg,September16th,2016
WP2
• C/C++interfacesforLabSteamingLayerbuiltupontheSDKsofeachdevice
MAMEMReview,Luxembourg,September16th,2016 17
Eyetrackersetup/calibra<on GSRsensorsetup(nocalibra<onrequired)
Headsetsetup+Ohmetercheck
Somwareforconnec<ngtoMiddleware
WP2
Systemarchitecture&implementa<on
18
• Abstractdescrip<on
• Requirementforflexibility&extensibility
• Serveasamiddlewarefornovelmul<-modalinterfaces
• Fourarchitecturallayers
• Modulesineachlayer
• Input/outputinterfacesforeachmodule
• Relyingonexis<ngframeworks
WP4
Translateeye-gazesignalsintocontrols
19-06-18 19
eyeGUI
GazeTheWeb
• AdoptablelayoutdefinedbyXMLfiles
• Visualelementsforgaze-basedinterac<on
• Ready-madeinterac<oncomponentsthatcanbeeasilyre-used
• Browser-basedframeworkofferingalltypicalopera<onsthrougheyes
• ReliesontheeyeGUIlibrary
• Serveascontainerformanydifferentapplica<ons
WP3
TranslateEEGsignalsintocontrols
19-06-18 MAMEMReview,Luxembourg,September16th,2016 20
Features Classifier Time(s) Accuracy(%)
PWelch
LinearSVM 5.98 86.43
RBFSVM 5.95 87.14
MLR 6.05 87.86
LDA 5.78 70.71
PYAR
LinearSVM 5.72 87.14
RBFSVM 5.78 87.86
MLR 5.6 88.57
LDA 5.65 79.29
DWT
LinearSVM 5.86 89.29
RBFSVM 5.86 90.71
MLR 5.66 88.57
LDA 5.66 87.86
CSP
LinearSVM 6.99 83.57
RBFSVM 7.09 83.57
MLR 5.69 85
LDA 5.81 85.71
CSPwithFB
LinearSVM 5.63 84.29
RBFSVM 7.08 84.29
MLR 5.69 85.71
LDA 6.36 89.29
SubjectID Op]mal Default
S001 99.99 98.55
S002 99.13 87.82
S003 47.82 34.78
S004 97.82 77.17
S005 26.08 30.43
S006 83.47 86.08
S007 98.26 60
S008 42.02 31.88
S009 99.99 99.99
S010 96.52 92.17
S011 98.26 98.26
MeanAcc. 80.85 72.47
SSVEPs SMRs
ErrPs
• ReliablesetupofEEGsensorinfrastructure
• In-houseexperimentalprotocols
• ReplicateSoTAresults
• Focusinon-lineandlow-costBCIs
WP3
HowtocombineET&EEG
19-06-18 MAMEMReview,Luxembourg,September16th,2016 21
• CopewiththeMidasTouchproblemineye-trackinginterfacesusingtheSenso-Motoric-Rhythms(EEG)
• Facilitateerror-correc<onineye-trackingintefacesbyperformingundousingError-relatedpoten<als(EEG)
WP3
• MAMEMbouwen– Technisch– Gebruikersinbeeldbrengen– S<mulerentotgebruik
• Veldtest1:Gebruikersinhetlab• MAMEMaanpassen
– Socialeinclusie– S<mulerentotsocialeinclusie
• Veldtest2:Bijgebruikersthuis
Wiezijndegebruikers?
19-06-18 23
Exampleofreq.Descrip]onRelevantDimension
Compa<blewithdifferentageorcogni<veprofiles
personalvariables,e.g.,age,individualworkspaces
Personaliza<on/adapta<on
Enableusingacomputerinavarietyofdifferentbodyposi<ons
Unifiedrequiredac<onsfromthesystem
Performance
Trainingphaseshouldprovide‘fun’fortheuser
Persuasivedesign(seeWP5)Emo<ons/Mo<va<on
ShouldworkwithMicrosomwindowsinterfaceswithothersomwareandhardware
Interoperability
Todis<nguishbetweenaninten<onalanduninten<onalmovement
easeofopera<onanduserexperience
Usability
Tremorgeneratedartefacts(4-6Hz)anddyskinesiageneratedartefactsinPDpa<entsshouldbetakenintoaccount
Specifictoeyemovementsandbrainwaves(i.e.,reclinicalcohorts)
Physiology
• Literatuur,focusgroupsenvragenlijsten• Qualita<eveandsta<s<scheanalyses• Usersbehoemesengewoontes• Requirementsop5dimensies
WP6
Task 1: User Profiles & Personas
Ø Spinal cord injury disorder Ø Neuromuscular disorders Ø Parkinson’s Disorder
19-06-18 24
ModelusersusingPersonas
19-06-18 MAMEMReview,Luxembourg,September16th,2016 25
• Basedonrequirements
• Userneedsandcomputerhabits
• Medicalcondi<onandpsychologicalstatus
• Makeuserprofiles• Buildpersonasforeachcohort
• Demographic• Medicalinfo• Computeruserinfo• Goals&avtudes
WP5
Usagescenarios
19-06-18 MAMEMReview,Luxembourg,September16th,2016 26
• Basictasks
• Intendedfortrainingandfamiliarizingwiththesystem
• Sophis<catedtasks
• Intendedforbecomingmoresociallyac<ve
WP6
• MAMEMbouwen– Technisch– Gebruikersinbeeldbrengen– S]mulerentotgebruik
• Veldtest1:Gebruikersinhetlab• MAMEMaanpassen
– Socialeinclusie– S<mulerentotsocialeinclusie
• Veldtest2:Bijgebruikersthuis
Stimuleren
19-06-18 WP5-Usermodellingandinterfacedesign,Luxemburg,September2016 28
1. User acceptance and
training engagement
2. Keep using to increase social
participation
HowtoinfluencetheacceptanceanduseofMAMEMtechnologyforthedifferenttargetgroups?
19-06-18 29
PersuasiveDesign
19-06-18 MAMEMReview,Luxembourg,September16th,2016 30
• Interven<onMappingFramework
• Sixstages
• Determinestheinterven<ons
WP5
• Changeobjec<ve
• Determinant
• Persuasivestrategy
Changeobjec]ve
Determinant Persuasivestrategies(examples)
Chooserealis<cgoalsandappropriatetasks
Effortexpectancy:Beliefthatgoalsyouchoosearedo-able,appropriateandselectedforyou.
Sugges]on:letthesystemmakesugges<onsandletusersmakechoiceguidedbywhatthesystemprovidesasgoodop<onTailoring:makesurethatofferedop<onsarerelevanttotheuserprofile
Gamesomteleren
31
Gamesbedenken
19-06-18 32
b)Ac<oncardupdates
• Removeanytaskexplana<ons;• Provideinstruc<onsinameaningfulway
19-06-18 33
Zoekdetovenaar
19-06-18 34
Zoekdetovenaar
19-06-18 35
Nogmeerbeinvloeding:Scores
19-06-18 36
PlayormCommunica<on
19-06-18 37
• MAMEMbouwen– Technisch– Gebruikersinbeeldbrengen– S<mulerentotgebruik
• Veldtest1:Gebruikersinhetlab• MAMEMaanpassen
– Socialeinclusie– S<mulerentotsocialeinclusie
• Veldtest2:Bijgebruikersthuis
Alledeelnemerswerdenbeter
19-06-18 39
SCI,PD,NMDwerdenbeter
19-06-18 40
Able-bodiedwardenbeter
19-06-18 WP5-Usermodellingandinterfacedesign,Florence,June2017 41
Indegames,maarookin4web-taken
19-06-18 WP5-Usermodellingandinterfacedesign,Florence,June2017 42
Steedssnellertypen
19-06-18 WP5-Usermodellingandinterfacedesign,Florence,June2017 43
• MAMEMbouwen– Technisch– Gebruikersinbeeldbrengen– S<mulerentotgebruik
• Veldtest1:Gebruikersinhetlab• MAMEMaanpassen
– Socialeinclusie– S<mulerentotsocialeinclusie
• Veldtest2:Bijgebruikersthuis
Usageinnon-controlledenvironments
LaboratoryEnvironment
Heavy-weightinfrastructure
Trainingprocesses
PhaseI Home
Environment
Light-weightinfrastructure
Monitoron-lineac<vity
PhaseII
19-06-18 MAMEMReview,Luxembourg,September16th,2016 45
WP6
• MAMEMbouwen– Technisch– Gebruikersinbeeldbrengen– S<mulerentotgebruik
• Veldtest1:Gebruikersinhetlab• MAMEMaanpassen
– Socialeinclusie– S<mulerentotsocialeinclusie
• Veldtest2:Bijgebruikersthuis
Socialeinclusie
19-06-18 47
• Methodologysynthesizingdigitalindicatorswithsocialinclusionareas
• Par<cipa<onandsocial
capital
• Empowermentandwellbeing
• Educa<onandemployment
WP7
19-06-18 48
Computermonitoredac<vitypreandpostMAMEMuseQues<onnairefilledinbysamplewithdisabilityQues<onnairefilledinbyrespec<vecaretakersQues<onnairebenchmarkedagainsthealthysampleAudiodiariestocap<vateusagemomentumandinsightsaboutMAMEMop<miza<on
Toolsforquan<fyingsocialintegra<onWP7
• MAMEMbouwen– Technisch– Gebruikersinbeeldbrengen– S<mulerentotgebruik
• Veldtest1:Gebruikersinhetlab• MAMEMaanpassen
– Socialeinclusie– S]mulerentotsocialeinclusie
• Veldtest2:Bijgebruikersthuis
MAMEMDashboard/Homepage
19-06-18 50
S<mulereninGazeTheWeb
19-06-18 51
• MAMEMbouwen– Technisch– Gebruikersinbeeldbrengen– S<mulerentotgebruik
• Veldtest1:Gebruikersinhetlab• MAMEMaanpassen
– Socialeinclusie– S<mulerentotsocialeinclusie
• Veldtest2:Bijgebruikersthuis
19-06-18 53
19-06-18 WP5-Usermodellingandinterfacedesign,Florence,June2017 54
19-06-18 55
19-06-18 56
19-06-18 WP5-Usermodellingandinterfacedesign,Florence,June2017 57
19-06-18 WP5-Usermodellingandinterfacedesign,Florence,June2017 58
19-06-18 WP5-Usermodellingandinterfacedesign,Florence,June2017 59
19-06-18 WP5-Usermodellingandinterfacedesign,Florence,June2017 60
19-06-18 WP5-Usermodellingandinterfacedesign,Florence,June2017 61
19-06-18 WP5-Usermodellingandinterfacedesign,Florence,June2017 62
19-06-18 WP5-Usermodellingandinterfacedesign,Florence,June2017 63
19-06-18 WP5-Usermodellingandinterfacedesign,Florence,June2017 64
MAMEMDashboard/Homepage
19-06-18 65
• MAMEMbouwen– Technisch– Gebruikersinbeeldbrengen– S<mulerentotgebruik
• Veldtest1:Gebruikersinhetlab• MAMEMaanpassen
– Socialeinclusie– S<mulerentotsocialeinclusie
• Veldtest2:Bijgebruikersthuis
Jecomputerbesturenmetjeogenenjehersenen
JaapHam
19-06-18 67MAMEMpresenta<ve–Duchenne,Luxembourg,September16th,2016