Attention detection in driver simulator project #4 INPG : L. Bonnaud, A. Caplier UCL : D. trevisan,...
-
date post
19-Dec-2015 -
Category
Documents
-
view
213 -
download
0
Transcript of Attention detection in driver simulator project #4 INPG : L. Bonnaud, A. Caplier UCL : D. trevisan,...
![Page 1: Attention detection in driver simulator project #4 INPG : L. Bonnaud, A. Caplier UCL : D. trevisan, B. Macq Participants : A. Benoit, G. Chanel, P. Ngo,](https://reader030.fdocuments.in/reader030/viewer/2022032800/56649d2a5503460f949ffaf3/html5/thumbnails/1.jpg)
Attention detection in driver simulator
project #4
INPG : L. Bonnaud, A. CaplierUCL : D. trevisan, B. Macq
Participants : A. Benoit, G. Chanel, P. Ngo,V. Levacic, C. Thillou
Guest star : L. Lawson
![Page 2: Attention detection in driver simulator project #4 INPG : L. Bonnaud, A. Caplier UCL : D. trevisan, B. Macq Participants : A. Benoit, G. Chanel, P. Ngo,](https://reader030.fdocuments.in/reader030/viewer/2022032800/56649d2a5503460f949ffaf3/html5/thumbnails/2.jpg)
04/18/23 2
Objectives
• Multimodal detection of user state– Hypo vigilance– Stress level– In the context of driving simulator
• Alarm => multimodal feedback to the user
• Test of OpenInterfaceas an integration platform
![Page 3: Attention detection in driver simulator project #4 INPG : L. Bonnaud, A. Caplier UCL : D. trevisan, B. Macq Participants : A. Benoit, G. Chanel, P. Ngo,](https://reader030.fdocuments.in/reader030/viewer/2022032800/56649d2a5503460f949ffaf3/html5/thumbnails/3.jpg)
04/18/23 3
Signals used foruser state assessment
Video(camera)
Physiological (biopac)
Head Eyes Mouth ECG GSR
Hypovigilance ? ?
Stress ?
![Page 4: Attention detection in driver simulator project #4 INPG : L. Bonnaud, A. Caplier UCL : D. trevisan, B. Macq Participants : A. Benoit, G. Chanel, P. Ngo,](https://reader030.fdocuments.in/reader030/viewer/2022032800/56649d2a5503460f949ffaf3/html5/thumbnails/4.jpg)
04/18/23 4
General architecture
Significative rotation ?
High frequency ?
Both eyes closed ?
Video
Mouth opening detector
ECG analysis
Head rotation estimation
Eyes blinks detector
GSR analysis
GSRECG
Yawn ?
Fusion (redundancy, equivalence)
High heart rate variation ?
High GSR ?
Local Peak ?
Fission (redundancy, equivalence)
Sonic feedback
Graphical feedback
Haptic feedback
Screen loudspeaker Vibrating wheel
Dialog controller
![Page 5: Attention detection in driver simulator project #4 INPG : L. Bonnaud, A. Caplier UCL : D. trevisan, B. Macq Participants : A. Benoit, G. Chanel, P. Ngo,](https://reader030.fdocuments.in/reader030/viewer/2022032800/56649d2a5503460f949ffaf3/html5/thumbnails/5.jpg)
04/18/23 5
Face motions analysis
![Page 6: Attention detection in driver simulator project #4 INPG : L. Bonnaud, A. Caplier UCL : D. trevisan, B. Macq Participants : A. Benoit, G. Chanel, P. Ngo,](https://reader030.fdocuments.in/reader030/viewer/2022032800/56649d2a5503460f949ffaf3/html5/thumbnails/6.jpg)
04/18/23 6
Physiological signals “database”
• 3 subjects : thanks to participants!• 3 situations :
– rest / relaxation– Stress stimuli while reading a newspaper
• Hand clapping• Light in eyes• Answering simple question• Telephone call, “your dead” (killer game)
– Playing with the driving simulator (difficult tracks)
![Page 7: Attention detection in driver simulator project #4 INPG : L. Bonnaud, A. Caplier UCL : D. trevisan, B. Macq Participants : A. Benoit, G. Chanel, P. Ngo,](https://reader030.fdocuments.in/reader030/viewer/2022032800/56649d2a5503460f949ffaf3/html5/thumbnails/7.jpg)
04/18/23 7
GSR signal
![Page 8: Attention detection in driver simulator project #4 INPG : L. Bonnaud, A. Caplier UCL : D. trevisan, B. Macq Participants : A. Benoit, G. Chanel, P. Ngo,](https://reader030.fdocuments.in/reader030/viewer/2022032800/56649d2a5503460f949ffaf3/html5/thumbnails/8.jpg)
04/18/23 8
GSR signal
![Page 9: Attention detection in driver simulator project #4 INPG : L. Bonnaud, A. Caplier UCL : D. trevisan, B. Macq Participants : A. Benoit, G. Chanel, P. Ngo,](https://reader030.fdocuments.in/reader030/viewer/2022032800/56649d2a5503460f949ffaf3/html5/thumbnails/9.jpg)
04/18/23 9
GSR signal
![Page 10: Attention detection in driver simulator project #4 INPG : L. Bonnaud, A. Caplier UCL : D. trevisan, B. Macq Participants : A. Benoit, G. Chanel, P. Ngo,](https://reader030.fdocuments.in/reader030/viewer/2022032800/56649d2a5503460f949ffaf3/html5/thumbnails/10.jpg)
04/18/23 10
ECG signal : preprocessing
![Page 11: Attention detection in driver simulator project #4 INPG : L. Bonnaud, A. Caplier UCL : D. trevisan, B. Macq Participants : A. Benoit, G. Chanel, P. Ngo,](https://reader030.fdocuments.in/reader030/viewer/2022032800/56649d2a5503460f949ffaf3/html5/thumbnails/11.jpg)
04/18/23 11
ECG : stress assessment
![Page 12: Attention detection in driver simulator project #4 INPG : L. Bonnaud, A. Caplier UCL : D. trevisan, B. Macq Participants : A. Benoit, G. Chanel, P. Ngo,](https://reader030.fdocuments.in/reader030/viewer/2022032800/56649d2a5503460f949ffaf3/html5/thumbnails/12.jpg)
04/18/23 12
Simplified architectureof 1st prototype
Significative rotation ?
High frequency ?
Both eyes closed ?
Video
Mouth opening detector
ECG analysis
Head rotation estimation
Eyes blinks detector
GSR analysis
GSRECG
Yawn ?
Fusion (redundancy, equivalence)
High heart rate variation ?
High GSR ?
Local Peak ?
Fission (redundancy, equivalence)
Sonic feedback
Graphical feedback
Haptic feedback
Screen loudspeaker Vibrating wheel
Dialog controller
Network messages (XML)
1st PC, Matlab, Linux
2nd PC, Simulator, Windows
![Page 13: Attention detection in driver simulator project #4 INPG : L. Bonnaud, A. Caplier UCL : D. trevisan, B. Macq Participants : A. Benoit, G. Chanel, P. Ngo,](https://reader030.fdocuments.in/reader030/viewer/2022032800/56649d2a5503460f949ffaf3/html5/thumbnails/13.jpg)
04/18/23 13
Realization of 1st prototype
Significative rotation ?
High frequency ?
Both eyes closed ?
Video
Mouth opening detector
ECG analysis
Head rotation estimation
Eyes blinks detector
GSR analysis
GSRECG
Yawn ?
Fusion (redundancy, equivalence)
High heart rate variation ?
High GSR ?
Local Peak ?
Fission (redundancy, equivalence)
Sonic feedback
Graphical feedback
Haptic feedback
Screen loudspeaker Vibrating wheel
Dialog controller
Network messages (text)
![Page 14: Attention detection in driver simulator project #4 INPG : L. Bonnaud, A. Caplier UCL : D. trevisan, B. Macq Participants : A. Benoit, G. Chanel, P. Ngo,](https://reader030.fdocuments.in/reader030/viewer/2022032800/56649d2a5503460f949ffaf3/html5/thumbnails/14.jpg)
04/18/23 14
Conclusion
• First working prototype (demo available)– Firewire camera Matlab interface under Linux– Modification of the simulator (multi threading, TCP server,
message display)– TCP client in Matlab
• First try with Open Interface integration problem with Matlab integration to be solved by Lionel next week
• Usefulness of physiological signals• Reliable online face motions analysis
Still a lot of work to do !!!