Social and Emotional Intelligence in AI and Agents fileComputers in Human Behavior, 37, pp. 94-100....
Transcript of Social and Emotional Intelligence in AI and Agents fileComputers in Human Behavior, 37, pp. 94-100....
![Page 1: Social and Emotional Intelligence in AI and Agents fileComputers in Human Behavior, 37, pp. 94-100. Sensing Camera Physiological measurement (PPG) Camera Facial expressions Scene Understanding](https://reader030.fdocuments.in/reader030/viewer/2022041205/5d57306a88c99370428b7c18/html5/thumbnails/1.jpg)
![Page 2: Social and Emotional Intelligence in AI and Agents fileComputers in Human Behavior, 37, pp. 94-100. Sensing Camera Physiological measurement (PPG) Camera Facial expressions Scene Understanding](https://reader030.fdocuments.in/reader030/viewer/2022041205/5d57306a88c99370428b7c18/html5/thumbnails/2.jpg)
Daniel McDuff
Social and Emotional Intelligence in AI and Agents
![Page 3: Social and Emotional Intelligence in AI and Agents fileComputers in Human Behavior, 37, pp. 94-100. Sensing Camera Physiological measurement (PPG) Camera Facial expressions Scene Understanding](https://reader030.fdocuments.in/reader030/viewer/2022041205/5d57306a88c99370428b7c18/html5/thumbnails/3.jpg)
Applications
Personal Assistance
Education
Transportation
Gaming
Health and Care
Marketing
Governance
![Page 4: Social and Emotional Intelligence in AI and Agents fileComputers in Human Behavior, 37, pp. 94-100. Sensing Camera Physiological measurement (PPG) Camera Facial expressions Scene Understanding](https://reader030.fdocuments.in/reader030/viewer/2022041205/5d57306a88c99370428b7c18/html5/thumbnails/4.jpg)
Applications
Personal Assistance
Education
Transportation
Gaming
Health and Care
Marketing
Governance
![Page 5: Social and Emotional Intelligence in AI and Agents fileComputers in Human Behavior, 37, pp. 94-100. Sensing Camera Physiological measurement (PPG) Camera Facial expressions Scene Understanding](https://reader030.fdocuments.in/reader030/viewer/2022041205/5d57306a88c99370428b7c18/html5/thumbnails/5.jpg)
For an assistant to truly become valuable to the user, it must be natural to interact with and engaging.
• T Bickmore and J Cassell. 2001. Relational agents: a model and implementation of building user trust. Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 396-403. ACM.
• J Gratch, N Wang, J Gerten, E Fast, and R Duffy. 2007. Creating rapport with virtual agents. International Conference on Intelligent Virtual Agents, pp. 125-138. Springer.
• GM Lucas, J Gratch, A King, and LP Morency. 2014. It’s only a computer: virtual humans increase willingness to disclose. Computers in Human Behavior, 37, pp. 94-100.
![Page 6: Social and Emotional Intelligence in AI and Agents fileComputers in Human Behavior, 37, pp. 94-100. Sensing Camera Physiological measurement (PPG) Camera Facial expressions Scene Understanding](https://reader030.fdocuments.in/reader030/viewer/2022041205/5d57306a88c99370428b7c18/html5/thumbnails/6.jpg)
For an assistant to truly become valuable to the user, it must be natural to interact with and engaging.
• T Bickmore and J Cassell. 2001. Relational agents: a model and implementation of building user trust. Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 396-403. ACM.
• J Gratch, N Wang, J Gerten, E Fast, and R Duffy. 2007. Creating rapport with virtual agents. International Conference on Intelligent Virtual Agents, pp. 125-138. Springer.
• GM Lucas, J Gratch, A King, and LP Morency. 2014. It’s only a computer: virtual humans increase willingness to disclose. Computers in Human Behavior, 37, pp. 94-100.
![Page 7: Social and Emotional Intelligence in AI and Agents fileComputers in Human Behavior, 37, pp. 94-100. Sensing Camera Physiological measurement (PPG) Camera Facial expressions Scene Understanding](https://reader030.fdocuments.in/reader030/viewer/2022041205/5d57306a88c99370428b7c18/html5/thumbnails/7.jpg)
Sensing Camera
Physiological measurement (PPG)
Camera
Facial expressions
Scene Understanding
Cellphone
Personal Agent
Activities + Experience Sampling
Microphone
Speech-to-Text and Sentiment
Voice prosody
![Page 8: Social and Emotional Intelligence in AI and Agents fileComputers in Human Behavior, 37, pp. 94-100. Sensing Camera Physiological measurement (PPG) Camera Facial expressions Scene Understanding](https://reader030.fdocuments.in/reader030/viewer/2022041205/5d57306a88c99370428b7c18/html5/thumbnails/8.jpg)
Example
PSI
Physiology (HR, HRV, Respiration)
Facial Actions and Expressions
Speech Sentiment
Voice Tone/Prosody
Scene Understanding + Object Detection
Webcam and Microphone
![Page 9: Social and Emotional Intelligence in AI and Agents fileComputers in Human Behavior, 37, pp. 94-100. Sensing Camera Physiological measurement (PPG) Camera Facial expressions Scene Understanding](https://reader030.fdocuments.in/reader030/viewer/2022041205/5d57306a88c99370428b7c18/html5/thumbnails/9.jpg)
Multimodal
Combining modalities consistently improves recognition performance and can help generalize affect recognition.
Longitudinal
Emotions are short-lived and moods are longer, detecting an emotion without knowledge of a persons typical behavior is hard.
Personalized
Interpersonal variability is large, often larger than cultural or other demographic variability.
![Page 10: Social and Emotional Intelligence in AI and Agents fileComputers in Human Behavior, 37, pp. 94-100. Sensing Camera Physiological measurement (PPG) Camera Facial expressions Scene Understanding](https://reader030.fdocuments.in/reader030/viewer/2022041205/5d57306a88c99370428b7c18/html5/thumbnails/10.jpg)
age gender culture
years men women migration
po
siti
ve e
mo
tio
n
po
siti
ve e
mo
tio
n
po
siti
ve e
mo
tio
n
![Page 11: Social and Emotional Intelligence in AI and Agents fileComputers in Human Behavior, 37, pp. 94-100. Sensing Camera Physiological measurement (PPG) Camera Facial expressions Scene Understanding](https://reader030.fdocuments.in/reader030/viewer/2022041205/5d57306a88c99370428b7c18/html5/thumbnails/11.jpg)
Learning Across Datasets
Anonymized/Aggregated
Large-scale Meta Data
Dataset A Dataset B Dataset C