Paper.pdf

Computer Vision based emotional and behavioral response analysis in a diverse group of children

Brief

This study is an extended and an automated version of the research done by Dr. William Roberts & Janet Strayer (1997) on‘Facial and Verbal Measures of Children’s Emotions and Empathy

Overview & Method

The relation between children’s emotional expressionsand their reported experience of emotions is relevant tomulticomponent models of both emotion and empathydevelopment

Children with varied Age Groups, Gender and Mentalability participated in a study of their facial and verbalresponses to emotionally evocative video content.

The Children were unobtrusively video recorded whilethey watched these 2 stimulus which were designed toevoke ‘Happy’ and ‘Disgust’ emotions. The facialexpressions, head movements and Eye Blinks of thechildren were auto processed using Cladoop’s Emotion &Behavioral health analysis engine.

Purpose

There have been numerous manual studies done till datethat assess the convergence of children’s facialexpressive responses to emotions they witness in astimuli, however, there is no published research thatextends the responses to Head Movements, Attentionand assessment of the Eye Blink Rate.

The purpose of the present study is to assess theconvergence of children’s verbal, facial expressive, headmovements and eye blink responses to emotions theywitness in a stimuli; to examine Emotional and Behavioraldisorder (autism and ADHD) related differences,academic performance related differences age andgender differences and parental involvement baseddifferences

and to examine each as an index of empathy, orconcordant emotional response to others’ emotions.Emotional experience, as assessed by facial andphysiological measures, and awareness of one’s emotionalexperience, have been said to comprise two major aspectsof emotional development (Lewis & Michalson, 1983).Although facial expressions provide more specificinformation regarding valence and type of emotion than domore general arousal measures (Izard, 1982; Lewis &Michalson, 1983), however it would be interesting tocorrelate the facial expression with the head motion andeye blink related indices

89children

5child groups

4modalities

5 mil+

data points analyzedautism, adhd, normal,

separated parents, orphans

facial expressions, head motion, attention,

eye blinks

3

emotional competence,

hyperactivity, attention

kpis

4age groups

3-6 years, 6-9 years, 9-12 years and 12-15

years

The Studies

The emotional and behavioral variances of children against a different group is studied in detail to understand the role ofemotions and facial behavior that attributes these children to a specific group. The Capture phase of this study focusedon showing two stimulus videos to all participants. The First video has babies and kids playing with their parents whichnaturally should invoke a happy expression, the second video is an excerpt from the popular tv show ‘ Man vs Wild’, thisvideo should ideally invoke a disgust expression. The stimuli have been divided into two segments – Key and Non Key.The Key segment should ideally invoke a higher intensity expression vs the Non Key segment. The followinggroups/comparisons were drawn as part of the analysis of the study

EBDs Academic Performance Age & Gender Parental InvolvementVariance of emotional

traits within ADHD, Autistic and Normal

Children

Variance of emotional traits of High vs low

academic performers

Variance of emotional traits of

various age and gender groups

Variance of emotional traits of children with

varied degree of parental involvement

© 2015 Cladoop, INC – Confidential & Proprietary

DR. NANCY EISENBERGHead - Dept of Child

Psychology - ASU

MUZAMMIL ALAMCoFounder & COO

Cladoop, INC

DR. RICHARD FABESChair - Dept of Family &

Human Development - ASU

36%more head movements

when compared to normal children overall across all

stimuli

‘Disgust’ Stimuli

Results

A. Variance of Emotional Health between ADHD, Autistic and Normal Children

‘Happy’ Stimuli

1. Facial Expressions

ADHD

Autism

Normal

Key Segment

Full VideoKey

SegmentNon Key Segment

24% 27% 24%

17% 36% 12%

14% 20% 13%High Intensity Med Intensity Low Intensity No Facial Expression

ADHD

Autism

Normal

Key Segment

Full VideoKey


5.6% 12.4% 1.9%

0% 0% 0%

4.2% 9.5% 1.2%High Intensity Med Intensity Low Intensity No Facial Expression

ADHD

children

48%higher expression intensity overall for 2 stimuli when

compared to normal children

58%higher expression intensity in ‘Non Key Segment’ for 2 stimuli when compared to

normal children

29%higher expression intensity

in ‘Key Segment’ for 2 stimuli when compared to

normal children

Autism

0Instances of disgust

expression overall for the ‘Disgust’ Stimuli

58%higher expression intensity

in ‘Key Segment’ for Happy Stimuli when compared to

normal children

Overall Higher Expression instances and intensity when compared to normal children

Unusual patterns. No disgust expression and higher intensity during key segment of the happy stimuli

2. Head Motion

ADHD

‘Happy’ Stimuli ‘Disgust’ Stimuli

Key Segment

Non Key SegmentFull Video

Autism

Normal

Key Segment


ADHD


when compared to normal children overall across all

stimuli


when compared to normal children during the key

segment of disgust stimuli

Very high Head Movement Intensity when compared to normal children across all stimuli and segments

Autism

49%Increase in head

movements during the key segment of disgust stimuli

vs non key. Normal children show only 5% difference

High Head movement intensity when compared to normal children ; extremely high movements between Non Key and

Key Segment of Disgust Stimuli

*Yaw, Pitch, Roll, X and Y Displacement. Units in Radians


31%Lesser blinks when

compared to normal children across both

stimuli

Results

A. Variance of Emotional Health between ADHD, Autistic and Normal Children

3. Attention

94.1

88.7

97.0

82.4

81.8

82.7

89.6

83.6

92.9

76.5

73.3

77.3

67.0

74.6

65.2

91.0

91.9

90.8


ADHD

Autism

Normal

Full Video

Key Segment

Non Key Segment

Full Video

Key Segment

Non Key Segment

Full Video

Key Segment

Non Key Segment

5.4

5.3

5.4

6.9

6.1

7.5

9.4

9.5

9.3

10.6

18.0

8.9

6.1

6.7

6.0

9.9

15.3

9.2

ADHD

Autism

Normal

Full Video

Key Segment

Non Key Segment

Full Video

Key Segment

Non Key Segment

Full Video

Key Segment

Non Key Segment

Attention % Attention %


Seconds per Blink Seconds per Blink

ADHD

1%ATTENTION variance for

NORMAL CHILDREN ACROSS BOTH STIMULI

21%ATTENTION variance for

ADHD Children across both stimuli

Autism

8%Lesser attention than normal children in the

disgust stimuli

30%Lesser attention that

normal children in the happy stimuli

Normal Children show equal interest in both the stimuli with only a 1% variance of attention however, ADHD children show very strong attention for the Disgust Stimuli vs the Happy Stimuli, the attention builds up during

the Non Key Segment and reduces slightly in the key segment

Autistic children show attention problems more than ADHD children. An overall 30% lesser attention in the Happy stimuli

vs normal children. Also, interesting to note that the attention drop is just 8% for the disgust stimuli

4. Eye Blinks

ADHD

5%Reduction in Blink rate variance for NORMAL

CHILDREN ACROSS BOTH STIMULI

65%Reduction in Blink rate

variance for ADHD children across both stimuli

Autism

21%Lesser blinks during the

key segment of the disgust stimuli vs the

non key segment

While the overall blink rate for ADHD vs normal are pretty much the same, however the eye blink variance between exhibiting different

expressions is significant.

Autistic children overall blink more when compared to normal children. Also, there is a huge variance in the blink rate during the Disgust

stimuli’s key and non key segments. NO variance for normal children

110%Lesser blinks during the

happy emotion vs the disgust emotion’s key

segments

ADHD affected children overall show more emotional competence, significantly higher head movements, lower attention and higher blink rate when compared to normal children, while children affected with Autismshow some abnormal patterns. Their inability to empathize with the stimuli (as seen in the disgust results) is compensated with unusually high Head Movements (49%), lesser attention drop (vs normal) and a lesser blink rate (21%) during the key segment vs the non key segment.



Results

B. Variance of Emotional Health between High Academic performance vs Low performing children

‘Happy’ Stimuli


A

B

C

Key Segment

Full VideoKey


High Intensity Med Intensity Low Intensity No Facial Expression


2. Head Motion

A


Key Segment


B

C

Key Segment




E

D

Full VideoKey


33.8% 38% 32.7%

11.8% 14.9% 11%

14.5% 22.9% 12.5%

13.9% 26% 11%

9.3% 10.6% 9%

A

B

C

E

D

Key Segment

Full VideoKey


3% 8% 1%

3% 7% 0%

6% 13% 3%

4% 9% 0%

6% 16% 0%

D

E

Overall Positive Valence is key : During the Happy Stimuli, high performers (A&B with 26.4% intensity) had 36% more intensity when compared to the low performers (18.3%), however, when the overall intensity during the entire stimuli is considered, high performers (23%) were a staggering 66% more happier than the low performers (11.6%)

Moderate Negative Valence : During the Disgust Stimuli, high performers had 50% lower disgust expression intensity when compared to the low performers

No Significant head movement variations between High and Low performers

95.5

90.0

98.5

93.6

89.5

95.9

90.0

83.8

93.5

70.9

66.5

73.4

82.9

69.2

90.5

94.597.093.993.292.293.493.495.093.0

67.270.966.367.068.066.8

3. Attention‘Happy’ Stimuli ‘Disgust’ Stimuli

A

Full Video

Key SegmentNon Key Segment




4. Eye Blinks

High Performing children show some distinct traits when compared to low performing children – They are more happier, show moderate negative valence, are

more attentive and have a lesser blink rate variance during attention shift


Results

B. Variance of Emotional Health between High vs Low performing children in Academics

9.411.4

9.07.4

13.76.7

13.318.6

12.410.412.6

9.912.3

19.510.7

8.97.59.8

7.36.67.79.211.9

8.210.1

8.511.2

24.426.0

23.6

B

Full Video


C

Full Video


D

Full Video


E

Full Video

Key Segment

Non Key Segment

A

Full Video


B

Full Video


C

Full Video


D

Full Video


E

Full Video

Key Segment

Non Key Segment

Higher Attention Span during uninteresting scenes : The Happy Stimuli spans well over 2 minutes with a few portions that are much more interesting that the rest. The High performers showed a consistent attention span across the entire stimuli - average 94% when compared to an average 60% attention of the low performers.

Lesser Attention Shift : The Disgust stimuli lasts roughly for 90 seconds and first 2/3rd of the stimuli builds up hype of the viewer. The low performers had a significantly higher attention of 82% in the non key segment (hype build segment) vs 68% during the ‘Key Segment’ which is a striking 19% variance. The high performers had virtually no attention shift – 92% during key segment and 95% for overall – a mere 3% variance

Lower Exertion for being more Attentive : High performers had the same Blink rate during the inattentive segments as well, however, for Low performers (whose attention varied from 67% in the Happy stimuli to 83% in the disgust stimuli), the blink rate doubled during the attempt to be more attentive.


Results

C. Variance of Emotional Health between different Age & Gender groups

‘Happy’ Stimuli


Key Segment High Intensity Med Intensity Low Intensity No Facial Expression


2. Head Motion

F –

3 to 6


Key Segment

Non Key SegmentFull Video Key

SegmentNon Key SegmentFull Video


F – 9 to 12

F – 6 to 9

F – 3 to 6

M – 6 to 9

M – 3 to 6

Key Segment

Full Video Key SegmentNon Key Segment

2.9% 6% 1.2%

9.6% 20.9% 3.3%

7.1% 20% 0%

2% 5.5% 0%

0.8% 2.2% 0%

0% 0% 0%M – 9 to 12

F – 9 to 12

F – 6 to 9

F – 3 to 6

M – 6 to 9

M – 3 to 6

Full Video Key SegmentNon Key Segment

10% 7% 10%

25% 32% 23%

41% 48% 40%

12% 18% 10%

10% 17% 8%

10% 24% 8%M – 9 to 12

F –

6 to 9

F –

9 to 12

M –

3 to 6

M –

6 to 9

M –

9 to 12

Girls are overall more expressive than boys: Girls had an overall 16% expression intensity across both stimuli. On the other hand, boys had only 6% overall expression intensityVirtually opposite Age & Expression Trend : The overall expression intensity trend for Girls Increase with age, however a reverse trend in boys has been noted


No Significant head movement variations across Age and Gender

13.7

18.0

12.3

10.4

14.7

9.8

13.0

9.8

14.4

8.1

10.6

7.6

8.5

19.0

7.4

7.0

8.3

6.8

98.2

97.7

98.5

89.8

81.1

94.6

76.0

37.1

97.6

75.6

67.5

80.0

94.4

92.0

95.8

92.6

96.0

90.7

89.6

84.1

92.6


F

– 3 to 6

Full Video





4. Eye Blinks

Girls are more expressive than boys, Expression display intensity increases with age in girls whereas it’s the opposite in boys. Young males are lesser attentive and the

Eye Blinks reduce as age matures across both genders.


Results

F

– 6 to 9

Full Video


F

– 9 to 12

Full Video


M

– 3 to 6

Full Video


M

– 6 to 9

Full Video

Key Segment

Non Key Segment

97.0

98.6

96.6

91.1

93.3

90.6

75.5

90.8

71.9

70.1

70.2

70.1

92.0

89.1

92.7

91.8

93.2

91.5

8.2

7.0

9.2

10.1

10.8

9.9

1.9

3.4

1.6

9.4

8.1

10.2

8.9

8.7

9.1

15.9

10.9

19.0

M

– 9 to 12

Full Video

Key Segment

Non Key Segment

F

– 3 to 6

Full Video


F

– 6 to 9

Full Video


F

– 9 to 12

Full Video


M

– 3 to 6

Full Video


M

– 6 to 9

Full Video

Key Segment

Non Key Segment

M

– 9 to 12

Full Video

Key Segment

Non Key Segment

C. Variance of Emotional Health between different Age & Gender groups

Young Males are less attentive : Whilst attention is almost the same between genders and age, one notable trend is the difference of attention between. Boys between the age of 3 to 6 years had an average attention of 73% when compared to girls of the same age group whose average attention across both stimuli was 97.6%

Higher Blink rate during more engaging content for girls in the 9-12 year group: The happy stimuli is probably the more engaging of the two stimulus, while it took 2 seconds for a blink during the disgust stimuli, it almost took 6 times more time during the happy stimuli. Lesser blinks on maturity : The blinks per second trend suggests an overall lesser number of blinks as age matures, across both genders


Results

D. Variance of emotional traits of children with varied degree of parental involvement

‘Happy’ Stimuli


Available

Deserted

Full Orphan

Key Segment

Full VideoKey




2. Head Motion

Available


Key Segment


Deserted

FullOrphan

Key Segment



Semi Orphan

Separated

Full VideoKey


21% 26% 20%

13% 21% 11%

6% 18% 4%

16% 5% 19%

14% 21% 12%

Available

Deserted

Full Orphan

Semi Orphan

Separated

Key Segment

Full VideoKey


2% 5% 0%

7% 15% 3%

6% 15% 1%

0% 1% 0%

4% 9% 1%

Separated

Semi Orphan

Children with both parents available are overall more happy: The happy stimuli is basically about Babies and Children playing with their parents. During the Stimuli, children with both parents available had an overall happy expression intensity of 21%, also important to note that during the key segment of the stimuli, the expression intensity grew up to 26%, a mere 4% increase. Also interesting to note that the intensity drop of Deserted children is also pretty high (47%)Orphans are equally expressive but overall less happy: Full Orphans had only 6% overall intensity during the Happy Stimuli, a staggering 3 times lesser when compared to children with parents, however, when it came to the segment of the video, the expression intensity increased three fold to 18% from 6%


No Significant head movement variations


Available

Full Video





4. Eye Blinks

Children with full parental involvement are generally more happier than children with half or no parental involvement. Orphans are equally expressive, however are

lesser happy in general.


Results

Deserted

Full Video


FullOrphan

Full Video


Separated

Full Video


Semi Orphan

Full Video

Key Segment

Non Key Segment

Available

Full Video


Deserted

Full Video


FullOrphan

Full Video


Separated

Full Video


Semi Orphan

Full Video

Key Segment

Non Key Segment

84.1

85.6

83.7

82.2

83.0

82.1

97.4

99.0

97.0

100.0

99.9

100.0

93.2

94.0

93.0

8.6

10.4

8.3

9.6

22.2

8.4

18.5

24.0

15.5

15.3

12.6

16.2

8.0

14.8

7.3

91.5

87.8

93.6

78.3

68.0

84.1

98.4

98.1

98.6

90.4

79.2

96.7

92.9

88.3

95.4

8.9

8.4

9.2

9.1

8.1

9.7

19.9

14.2

25.7

3.4

6.0

2.8

8.1

7.8

8.3

D. Variance of emotional traits of children with varied degree of parental involvement

Children with both parents available are overall more happy: The happy stimuli is basically about Babies and Children playing with their parents. Its interesting to note that orphans paid overall more attention (97%) when compared to childrenwith available parents (84%) which shows a higher degree of inquisitiveness. Another trend to note is that Children whose parents deserted them were probably lesser inquisitive as their attention was almost equal to children with parents (82% vs84%)

Orphans Blink Lesser: The Average seconds per blink for Orphans across both stimuli is 19 Seconds per blink. Children with both parents available take half the time for a blink – reasons unknown

Paper.pdf

Documents

Transcript of Paper.pdf