Ultimul Studiu Arata CA Sunt 20 de Emotii d Baza Combinate

2
Supporting Information Du et al. 10.1073/pnas.1322355111 SI Text Subordinate Relationship. In Fig. 2, we showed six examples of the subordinate relationship between six compound facial expres- sions of emotion and their corresponding basic categories. In Fig. S1, we detail the subordinate relationships for the remaining nine compound categories. As in Fig. 2, this figure illustrates which prototypical action units (AUs) are kept and which need to be eliminated when producing a facial expression of a compound emotion category. The AUs that are eliminated cannot be kept because they cannot be readily produced simultaneously with the others. As a result, some emotion categories are easier to rec- ognize than others; this was evident in the results of Table 4. These coarticulations and how they are generated are essential for disambiguating expressions such as appalled and hate, which, despite of having the same subordinate basic emotion categories, are produced quite distinctly, which facilitates their recognition even if the intensities of their AUs are not known. In Table 1, we also noted that some less common AUs are added to the production of a facial expression of a compound emotion; i.e., some facial expressions of compound emotion categories include AUs that are not present in either of their subordinate categories. In some cases, an additional AU may be elicited to facilitate the recognition of the compound expression. For example, sadly disgusted includes AU 25; this emphasizes disgust while keeping the corners of the mouth pointing down- ward to specify sadness. Fig. S1 also shows a few AU particu- larities of this subject. The AUs identified in gray font are those used by this subject but not generally observed in the majority. The subject shown in this figure, for instance, has a preference for incorporating AU 11, which creates the wrinkle in the nasolabial area. Database. Participants. Subjects were asked to practice each ex- pression before photos were taken. A mirror was placed next to the camera to facilitate practicing while looking forward. During acquisition, the experimenter instructed the subject to look frontally at the camera. This allowed us to obtain fully frontal views of the subjectsfaces. Subjects were given the opportunity to take breaks to avoid production differences between facial expressions due to boredom or muscle fatigue. The picture of the production was taken at the apex of the expression. Before proceeding, this picture was visually evaluated by the experi- menter, e.g., to guarantee that the picture was taken at the apex and that the subject was facing frontally. If the experimenter believed one of these problems occurred, the subject was asked to repeat that expression. Participants were non-FACS coders and were naive to the scientific question or procedure used to analyze the images. Subjects were asked to think of an experience where the facial expression of emotion to be produced was typical. Sample sce- narios were given to participants to facilitate this process. Subjects were not instructed to move any specific facial muscles. Rather, they were asked to express each emotion category as clearly as possible according to the provided definitions and the sample situations where such an expression would be produced. Detection errors. The results in Table 2 are the average pixel dif- ferences between the specified computational models and the manual markings obtained by the authors. Manual markings (as given by a trained eye) have proven invaluable to test face and facial feature detection algorithms. We manually marked the coordinates of 94 fiducial points (i.e., face landmarks) in all of the 5; 060 images in our database. These landmarks were carefully selected to represent the contour of the face (i.e., jaw line) as well as the shape of each of the internal facial components (Fig. 3). Manually marking the same landmark in each face would of course be impossible to do by a humaneach time marking a close-by pixel, not the intended one; this is resolved as follows. Once the 94 fiducials are marked in an image, the best-fitted cubic splines for each of the seven facial components (eyes, eyebrows, nose, mouth, face outline) are computed. Each spline is then resampled by a constant number of equally distant landmarks. We used 8 fiducials to represent each brow, 9 for each eye, 15 for the nose, 30 for the mouth, and 15 for the face outline. The same spline and resampling procedure was used for each of the detection algorithms. The detection errors in Table 2 were then computed as follows. For each of the algorithms used, the average Euclidean distance between the manual and auto- matic detected landmarks defining each facial component (eyes, eyebrows, nose, mouth, face outline) is computed. Then the average of all of the landmark points is calculated. These Eu- clidean distances are given in image pixels. Du et al. www.pnas.org/cgi/content/short/1322355111 1 of 2

description

Aleix Martinez-Universitatea din Ohio

Transcript of Ultimul Studiu Arata CA Sunt 20 de Emotii d Baza Combinate

  • Supporting InformationDu et al. 10.1073/pnas.1322355111SI TextSubordinate Relationship. In Fig. 2, we showed six examples of thesubordinate relationship between six compound facial expres-sions of emotion and their corresponding basic categories. In Fig.S1, we detail the subordinate relationships for the remaining ninecompound categories. As in Fig. 2, this figure illustrates whichprototypical action units (AUs) are kept and which need to beeliminated when producing a facial expression of a compoundemotion category. The AUs that are eliminated cannot be keptbecause they cannot be readily produced simultaneously with theothers. As a result, some emotion categories are easier to rec-ognize than others; this was evident in the results of Table 4.These coarticulations and how they are generated are essential fordisambiguating expressions such as appalled and hate, which,despite of having the same subordinate basic emotion categories,are produced quite distinctly, which facilitates their recognitioneven if the intensities of their AUs are not known.In Table 1, we also noted that some less common AUs are

    added to the production of a facial expression of a compoundemotion; i.e., some facial expressions of compound emotioncategories include AUs that are not present in either of theirsubordinate categories. In some cases, an additional AU may beelicited to facilitate the recognition of the compound expression.For example, sadly disgusted includes AU 25; this emphasizesdisgust while keeping the corners of the mouth pointing down-ward to specify sadness. Fig. S1 also shows a few AU particu-larities of this subject. The AUs identified in gray font are thoseused by this subject but not generally observed in the majority.The subject shown in this figure, for instance, has a preference forincorporating AU 11, which creates the wrinkle in the nasolabialarea.

    Database. Participants. Subjects were asked to practice each ex-pression before photos were taken. A mirror was placed next tothe camera to facilitate practicing while looking forward. Duringacquisition, the experimenter instructed the subject to lookfrontally at the camera. This allowed us to obtain fully frontalviews of the subjects faces. Subjects were given the opportunityto take breaks to avoid production differences between facialexpressions due to boredom or muscle fatigue. The picture of theproduction was taken at the apex of the expression. Before

    proceeding, this picture was visually evaluated by the experi-menter, e.g., to guarantee that the picture was taken at the apexand that the subject was facing frontally. If the experimenterbelieved one of these problems occurred, the subject was askedto repeat that expression.Participants were non-FACS coders and were naive to the

    scientific question or procedure used to analyze the images.Subjects were asked to think of an experience where the facialexpression of emotion to be produced was typical. Sample sce-narios were given to participants to facilitate this process. Subjectswere not instructed to move any specific facial muscles. Rather,they were asked to express each emotion category as clearly aspossible according to the provided definitions and the samplesituations where such an expression would be produced.Detection errors. The results in Table 2 are the average pixel dif-ferences between the specified computational models and themanual markings obtained by the authors. Manual markings (asgiven by a trained eye) have proven invaluable to test face andfacial feature detection algorithms. We manually marked thecoordinates of 94 fiducial points (i.e., face landmarks) in all of the5; 060 images in our database. These landmarks were carefullyselected to represent the contour of the face (i.e., jaw line) aswell as the shape of each of the internal facial components (Fig.3). Manually marking the same landmark in each face would ofcourse be impossible to do by a humaneach time markinga close-by pixel, not the intended one; this is resolved as follows.Once the 94 fiducials are marked in an image, the best-fittedcubic splines for each of the seven facial components (eyes,eyebrows, nose, mouth, face outline) are computed. Each splineis then resampled by a constant number of equally distantlandmarks. We used 8 fiducials to represent each brow, 9 foreach eye, 15 for the nose, 30 for the mouth, and 15 for the faceoutline. The same spline and resampling procedure was used foreach of the detection algorithms. The detection errors in Table 2were then computed as follows. For each of the algorithms used,the average Euclidean distance between the manual and auto-matic detected landmarks defining each facial component (eyes,eyebrows, nose, mouth, face outline) is computed. Then theaverage of all of the landmark points is calculated. These Eu-clidean distances are given in image pixels.

    Du et al. www.pnas.org/cgi/content/short/1322355111 1 of 2

  • Fig. S1. Shown here are the AUs of the remaining nine facial expressions of compound emotion categories. The prototypical AUs of the basic emotions arecombined as shown to produce the compound category. The prototypical and variant AUs of the basic expressions kept to produce the compound emotion aremarked with a bounding box. These relationships define the subordinate classes of each category. The results thus define possible confusion of the compoundemotion categories by their subordinates and vice versa. AUs only used by these subjects (or a small number of subjects, smaller than 20%) are shown in grayfont; these represent particularities of the subject.

    Du et al. www.pnas.org/cgi/content/short/1322355111 2 of 2