Eyebrow Shape Analysis by Using a Modified Functional Curve Procrustes

Distance

Yishi Wang, Cuixian Chen, Midori Albert, Yaw Chang, Karl Ricanek

University of North Carolina Wilmington

{wangy, chenc, albertm, changy, ricanekk}@uncw.edu

Abstract

To tackle the problem of automatic recognition of hu-

man eyebrow, a novel approach for shape analysis based

on frontal face images is proposed in this paper. First, eye-

brow curves are acquired by fitting cubic splines based on

landmark points. Next, we propose to use a modified func-

tional curve procrustes distance to measure the similarities

among the cubic splines, and finally a multidimensional

scaling method is adopted to evaluate the effectiveness of

the distance. This work extends previous work in analyzing

the eyebrow for both human and machine recognition by

providing a framework based on shape contours. Further

this work demonstrates the effectiveness of eyebrow shape

for discrimination when teamed with the appropriate metric

distance.

1. Introduction

As the most common facial hairs among men and

women, eyebrows can convey both subtle and bold expres-

sions such as fear, anger, surprise, contempt, happiness,

sadness, etc. However, the human eyebrow is an often over-

looked facial component thought to not to have much value

in for automatic recognition. In the area of identity sciences

including biometrics, where the major interests lie in iris,

ear, palm, fingerprint, and the face as a whole, there have

been very few studies focused on the eyebrow for recogni-

tion and/or soft-biometrics. There has been some interest in

the periocular recognition and soft-biometrics, which may

or may not include the eyebrow. It is this inconsistency of

definition for the periocular region, which has led to this

study.

In a forensic context, faces of suspects captured in

surveillance photographs may be partially covered, such as

by masks or sunglasses. An interesting question is: when

occlusions exist, based on an exposed part, such as eye-

brows, would it be possible to use eyebrow as a reliable

tool to identify a person?

Figure 1. Example of importance of Eyebrow in face recognition.

1.1. Prior work

As indicated in Fig 1, It is a challenge to examine the

magnitudes of difference in the eyebrow between different

individuals as well as among different images of the same

individual. The role of eyebrows in face recognition has

been studied in [10], where the work revealed that the eye-

brow was far more important than eyes for human recog-

nition. However, in [10] only human perception was dis-

cussed, rather than using biometric modeling. Bharadwaj et

al. [1] demonstrated that better automatic recognition was

possible with the periocular region if the eyebrow was in-

cluded. Since the focus of this work was on the texture

from the periocular region, it is unclear whether shape was

a factor in this work. In [4], eyebrow region was manually

segmented, and features were extracted and calculated ac-

cording to three categories: global shape feature, local area

feature, and critical point feature. Classifications were then

conducted based on the three feature categories. These fea-

tures, especially the global shape feature, maybe subjective

and not general enough to be finely quantified. Li et al. [8]

studied an automatic human eyebrow recognition system

via fast template matching and Fourier spectrum distance,

and concluded that eyebrow can serve as an independent

biometric for human recognition.

1.2. Contribution of work

In this work, we study the shape of the human eyebrows

by using a modified functional curve procrustes distance to

1

Figure 2. Framework for eyebrow classification.

develop a eyebrow classification system. The system may

have a great potential for person identification or verifica-

tion. Our major research questions are: (1) how do we rep-

resent the shape of eyebrows quantitatively and effectively,

(2) how can we measure the distance between different eye-

brows, and most importantly, (3) how can we present the

difference in an efficient and meaningful way?

To answer all these questions, we design multiple ex-

periments to discover whether or not eyebrow measures are

unique enough to be a good identifier. Through systematic

evaluation, theoretical and experimental results suggest that

our novel functional curve procrustes distance is an effec-

tive metric distance. The proposed framework is shown in

Figure 2. First, the eyebrows are labeled by ten landmarks

that best describe the general boundaries. Then we fit those

top five points by using cubic spline. Hereafter we have

a set of functions with both ending points known. In this

work, we only consider the top boundaries of left eyebrows,

focusing on the curve comparison. This approach can easily

be extended to address the shape comparison of the entire

eyebrows.

The next challenge is how to compare different eyebrow

curve shapes. In the literature, there have been numerous re-

search on comparing/contrasting shape of objects and pla-

nar curves, such as Hausdorff distance, Frechet distance,

procrustes distance [5], inner distance and etc. In this work,

we propose the modified functional curve procrustes dis-

tance because of its simplicity for calculation and effective-

ness for distance representation. From the previous steps,

with n eyebrows, we end up with n(n − 1)/2 distances. A

natural question is how can we present these distances in

an effective way, i.e., can we project the distances into a

lower dimensional space? We use multidimensional scaling

to reduce the dimension of the distances.

This contribution can be summarized as: (1) We develop

a descriptor of the landmarks shown in Figure 3 and their

location for the quantitative analysis. (2) In our preprocess-

ing step, we use cubic spline to fit the landmarks and ob-

tain a smooth curve. (3) We propose a modified functional

Figure 3. Landmarking scheme for left eyebrow and example of

landmarking.

curve shape procrustes distance to measure the differences

of eyebrow curves, to overcome the difficulties caused by

the relative flat curvatures. (4) To evaluate our proposed

approach systematically, first we manually select a set of

28 Caucasian female eyebrows from the UIUC-PAL dataset

for automatic eyebrow classification. Next, we compare the

results from the proposed approach with human perception

results. Finally, the proposed approach is evaluated on 437

Caucasian eyebrows from PAL-dataset. Our experiment on

PAL dataset suggests that our proposed approach to au-

tomatic eyebrow classification achieves higher recognition

rates than humans.

The organization of this paper is laid out as follows: Sec-

tion 2 presents landmarking scheme and cubic-spline curve

fitting. Section 3 introduces the properties of the new func-

tional curve shape procrustes distance to measure the dis-

tance between functional curves. Section 4 presents the

techniques of Multidimensional Scaling (MDS). The exper-

iment results are presented in Section 5, and conclusions are

drawn in final section of this paper.

2. Eyebrow shapes

In our preliminary study, we find out that it is very

difficult to determine the boundaries of human eyebrows.

Some people have very bushy eyebrows, while it is com-

mon for the seniors to have sparse ones. Therefore, we

develop a descriptor of the landmarks shown in Fig 3 and

their location for the quantitative shape analysis with the

top left eyebrows of n subjects, (xi, yi), for i = 1, · · · , n,

where xi = (xi,0, ..., xi,4)T be the x coordinates and yi =

(yi,0, ..., yi,4)T be the y coordinates of the labeled landmark

points 0-4. Due to the nature of the human eyebrows, it is

easier to consider the general eyebrow shapes as intersec-

tions of two continuous curves.

Next, we use statistical curve fitting to approximate the

boundaries of eyebrows. The general idea of curve fitting

with given landmark points is to find a continuous function

that passes through those landmark points while being rel-

atively smooth. The way to achieve such a goal is to find a

twice differentiable function such that among all functions

f(x), the following objective function is minimized:

L(f, λ) =5

∑

i=1

(yi − f(xi))2 + λ

∫

(f′′

(x))2dx.

2

According to Green and Silverman (1994) [6], under certain

conditions, the above loss function has a unique minimizer:

a natural cubic spline that passes through (xi,j , yi,j) for

each fixed i and j = 0, · · · , 4. In our preliminary study, we

test on multiple curve fitting techniques and find out that the

cubic spline is effective and the resulting curves are good

approximation of the true eyebrow curves. With a group

of curves, our next goal is to define the distances between

these curves, such that the distances can be used to measure

the similarities and differences of the shapes among the eye-

brows.

Let F = {fi : fi is the top-left eyebrow spline function

of the ith subject} be a set of continuous function both end-

ing points provided. Without loss of generality, we may as-

sume that the slopes of the straight lines connecting the two

ending points are zero and the Euclidean distance is one.

3. Distances of planar functional curves

We propose to use Modified Functional Curve Procrustes

Distance, to measure the similarities between different con-

tinuous curves. The definitions were proposed as following:

Definition 1. (Equally spaced heights with zero ending

points). Let f : [0, 1] 7→ R be a continuous function

with f(0) = f(1) = 0, and let 0 = t1 < ... <tn = 1 be n equally spaced points on [0, 1] , then F :=(f(t1), f(t2), · · · , f(tn))T is called equally spaced heights

of f on [0, 1].

Hereafter, we assume that all curves considered are zero

at ending points of their domain. Therefore, when the

equally spaced heights with zero ending points of a function

is a constant, it indicates that the constant is zero. Gener-

ally, the equally spaced heights may be positive or negative,

depending on the curve f and where the sampled points are

located.

Definition 2. (Equally spaced distance (ESD) between two

continuous functions) Let g and h be two continuous curves

on closed domains, and let U = (u1, ..., un)T and V =(v1, ..., vn)T be the equally spaced heights with zero ending

points of g and h respectively. When ||U ||2 · ||V ||2 > 0, the

equally spaced distance between curves g and h is

Dn(U, V ) :=

√

1 −< U, V >2

||U ||22 · ||V ||22, (1)

where < ·, · > is the inner product of two vectors, and || · ||2is the L2 norm.

Let f1, f2 and f3 be continuous functions on [0, 1] with

both ending points equal to zero, let X = (x1, ..., xn)T ,

Y = (y1, ..., yn)T and Z = (z1, ..., zn)T be their corre-

sponding equally spaced heights, with ||X|| · ||Y || · ||Z|| >

0, it is obvious that Dn(f1, f1) = 0, and D(f1, f2) =D(f2, f1) ≥ 0. If D(f1, f2) = 0, it indicates that∑n

i=1 |xiyi| = ||x||||y||. By Cauchy-Schwartz inequality,

we have X = rY , where r is a scalar. As to the triangle

inequality, it can be proved that

Theorem 3. With the previous notations ,

Dn(f1, f2) ≤ Dn(f1, f3) + Dn(f2, f3).

Therefore Dn is a pseudo metric.

Notice that for (1), if we replace U and V by function gand h respectively, change the inner product and L2 norm

of vectors to the ones for functions, we have

D(g, h) :=

√

1 −< g, h >2

||g||22 · ||h||22

, (2)

where

< g, h >=

∫ 1

0

g(x)f(x)dx, and ||g||22 =

∫ 1

0

g2(x)dx.

In fact, the distance D(g, h) in (2) is the limit of Dn(g, h)in (1) as n approaches to infinity. It can be verified that

D(g, h) is also a pseudo metric with the following theorem.

Theorem 4. With the conditions and notation in Theorem 3,

D(f1, f2) ≤ D(f1, f3) + D(f2, f3).

The distance D(g, h) is similar with the the distance pro-

posed in [7], in which the functions take a complex format

to accommodate both the x and y variables. In this work,

because of the way we register our function, we can only

focus on the y variable, which is exactly the reason why

all we need is the vertical heights used in the definition of

Dn in (1). This simplification should reduce the compu-

tation time since only y coordinates are involved. Similar

with the distance proposed in [7], all the above mentioned

distances do not allow constant functions or vectors. How-

ever, in many real world application, such a limitation can

be a big hurdle. There is hardly a function that is constant,

but there are many planar curves that are very similar with

constant curves. For example, in our eyebrow classification

experiment, some eyebrows demonstrate relatively flat cur-

vature. In order to overcome this difficulty, we propose to

add a constant parameter k > 0 to U , V , g and h in (1) and

(2), and therefor the revised distance of (1) and (2) are:

Dkn(U, V ) :=

√

1 −< U + k, V + k >2

||U + k||22 · ||V + k||22, (3)

which we name as modified discrete procrustes distance,

and

3

Dk(g, h) :=

√

1 −< g + k, h + k >2

||g + k||22 · ||h + k||22, (4)

which we name as modified functional curve procrustes dis-

tance. It can be proved that both (3) and (4) are metric dis-

tance and now it allows for U or V , or g or h to be con-

stant(s).

Let fi : [0, 1] 7→ R for 0 ≤ i ≤ n be n cubic splines (or

let F1, ..., Fn be the corresponding equally spaced heights),

we may calculate the modified functional curve procrustes

in (4) for each pair, and hence there aren(n−1)

2 distances

among all pairs. A n × n matrix S is often used with the

(i, j) element represents the distance between the ith and

jth functions, i.e., si,j = Dk(Fi, Fj). Obviously, S is sym-

metric with diagonal elements equal to zero.

4. Multidimensional scaling

Because of the nature of the matrix S, it is difficult to

have a visual idea on the relationships among all functions,

in terms of the similarities of shapes. The challenge is

then to reduce the dimension of the distances fromn(n−1)

2to two or three dimensions where visual understanding of

the similarities is easier. In this work, we use Multidimen-

sional Scaling (MDS) [11] to reduce the dimension. With

the aforementioned distance matrix Sn×n = (si,j), MDS

searches vectors z1, · · · , zn ∈ Rd to minimize the follow-

ing objective function:

LM (z1, · · · , zn) :=∑

i 6=j

(si,j − ||zi − zj ||2)2. (5)

The major idea of MDS is to use zi, for i ∈ 1, · · · , n as a

lower dimensional representation of the data that preserves

the pairwise distances, si,j , as much as possible. For vari-

ous objective functions for different types of MDS, [2] con-

tains a great amount of information. For (5), essentially we

are looking for the approximation that

s2i,j ≈ ||zi − zj ||

22 = ||zi||

22 + ||zj ||

22 − 2zT

i zj

Since it is the inner product of zi and zj that we are most

interested in, the above expression is equivalent with

−1

2s2

i,j ≈ zTi zj − ||zi||

22/2 − ||zj ||

22/2.

Let A = (− 12s2

i,j) and let B be the centered version of A in

the following sense

B = HAH where H = In − Jn/n,

and Jn is an n × n matrix of one.

Thus, the optimization of objective function in (5) be-

comes an eigenvalue problem. The minimizer of (5) has an

explicit solution by using the largest d eigenvalues of matrix

B and the corresponding eigenvectors. Let λ1, · · · , λd be

the largest eigenvalues of matrix B with associated eigen-

vectors e1, · · · , ed, then

(zT1 , · · · , zT

n )T = (√

λ1e1, · · · ,√

λded).

In the following experiments, we will present the effec-

tiveness of the distance in (4) by using MDS with d = 2.

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.1

0.2

0.3

0.4

0.5

0.6

seq(0, 1, length = 100)

sim

.fit[1

, ]

A

B

C

DE F

Figure 4. Simulated eyebrows: flat (type A), slightly round (type

B), moderately round (type C), highly round (type D), skewed to

the left/arched (type E), and skewed to the right/arched (type F).

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.1

0.2

0.3

0.4

0.5

0.6

seq(0, 1, length = 100)

rea

l.T

L.f

it[i,

]

Figure 5. Top-left eyebrows for 28 manually selected females on

PAL dataset.

5. Experiment

In this section we shall systematically evaluate the ef-

fectiveness of eyebrow classification system based on the

modified functional curve procrustes distance.

5.1. Simulated eyebrows

In order to access the effectiveness of the distances in (4)

as well as for shape classification purpose, six types of sim-

ulated eyebrows are generated: flat (type A), slightly round

(type B), moderately round (type C), highly round (type D),

skewed to the left (type E), and skewed to the right (type F).

Note that the simulated eyebrows are generated from func-

tions of probability density function of Uniform distribu-

tion, Normal distribution and F distribution independently.

4

For each type of eyebrow, we generate four variations by

adding normal random noises. Thus we have 24 simulated

eyebrow totally. Here the random noises are carefully cho-

sen, so that the general shape is not affected. The shapes of

the simulated eyebrows are in Figure 4. We aim to reveal

which categories, A-F, our real eyebrows fall into or close

to. It is worth to emphasize that in our human perception

experiment, we only classify the real eyebrows into three

categories: flat (type A), round (type B-D), angled (type E,

F).

Figure 6. PAL Sample images: flat, round, round, and angled. Im-

age ID from left to right: 9, 17, 27, 13. We conduct two sets of

human perceptions to classify the eyebrows into three categories:

flat, round, and angled.

5.2. Face and eyebrow database

The Productivity Aging Laboratory (PAL) face database

[9] is selected for this experiment due to its quality of im-

ages. Only the frontal images with neutral facial expression

are selected for our eyebrow classification algorithm. (See

Figure 6 for sample images.)

First, for our eyebrow classification and human percep-

tion tasks, we manually select 28 Caucasian female images,

with age range 18-86, to control ethnicity and gender ef-

fects, as shown in Figure 5. In our preliminary study, we de-

velop a descriptor of the landmarks assigned and their loca-

tions for quantitative analysis, which is sufficient to capture

the details of the shape of eyebrow, yet not redundant. Then

each eyebrow is manually labeled with landmarks assigned

as points along the eyebrow. Next, for our 28 females im-

ages, all top left eyebrows are fitted by using cubic splines

discussed in section 2. Finally, the proposed approach is

test on 437 Caucasian eyebrows from PAL-dataset.

5.3. Experiment results

Compared with the curves from the top left eyebrows

of our 28 subjects, the simulated ones cover most of the

variations that we have from the real ones. In the following

experiments, we use the distance defined (4) with k = 0.5.

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.1

0.2

0.3

0.4

0

0

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.1

0.2

0.3

0.4

0

0

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.1

0.2

0.3

0.4

0

0

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.1

0.2

0.3

0.4

0

0

Simulated eyebrows

Flat eyebrows

Round eyebrows

Angled eyebrows

Figure 7. Eyebrow classification results based on 3D MDS of eye-

brow shapes by the Modified Functional Curve Shape Procrustes

Distance.

Image ID

green 7, 9, 11, 21, 24, 26, 15, 10, and 18

black 1-6, 16, 17, 19, 20, 23, 25, and 27

blue 8, 12, 13, 14, 22, and 28.Table 1. Summary to eyebrow classification results on 28 selected

female eyebrows.

First, we aim to explore the simulated eyebrows only.

Figure 8 clearly shows that the four symmetric types (A, B,

C, D) stay in the center, while type E and type F are on the

two sides. The layout of type A through type D is according

to the round levels of the simulated eyebrows, with type A

and type D on the two far ends and type B and type C in

the middle. Moreover, the variations of each eyebrow type

stay much closer to the ones of its own type than the ones

of the other types. This figure validates the effectiveness of

the distance in (4) for the simulated eyebrows.

Next, when we consider the 28 selected eyebrows with

the simulated ones from Figure 8, it suggests that the eye-

brow shapes are within the triangle of type A, Type C and

Type F. The most flat (type A) eyebrows are from images

with ID: 9 and 21, the most round eyebrow is from im-

age with ID 17, and the 28th and 13th images are the most

skewed ones. However, no eyebrow is as skewed to the

right as the 13th skewed to the left. Eyebrows from subject

number 9, 3, 27, 17, 13 etc. are among the most extreme

ones. Moreover, from the face and eyebrow sample images,

it can be clearly seen that the eyebrow shapes change con-

tinuously and smoothly, from bottom (flat) to top (round),

from left to right (change of skewness). The bottom left

corner image shows the shapes of simulated eyebrows, il-

lustrating modes of variation in shapes.

For human perception purpose, three professional and

well-trained anthropology scholars involve in our human

5

Figure 8. Distribution of the 24 simulated eyebrows and 28 female top-left sample eyebrows from PAL dataset. Simulated eyebrows include

Types of A to F with four variations in each type by adding normal random noise.

perception study. We conduct two sets of human percep-

tions to classify the eyebrows into 3 categories as: flat,

round, and angled. In the first experiment, three scholars

are required to classify the global eyebrow features with all

facial features. In the second experiment, three scholars are

required to classify the fitting curves from landmarks as lo-

cal eyebrow features without any facial features. In the first

experiment, there is only 16 out of 28 eyebrow shapes are

classified by all three scholars consistently, which gives us a

57.14% 3-way agreement. While in the second experiment,

there is a significant improvement of 3-way agreement of 22

out of 28, that is 78.57%. From this perception study, we

find out that it is subjective for humans to classify shapes

even for well trained professional anthropologists and even

when the facial features are removed.

We then consider to build an automatic classifier to clas-

sify real eyebrows into three categories: flat, round, and an-

gles. We consider the Libsvm as our 3-class classifier [3].

Hereafter, we adopt the first three coordinates from MDS

to preserve more information. We use four simulated flat

eyebrows (type A) as our class 1 training set; four varia-

tions from type C as class 2, and four simulated angled eye-

brows (type F) for class 3. Due to lack of ground truth of

the shape classification from the human perception study, to

efficiently illustrate our classification outputs, we compare

the classification outputs of the real eyebrow samples to the

simulated data in Fig 7. The red curves are the simulated

eyebrow curves, while the green, black, and blue curves are

the real eyebrows that are classified as flat, round and an-

gled, respectively. Table 1 shows the specific classification

results. As we compare the classification results from Fig-

ure 7 with the 2D projection in Figure 8, the results are very

consistent and promising.

Finally, we evaluate our proposed approach on the PAL-

dataset with 437 Caucasian eyebrows. The results are

shown in Figure 9, which is similar to Figure 8.However,

it is interesting to find out the majority of eyebrows stay in

between round and skewed to the left, which is the same

finding from our perception study. From both Figure 7 and

Figure 9, similar eyebrows are clustered together while the

extraneous ones are around the boundaries, such as ID 362,

304, 355, 208, 247, etc. It indicates that our proposed ap-

proach provides an effective metric distances among eye-

brow curves for further classification task.

6. Conclusion and future research direction

In this work, a novel curve shape classification approach

is proposed and applied to the analysis of human eye-

brows, based on a modified functional curve procrustes dis-

tance.Experiment results suggest that, when combined with

MDS, the intrinsic distances between convex 2D shapes

can be efficiently represented with a two dimension space.

6

Figure 9. Distribution of the 24 simulated eyebrows and 437 Caucasian top-left sample eyebrows from PAL dataset.

Further research on this work includes but is not lim-

ited to: combination all portions of eyebrow curves with

length/width, area and color of eyebrows; evaluating the

performance on person identification/verification; investi-

gating the gender/age differences with eyebrows.

7. Acknowledgment

This work has been partially funded by Army Research

Lab in conjunction with the Center for Advanced Studies in

Identity Sciences and the FBI Biometric Center of Excel-

lence.

The authors would like to thank SAMSI for its workshop

series. The authors would also like to thank Christopher

Maier, Jessica Daley, Olivia Smith for assistance in eyebrow

landmarking and human perceptions.

References

[1] S. Bharadwaj, H. Bhatt, M. Vatsa, and R. Singh. Periocular

biometrics: When iris recognition fails. In BTAS 2010: 1-6.

[2] I. Borg and P. Groenen. Modern Multidimensional Scaling:

theory and applications. New York: Springer-Verlag, 2005.

[3] C.-C. Chang and C.-J. Lin. LIBSVM: a library for

support vector machines, 2001. Software available at

http://www.csie.ntu.edu.tw/ cjlin/libsvm.

[4] Y. Dong and D. Woodard. Eyebrow shape-based features for

biometric recognition and gender classification: A feasibil-

ity study. In International Joint Conference on Biometrics,

2011.

[5] I. L. Dryden and K. V. Mardia. Statistical Shape Analysis.

John Wiley and Sons, 1998.

[6] P. J. Green and B. W. Silverman. Nonparametric Regres-

sion and Generalized Linear Models: A roughness penalty

approach. Chapman and Hall, 1994.

[7] R. Larsen. Functional 2d procrustes shape analysis. In

SCIA’05, pages 205–213, 2005.

[8] Y. Li, H. Li, and Z. Cai. Human eyebrow recognition in

the matching-recognizing framework. Computer Vision and

Image Understanding, 117, 2013.

[9] M. Minear and D. C. Park. A lifespan database of adult facial

stimuli. Behavior Research Methods, Instruments, & Com-

puters, 36:630–633, 2004.

[10] J. Sadr, I. Jarudi, and P. Sinha. The role of eyebrows in face

recognition. Perception, 32(3):285–293, 2003.

[11] W. S. Torgerson. Multidimensional scaling: I. theory and

method. Psychometrika, 17(4):401–419, 1952.

7