EXPLOITING PASSIVE POLARIMETRIC IMAGERY FOR REMOTE...
121
EXPLOITING PASSIVE POLARIMETRIC IMAGERY FOR REMOTE SENSING APPLICATIONS BY THILAKAM VIMAL THILAK KRISHNA, B.E., M.S.E.E. A dissertation submitted to the Graduate School in partial fulfillment of the requirements for the degree Doctor of Philosophy, Engineering Specialization in Electrical Engineering New Mexico State University Las Cruces New Mexico May 2008
Transcript of EXPLOITING PASSIVE POLARIMETRIC IMAGERY FOR REMOTE...
EXPLOITING PASSIVE POLARIMETRIC IMAGERY FOR REMOTE
SENSING APPLICATIONS
BY
THILAKAM VIMAL THILAK KRISHNA, B.E., M.S.E.E.
A dissertation submitted to the Graduate School
in partial fulfillment of the requirements
for the degree
Doctor of Philosophy, Engineering
Specialization in Electrical Engineering
New Mexico State University
Las Cruces New Mexico
May 2008
“Exploiting Passive Polarimetric Imagery for Rote Sensing Applications,” a dis-
sertation prepared by Thilakam Vimal Thilak Krishna in partial fulfillment of
the requirements for the degree, Doctor of Philosophy, has been approved and
accepted by the following:
Linda LaceyDean of the Graduate School
Charles D. CreusereChair of the Examining Committee
Date
Committee in charge:
Dr. Charles D. Creusere, Chair
Dr. Robert L. Armstrong
Dr. Phillip De Leon
Dr. David Voelz
ii
DEDICATION
I dedicate this work to my parents and Trish.
iii
ACKNOWLEDGMENTS
I would like to thank my advisor, Chuck Creusere, for his constant encourage-
ment, support, interest, and patience. In particular, I would like to thank him for
sharing his knowledge and insights in signal, image and video processing that has
enriched my research experience at New Mexico State.
I would like to thank Dr. David Voelz, Dr. Phillip De Leon and Dr. Robert
Armstrong for serving on my doctoral committee. I am indebted to Dr. David
Voelz for the time he spent teaching me the basics of optics at the start of this
project. I would like to also thank Dr. De Leon for introducing me to advanced
matrix theory that has been of immense help to me during the course of my
doctoral work.
I am greatly indebted to the support provided by my family and friends during
my doctoral work. I would like to thank the Ashwini and Srivatsan Kandadai and
the Furth family for their friendship as well as the many dinners I enjoyed at their
home. I would like to thank the Humphreys for making me a part of their family.
I really cannot thank them enough for their love, companionship in addition to
inviting me to your family Thanksgiving dinner over the last three years.
I would like to thank my parents and Trish for their constant love and support
during the course of my doctoral studies. I am constantly amazed at the amount
of belief they have in me. This work would have never happened without them.
iv
VITA
February 16, 1978 Born at Madras, Tamil Nadu, India
1995-1999 B.E., S. J. College of Engineering,University of Mysore, India
2000-2002 M.S.E.E., University of Texas at DallasRichardson, Texas
2003-2007 Research Assistant, New Mexico State UniversityLas Cruces, New Mexico
PROFESSIONAL AND HONORARY SOCIETIES
Institute of Electrical and Electronic Engineers (IEEE)
Society of Photo-Optical Instrumentation Engineers (SPIE)
PUBLICATIONS [or Papers Presented]
1. V. Thilak, D. G. Voelz, and C. D. Creusere, Polarization-based index ofrefraction and reflection angle estimation for remote sensing applications,Appl. Opt., vol. 46(30), pp. 7527-7536, Oct. 2007.
2. V. Thilak, C. D. Creusere, and D. G. Voelz, Material classification usingpassive polarimetric imagery, Proc. IEEE Int’l Conf. Image Proc. SanAntonio, TX, vol. IV, September 2007, pp. 121-124.
3. V. Thilak, D. G. Voelz, and C. D. Creusere, Image segmentation from multi-look passive polarimetric imagery, in Polarization Science and Remote Sens-ing III, J. A. Shaw, J. S. Tyo, eds., Proc. SPIE, vol. 6682, San Diego, CA,September 2007, 668206.
4. A. Pamba, V. Thilak, D. G. Voelz, and C. D. Creusere, Estimation of inci-dence and reflection angles from passive polarimetric imagery: extension toout-of-plane scattering, in Polarization Science and Remote Sensing III, J.A. Shaw, J. S. Tyo, eds., Proc. SPIE, vol. 6682, San Diego, CA, September2007, 668200.
v
5. V. Thilak, C. D. Creusere, and D. G. Voelz, Estimating the complex indexof refraction and view angle of an object using multiple polarization mea-surements, in Proc. of the Fortieth Asilomar Conf. on Signals, Systems,and Computers, Pacific Grove, CA, Nov. 2006, pp. 1067-1071.
6. V. Thilak, D. G. Voelz, C. D. Creusere, and S. Damarla, Estimating therefractive index and reflected zenith angle of a target using multiple polar-ization measurements, in Polarization: Measurement, Analysis, and RemoteSensing VII, D. H. Goldstein, D. B. Chenault, eds., Proc. SPIE, vol. 6240,Orlando, FL, May 2006, pp. 35-42.
7. V. Thilak, J. Saini, D. G. Voelz and C. D. Creusere, Pattern recognitionfor passive polarimetric data using nonparametric classifiers, in PolarizationScience and Remote Sensing II, J. A. Shaw, J. S. Tyo, eds., Proc. SPIE.,vol. 5888, pp. 337-344, September 2005.
8. V. Thilak and C. D. Creusere, Tracking of extended size targets in H.264compressed video using the probabilistic data association filter, Proc. EU-SIPCO, , September 2004, pp. 281-285.
9. V. Thilak and A. Nosratinia, Signal Design for Robust Watermarking on ISIChannels, Proc. Asilomar Conf. on Signals, Systems and Computers, vol.2, , Pacific Grove, CA, November 2002, pp. 1210-1214.
10. V. Thilak and A. Nosratinia, Robust Band-limited Watermarking with Trel-lis Coded Modulation, Proc. IEEE Intl Conf. on Image Proc., vol. 2,September 2002, pp. 141-144.
FIELD OF STUDY
Major Field: Electrical Engineering
Signal Processing
vi
ABSTRACT
EXPLOITING PASSIVE POLARIMETRIC IMAGERY FOR REMOTE
SENSING APPLICATIONS
BY
Thilakam Vimal Thilak Krishna, B.E., M.S.E.E.
Doctor of Philosophy
New Mexico State University
Las Cruces, New Mexico, 2007
Dr. Charles D. Creusere, Chair
Polarization is a property of light or electromagnetic radiation that conveys
information about the orientation of the transverse electric and magnetic fields.
The polarization of reflected light complements other electromagnetic radiation
attributes such as intensity, frequency, or spectral characteristics. A passive po-
larization based imaging system records the polarization state of light reflected
by objects that are illuminated with an unpolarized and generally uncontrolled
source. The polarization due to surface reflections from such objects contains in-
formation about the targets that can be exploited in remote sensing applications
vii
such as target detection, target classification, object recognition and shape ex-
traction/recognition. In recent years, there has been renewed interest in the use
of passive polarization information in remote sensing applications.
The goal of our research is to design image processing algorithms for remote
sensing applications by utilizing physics-based models that describe the polar-
ization imparted by optical scattering from an object. In this dissertation, we
present a method to estimate the complex index of refraction and reflection angle
from multiple polarization measurements. This method employs a polarimetric
bidirectional reflectance distribution function (pBRDF) that accounts for polar-
ization due to specular scattering. The parameters of interest are derived by
utilizing a nonlinear least squares estimation algorithm, and computer simulation
results show that the estimation accuracy generally improves with an increasing
number of source position measurements. Furthermore, laboratory results indi-
cate that the proposed method is effective for recovering the reflection angle and
that the estimated index of refraction provides a feature vector that is robust to
the reflection angle.
We also study the use of extracted index of refraction as a feature vector in de-
signing two important image processing applications, namely image segmentation
and material classification so that the resulting systems are largely invariant to il-
lumination source location. This is in contrast to most passive polarization-based
image processing algorithms proposed in the literature that employ quantities
viii
such as Stokes vectors and the degree of polarization and which are not robust
to changes in illumination conditions. The estimated index of refraction, on the
other hand, is invariant to illumination conditions and hence can be used as an
input to image processing algorithms. The proposed estimation framework also is
extended to the case where the position of the observer (camera) moves between
measurements while that of the source remains fixed. Finally, we explore briefly
the topic of parameter estimation for a generalized model that accounts for both
specular and volumetric scattering. A combination of simulation and experimental
results are provided to evaluate the effectiveness of the above methods.
ix
CONTENTS
LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv
1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Contributions of Proposed Work . . . . . . . . . . . . . . . . . . . 5
1.2 Dissertation Outline . . . . . . . . . . . . . . . . . . . . . . . . . 6
2 POLARIZATION . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1 Stokes Vector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2 Mueller Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3 Bidirectional Reflectance Distribution Function . . . . . . . . . . 13
2.3.1 Polarmetric Bidirectional Reflectance Distribution Function 14
2.4 Polarimetric BRDF for the Microfacet Model . . . . . . . . . . . . 16
2.4.1 Microfacet Probability Density Function . . . . . . . . . . 16
2.4.2 Fresnel Reflectance Mueller Matrix . . . . . . . . . . . . . 17
3 PROPOSED PARAMETER ESTIMATION METHODOLOGY . 22
3.1 Feature Vector Creation for Classification . . . . . . . . . . . . . . 26
3.2 Problem Description . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.2.1 Fixed Receiver (Camera) Scenario . . . . . . . . . . . . . . 28
3.2.2 Fixed Illumination Source Scenario . . . . . . . . . . . . . 30
x
3.3 Degree of Polarization for In-Plane Scattering . . . . . . . . . . . 30
3.4 Estimation Approach . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.4.1 Index of Refraction and Reflection Angle Estimation . . . 34
3.4.2 Index of Refraction and Illumination Direction Estimation 36
3.5 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.5.1 Sensitivity Analysis . . . . . . . . . . . . . . . . . . . . . . 38
3.5.2 Results with Laboratory Data . . . . . . . . . . . . . . . . 46
3.5.3 Fixed Camera Scenario . . . . . . . . . . . . . . . . . . . . 47
3.5.4 Fixed Illumination Source/Moving Camera Scenario . . . . 53
3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4 APPLICATIONS OF PROPOSED ESTIMATION APPROACH . 56
4.1 Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
4.1.1 Segmentation Methodology . . . . . . . . . . . . . . . . . 57
4.1.2 c-means Clustering Method . . . . . . . . . . . . . . . . . 59
4.1.3 Silhouette Index . . . . . . . . . . . . . . . . . . . . . . . . 61
4.1.4 Probabilistic Rand Index (PRI) . . . . . . . . . . . . . . . 63
4.2 Material Classification . . . . . . . . . . . . . . . . . . . . . . . . 67
4.2.1 Classification Algorithm . . . . . . . . . . . . . . . . . . . 68
4.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . 68
4.3.1 Segmentation Results . . . . . . . . . . . . . . . . . . . . . 69
4.3.2 Material Classification Results . . . . . . . . . . . . . . . . 76
xi
4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
5 ESTIMATION FRAMEWORK FOR GENERALIZED SCATTER-
ING MODEL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5.1 pBRDF Model for Specular and Volumetric Scattering . . . . . . 83
5.2 Mixing Parameter Estimation . . . . . . . . . . . . . . . . . . . . 85
5.3 Simulation Results and Discussion . . . . . . . . . . . . . . . . . . 86
6 CONCLUSION AND FUTURE WORK . . . . . . . . . . . . . . 89
Appendices
A. NOTE ON THE COMPLEX INDEX OF REFRACTION . . . . . . 94
B. THE LEVENBERG-MARQUARDT ALGORITHM . . . . . . . . . 98
REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
xii
LIST OF TABLES
2.1 Stokes vector sign interpretation. . . . . . . . . . . . . . . . . . . 11
2.2 Stokes vector examples. . . . . . . . . . . . . . . . . . . . . . . . . 12
3.1 Simulation results for a dielectric surface for ”low-noise” scenario. 38
3.2 Simulation results for a dielectric surface for ”high-noise” scenario. 40
3.3 Simulation results for a metallic surface for ”low-noise” scenario. . 42
3.4 Simulation results for a metallic surface for ”high-noise” scenario. 44
3.5 Estimation results for green paint target for the fixed camera scenario. 49
3.6 Estimation results for roughened copper for the fixed camera scenario. 50
3.7 Estimation results for black paint target for the fixed camera scenario. 52
3.8 Estimation results for aluminum target for the fixed camera scenario. 52
3.9 Results for green paint for the fixed source scenario. . . . . . . . . 54
3.10 Results for black paint for the fixed source scenario. . . . . . . . . 55
4.1 PRI scores for the samples used in experiments. . . . . . . . . . . 71
4.2 Index of refraction values computed by segmentation method. . . 75
4.3 Reflection angle estimates for black paint target. . . . . . . . . . . 76
4.4 Prototype index of refraction values used by proposed classifier. . 77
4.5 Classification result using the nearest neighbor classifier. . . . . . 79
5.1 Simulation results for mixing parameter estimation. . . . . . . . . 86
xiii
LIST OF FIGURES
2.1 BRDF Geometry. . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.1 Geometry assumed for the fixed receiver (camera) case. . . . . . . 28
3.2 Geometry assumed for the fixed source scenario. . . . . . . . . . . 29
3.3 DOP plots derived from model and experimental data for green paint. 48
3.4 DOP plots derived from model and experimental data for copper. 50
3.5 DOP plots derived from model and experimental data for black paint. 51
3.6 DOP plots derived from model and experimental data for aluminum. 53
4.1 Clustering-based segmentation method. . . . . . . . . . . . . . . . 58
4.2 PRI computation when there is an ambiguity in boundary location. 66
4.3 Segmentation results for black paint sample (θr = 60◦). . . . . . . 69
4.4 Segmentation results for black paint sample (θr = 60◦). . . . . . . 70
4.5 Segmentation results for green paint sample. . . . . . . . . . . . . 71
4.6 Segmentation results for aluminum sample. . . . . . . . . . . . . . 73
4.7 Scene segmentation results for black paint sample. . . . . . . . . . 74
4.8 Target clusters and their respective centroids. . . . . . . . . . . . 78
4.9 Result obtained by using the nearest neighbor classifier. . . . . . . 80
4.10 Decision boundary for black and green paint targets. . . . . . . . 81
xiv
1 INTRODUCTION
Remote sensing refers to techniques that acquire information about target
objects within a sensed scene without physical, intimate contact with the object.
Remote sensing techniques have become popular in many applications includ-
ing weather forecasting, mine safety, security and defense applications. Remote
sensing systems collect information about a target of interest by recording energy
such as electromagnetic radiation reflected by the target, and they may be broadly
classified as active or passive, depending on the nature of the source of radiation.
A passive system senses energy reflected from a target irradiated with a natural
source, for example the sun, while active systems such as radar and lidar (laser
radar) utilize artificially-generated energy sources to irradiate targets.
Polarization is a property of light or electromagnetic radiation that conveys
information about the orientation of the transverse electric field, and it comple-
ments other electromagnetic radiation attributes such as intensity, frequency, or
spectral characteristics. Specifically, polarization of reflected light contains infor-
mation about material composition (dielectric or metal), shape, surface features,
and roughness of a surface [1]. The information conveyed by the intensity and
spectral data gathered using passive sensors has long been exploited in remote
sensing applications such as target detection, object recognition, change detec-
tion, shape extraction and anomaly detection. This is evidenced by the myriad
1
of algorithms available in literature that exploit intensity and spectral data — for
example see [2] and references therein. Until recently, however, the use of pas-
sive polarization information in remote sensing applications has received relatively
little attention.
Passive polarimetric imagery is attractive in remote sensing as it exhibits en-
hanced contrast over intensity imagery in certain situations. This additional con-
trast can be used to improve important remote sensing applications such as target
detection, target classification, object recognition and image segmentation. For
example, Sadjadi and Chun present a target detection method that uses Stokes
vector imagery acquired in the infrared (IR) portion of the spectrum [3, 4]. Their
method is applicable to small targets that subtend a few pixels in the recorded
polarization imagery. Also, Goudail et al. present parametric statistical signal
processing methods for detecting and segmenting objects using Stokes vector im-
ages [5]. They conclude that passive polarimetric systems have the potential to
improve target detection and segmentation performance. Early in our research, we
developed a method for classifying targets using polarimetric signatures [6]. This
method employed a nonparametric Parzen-windows based paradigm for classifying
objects directly from Stokes vector data. Recently, El Saba et al. demonstrated
the utility of passive polarimetric imagery in discriminating between targets and
decoys [7], showing that it is possible to discriminate between objects and decoys
that have identical color, paint type, and surface structure. Wolff has also studied
2
passive polarimetry as applied to material classification [8, 9] and has shown that
it is possible to distinguish between metals and dielectrics by recording the polar-
ization state of specularly reflected light using a passive polarimeter. In addition,
he has also shown that passive polarimetric images can be used to separate diffuse
and reflection components of an object [10, 8]. Finally, passive polarimetry has
also been shown to be useful in extracting shape information of surfaces. Exam-
ples of recovering shape information from passive polarimetric information have
been presented in [9, 11, 12, 13, 14].
While the methods mentioned above have clearly demonstrated the utility of
polarization in various applications, they all suffer from certain drawbacks. The
methods described in [3, 4, 5, 6] do not account for the orientation of the source
relative to the target object and the object. This affects the observed polarimetric
signature, and, consequently, the performance of the algorithms based on this
model must degrade with the uncontrolled light source/camera geometry that
exist in a realistic remote sensing environment. The methods presented in [11,
12, 13, 14] also assume that certain material properties of the target are known
which may not be the case in many remote sensing applications where we want to
actually identify the material itself from passive polarization measurements. While
the segmentation method presented in [8, 9] does not assume any knowledge of
material properties, its application is limited since it can only distinguish between
metals and dielectrics — a user may desire more information such as whether the
3
target is made of aluminum or copper. The method presented in this dissertation
does not suffer from these limitations.
The main goal of the research presented in this dissertation is to demonstrate
the utility of passive polarimetric imagery in remote sensing applications including
target classification, segmentation and object recognition. Passive polarimetric
sensors that work in the visible portion of the spectrum have shown promise for
widespread use in the future in remote sensing. However, the signatures collected
by these sensors are highly dependent on the relative geometry between light
source, imaging platform and the target position. We develop here a method to
extract target parameters that are highly invariant to the positions of the illumi-
nation source and imaging sensors. These parameters will then enable us to design
key remote sensing applications including target classification, segmentation and
object recognition that are robust to source and observational geometry. In order
to achieve this objective, we utilize a physics-based polarimetric bidirectional dis-
tribution function (pBRDF), which is applicable to specular targets in modeling
the observational geometry that accounts for the orientation of the source and the
target. The complex index of refraction, a fundamental property that describes a
material and the reflection angle, is estimated using this pBRDF model.
We then employ the extracted parameters as feature vectors in the design
of two important image processing applications, namely image segmentation and
material classification. Consequently, the resulting implementations are robust to
4
illumination conditions, specifically the position of the illumination source relative
to those of the target and the imaging sensor. We also briefly explore the idea
of using an extended pBRDF model that accounts for volumetric scattering in
addition to the specular component for parameter estimation.
1.1 Contributions of Proposed Work
The contributions of the proposed work to existing techniques for exploiting pas-
sive polarimetric imagery are described in the following:
1. The index of refraction, a fundamental material property of a target, is
estimated from multiple polarization measurements where it is assumed that
the illumination source position varies between measurements. Thus, the
proposed method provides features that describe an object that are invariant
to illumination conditions unlike existing work that does not account for
variations in illumination source position [3, 4, 5, 6].
2. The invariant parameters are used as feature vectors to design two image
processing applications, namely image segmentation and material classifica-
tion. Both algorithms are robust to illumination conditions since they use
the extracted index of refraction as the feature vector. Again, this is unlike
algorithms previously proposed in literature to exploit polarization [8, 9].
3. The proposed estimation framework is adapted to the case where the posi-
5
tion of the observer (camera) changes between measurements while that of
the source remains fixed. This scenario occurs often in practice; for example
in the case where the imaging platform is located on an unmanned aerial
vechile (UAV).
4. We lay the groundwork for handling the parameter estimation problem for
the case of an extended pBRDF model that accounts for volumetric scatter-
ing in addition to the specular component.
5. The proposed work does not assume any knowledge of material parameters
unlike previous work where the parameters are known in advance [11, 12, 13].
This allows our method to be employed in a wider variety of remote sensing
applications than is possible with previous algorithms.
6. It should be possible to obtain additional information about the object of
interest from the estimated parameters such as the index of refraction. For
example, it will be possible to determine whether an object is composed
of aluminum or copper. This differs from previous work that could only
distinguish between dielectrics and metals [8, 9].
1.2 Dissertation Outline
The dissertation is organized as follows: chapter 2 provides the definitions of basic
polarization terms as well as a detailed description of Priest and Meier pBRDF
6
model [15] used in this work. Chapter 3 provides a detailed description of the
estimation framework developed for recovering the index of refraction and the re-
flection angle from multiple polarization measurements where the position of the
source changes between measurements. We also study here the complementary
problem where the position of the observer changes and the source remains fixed
between measurements. We present the image segmentation and material classifi-
cation algorithm in chapter 4. Chapter 5 provides a description of our exploratory
study on the parameter estimation problem for the generalized pBRDF model and
we conclude this dissertation in chapter 6.
7
2 POLARIZATION
In this chapter we provide a brief description of the mathematical tools needed
to describe polarization. The description presented here is formulated in terms
of observable quantities, namely, irradiances [16]. This representation involves
using Stokes parameters or Stokes vectors and Mueller matrices for describing
polarized light. We then describe the polarimetric bidirectional reflectance distri-
bution function (pBRDF) which is a tool used to characterize optical scattering
from surfaces. Note that an alternate description of polarized light exists which
involves using the so called polarization ellipse to formulate an instantaneous rep-
resentation [16, 17]. However, this description involves observing fluctuations that
occur within a period on the order of 10−15 seconds and, consequently, cannot be
observed using a physical detector. The interested reader is referred to the text
book by Hecht [16] for a detailed description of the polarization ellipse.
2.1 Stokes Vector
A Stokes vector is a four element vector that completely characterizes polarization.
The first element of the Stokes vector denoted by s0 corresponds to irradiance.
The second and third elements (s1 and s2) describe the orientation of linear polar-
ization components and the third element s3 provides information about circular
8
polarization.
We now describe the relationship between the Stokes vector elements and the
components of the electric field vector. In the following discussion, the vectors
and matrices, unless specifically noted, are denoted by boldface letters. An elec-
tromagnetic wave traveling in the z direction can be written as
�E = E0 exp (i (ωt− k · r + δ)) (2.1)
where i =√−1, �E is the electric field vector of the wave as a function of position
vector r and time t, E0 is the amplitude of the wave, ω is the angular frequency
given by 2πcλ
where c is the speed of light in vacuum, δ is the phase associated
with the wave, and k is the wave vector pointing in the direction of propagation.
In free space the field propagates as a transverse wave so that �E will have a vector
direction that is normal to the direction of propagation.The wave vector has a
complex amplitude in general and is given by [18]
k =
(2π
λ− iα
2
)n (2.2)
where λ is the wavelength in the medium in which the wave is traveling and α is the
absorption coefficient [18] and n is a unit vector in the direction of propagation.
The electric field vector can be expressed in terms of the orthogonal x− and
y− components as
�E = Exx + Eyy (2.3)
9
where Ex and Ey are the amplitudes of the x− and y− components of the electric
field vector , and are given by
Ex = E0x exp (i (ωt − k · r + δx)) (2.4)
Ey = E0y exp (i (ωt − k · r + δy)) (2.5)
where δx and δy are the phase terms while E0x and E0y are the magnitudes of Ex
and Ey respectively. The electric field vector of a wave can therefore be rewritten
in terms of magnitude components of Ex and Ey as follows:
�E = [E0x exp (iδx) x + E0y exp (iδy) y] exp [i (ωt − k · r)] . (2.6)
The relative magnitudes and phases of Ex and Ey determine the net polarization
of the electric field. For example, if E0y = 0, then the electric field oscillates in
the x direction and corresponds to horizontal polarization.
The Stokes vector is defined by the electric field components given in (2.1) and
(2.6) as follows: ⎛⎜⎜⎝s0
s1
s2
s3
⎞⎟⎟⎠ =
⎛⎜⎜⎝〈ExE∗
x + EyE∗y 〉
〈ExE∗x − EyE∗
y 〉〈ExE∗
y + EyE∗x〉
i(〈ExE∗
y − EyE∗x〉)⎞⎟⎟⎠ (2.7)
using the complex field notation and⎛⎜⎜⎝s0
s1
s2
s3
⎞⎟⎟⎠ =
⎛⎜⎜⎝〈E2
0x + E20y〉
〈E20x − E2
0y〉〈2E0xE0y cos (δx − δy)〉〈2E0xE0y sin (δx − δy)〉
⎞⎟⎟⎠ (2.8)
in terms of the magnitudes and relative phase (δx − δy) of Ex and Ey. Note that
〈· · · 〉 denotes the time averaging operator. The Stokes vector components are
10
Table 2.1: Stokes vector sign interpretation.
Element Positive Negatives1 Horizontal polarization dominates Vertical polarization dominatess2 +45◦ polarization dominates −45◦ polarization dominatess3 Right-hand circular polarization Left-hand circular polarization
dominates dominates
normalized with respect to s0 which implies that the Stokes components takes on
values in the range from −1 to +1. The sign of the Stokes vector components s1,
s2 and s3 indicates the dominant polarization direction present in the observed
Stokes vector. Table 2.1 summarizes the convention used for Stokes vector sign
interpretation.
Table 2.2 provide several examples of Stokes vectors. Using the Stokes vector,
we can define the degree of polarization (DOP) as well as the degree of linear
polarization (DOLP) which are used in this dissertation for developing the esti-
mation methods presented in chapter 3. The DOP, denoted as P , is given by
P =
√s21 + s2
2 + s23
s0
(2.9)
while the DOLP is denoted by PL and is given by
PL =
√s21 + s2
2
s0
. (2.10)
Previous work by Coulson has indicated that the amount of circular polarization
collected by passive systems from real-world scenes is negligible [19], and thus
11
Table 2.2: Stokes vector examples. Here t denotes vector (matrix) transposeoperation.
Direction of polarization Stokes Vector
Horizontal(
1 1 0 0)t
Vertical(
1 −1 0 0)t
+45◦(
1 0 1 0)t
−45◦(
1 0 −1 0)t
Right-circular(
1 0 0 1)t
Left-circular(
1 0 0 −1)t
Unpolarized(
1 0 0 0)t
P = PL for the rest of this dissertation. We also note in the passing that the
following inequality holds for any polarization state:
s20 ≥ s2
1 + s22 + s2
3, (2.11)
implying that 0 ≤ P ≤ 1.
2.2 Mueller Matrices
A Mueller matrix is a mathematical tool that provides information about the
change in the state of a Stokes vector when it interacts with an optical surface.
Specifically, it is a 4 × 4 matrix that completely characterizes the polarization
reflection and transmission properties of a medium or surface for any incident
Stokes vector sin. The Mueller matrices considered in this dissertation reduce to
3×3 matrices since the amount of circular polarization from reflection recorded by
passive remote sensing systems is assumed to be negligible [19]. The output Stokes
12
i r
z
y
x
i
r
Figure 2.1: BRDF Geometry.
vector after interaction with an optical surface is thus given mathematically by
sout = Msin (2.12)
where M denotes the Mueller matrix. The above equation can be explicitly written
as ⎛⎝ sout0
sout1
sout2
⎞⎠ =
⎛⎝ m00 m01 m02
m10 m11 m12
m20 m21 m22
⎞⎠⎛⎝ sin0
sin1
sin2
⎞⎠ . (2.13)
The interested reader is referred to the text by Hecht [16] for examples of Mueller
matrices for various optical systems such as polarizers and quarter-wave plates.
2.3 Bidirectional Reflectance Distribution Function
A bidirectional reflectance distribution function (BRDF) characterizes optical
scattering from surface reflections. Fig. 2.1 illustrates the geometry required
to specify the BRDF. θi, φi are the incident zenith and azimuth angles, respec-
13
tively, while θr, φr are the reflected zenith (or the reflection angle) and azimuth
angles, respectively. The BRDF is given by
f(θi, φi, θr, φr, λ) =dLr(θr, φr)
dE(θi, φi)sr−1 (2.14)
where λ denotes the wavelength, Lr is the radiance leaving the surface with units
of watts per square meter per steradian Wm2−sr
and E is the irradiance incident
on the surface with units of watts per square meter Wm2 . The BRDF has units
of inverse steradians sr−1. Because most materials have azimuthal or rotational
symmetry about the surface normal, the azimuthal angle can usually be expressed
as a difference φ = φr−φi which reduces the number of degrees of freedom by one.
Finally, the wavelength can be dropped because the polarization measurements
are collected at a known wavelength by using a suitable spectral filter. For the
remainder of this dissertation, we assume that the BRDF has the functional form
f(θi, θr, φ).
2.3.1 Polarmetric Bidirectional Reflectance Distribution Function
The polarimetric BRDF (pBRDF) is a generalization of the scalar BRDF and
is capable of modeling polarization effects. The idea of extending the BRDF to
account for polarization can be found in the Flynn and Alexander’s paper [20].
The pBRDF can be formally written as
dLr(θr, φr) = F(θi, θr, φr − φi)dE(θi, φi) (2.15)
14
where F is the pBRDF Mueller matrix, Lr is the reflected Stokes vector and E is
the incident Stokes vector. The relationship in (2.15) can be explicitly written as⎛⎝ dL0
dL1
dL2
⎞⎠ =
⎛⎝ f00 f01 f02
f10 f11 f12
f20 f21 f22
⎞⎠⎛⎝ dE0
dE1
dE2
⎞⎠ . (2.16)
A number of pBRDF models are available in literature. Priest and Germer [21]
and Priest and Meier [15] presented a polarimetric version of the well-known
Torrance-Sparrow microfacet model [22]. This model accounts for the specular
component of optical scattering. Meyers [23] extended this model to account
for volumetric scattering by using the Maxwell-Beard [24] model. Other models
that are available in literature include that of Conant and Iannarilli [25] and
Duncan [26]. Conant and Iannarilli [25] presented a polarimetric version of the
Sanford-Robertson model [27]. Duncan et al. present a physics-based model that
accounts for specular scattering.
In this dissertation, we utilize the pBRDF model proposed by Priest and
Meier [15]. This model can characterize the specular component of scattering
for a wide variety of target materials, for example bare metals and painted sur-
faces [28], that are of interest in remote sensing applications. Here, we present
only the pBRDF equations used in our research, referring interested readers to
Priest and Meier [15] for additional details.
15
2.4 Polarimetric BRDF for the Microfacet Model
The microfacet pBRDF model due to Priest and Meier [15] assumes that an object
surface is rough and is composed of a collection of mircofacets. Each microfacet
is a specular reflector that obeys Fresnel’s equations [18, 16]. Furthermore, volu-
metric scattering is not accounted for in this model; thus, the optical scattering
is assumed to be caused entirely by specular scattering. The microfacet pBRDF
model has two components, namely the microfacet probability density function
p (θN ) where (θN ) denotes the angle of a microfacet relative to the object surface
normal z and the Fresnel reflectance Mueller matrix M. The two components are
described in detail below.
2.4.1 Microfacet Probability Density Function
The microfacet probability density function characterizes the distribution of mi-
crofacets about an object’s surface normal. In this work, we denote the microfacet
probability density function by p (θN). Priest and Meier [15] assume that the mi-
crofacets are distributed according to a Gaussian density function given by
p (θN) =1
2πσ2 cos3 (θN)exp
(− tan2 (θN )
2σ2
)(2.17)
where σ describes the surface roughness. A higher value for σ implies a rougher
surface while a lower value corresponds to a smoother or a ”mirror-like” surface.
Note that the surface roughness has units of either radians [23] or degrees [29].
16
The angle θN can be determined using the following relationships [?]:
θN = arccos
(cos (θi) + cos (θr)
2 cos (β)
), (2.18)
with
cos (2β) = cos (θi) cos (θr) + sin (θi) sin (θr) cos (φr − φi) (2.19)
where θi and φi are the incident zenith and azimuthal angles, θr and φr are the
reflected or scattered zenith and azimuthal angles and β is the local angle of
incidence onto (or equivalently angle of reflection from) a microfacet. Note that
out of all of the microfacets that make up the surface, only the ones oriented
at the angle θN return light directly to the observer. Also note that 2β is the
angle between the source and the camera; this angle is sometimes referred to
as the phase angle in literature [28]. Thus, the probability density function is
completely determined once the surface roughness and observational geometry
are known and assumption holds of normally-distributed microfacets.
2.4.2 Fresnel Reflectance Mueller Matrix
The Fresnel reflectance Mueller matrix M provides the magnitude and polariza-
tion of the specular reflectance off the microfacets. This matrix can be thought
of as a generalization of Fresnel reflectance to the case of polarization. The Fres-
nel reflectance in the scalar case is the average of the in-plane reflectivity (p)
and out-of-plane (s) Fresnel reflectivity [18, 16]. The extension from the scalar
17
case to polarization case is involved because of the coordinate transformations
needed to relate the microfacet Fresnel reflectance to the macrofacet or object
surface coordinate system which is used to define the pBRDF geometry. The
required transformations are given by a closed-form solution presented by Priest
and Meier [15] which is a key contribution of their work. It is assumed that the
position of the source (θi, φi) and the observer (camera) (θr, φr) are fixed. This
implies that the microfacet orientation θN is fixed for this geometry through (2.18)
and (2.19) and as noted previously, the microfacet orientation that contributes to
the specular signature is fixed by the observational geometry.
The required coordinate transformations are given in terms of a Jones Ma-
trix [15]. It is possible to define a Mueller matrix given a Jones matrix while the
converse is not true. The Jones matrix transforms the incident electric field ori-
ented in the in-plane (p) and out-of-plane (s) polarization states to the reflected
polarization state as follows:
[Er
s
Erp
]=
[rs 00 rp
] [Ei
s
Eip
](2.20)
where rs and rp are Fresnel amplitudes [18], Eis, Ei
s are the incident s and p polar-
ization electric field components and Ers , Er
s are the reflected s and p polarization
electric field components. Two coordinate transformations are needed to relate
the relative s and p polarization components incident upon the microfacet sur-
face normal to the target surface coordinate system used to described the pBRDF
18
geometry. The first transformation rotates the plane containing the incident ra-
diance and the target surface normal vector to the plane containing the incident
radiance and microfacet surface normal vector. This rotation is specified by an
angle ηi. The second transformation rotates the plane containing the reflected
radiance and the microfacet surface normal to the plane containing the reflected
radiance and the target surface normal vector. This rotation is specified by ηr.
The angles ηi and ηr are uniquely determined by the observational geometry and
are given by:
cos (ηi) =
cos(θi)+cos(θr)2 cos(β)
− cos (θi) cos (β)
sin (θi) sin (β)(2.21)
cos (ηr) =
cos(θi)+cos(θr)2 cos(β)
− cos (θr) cos (β)
sin (θr) sin (β). (2.22)
The reflected electric field components in terms of the observation geometry
are given by
[Er
s
Erp
]=
[cos (ηr) sin (ηr)− sin (ηr) cos (ηr)
] [rs 00 rp
] [cos (ηi) sin (ηi)− sin (ηi) cos (ηi)
] [Ei
s
Eip
](2.23)
or [Er
s
Erp
]=
[Tss Tps
Tsp Tpp
] [Ei
s
Eip
](2.24)
where
[Tss Tps
Tsp Tpp
]=
[cos (ηr) sin (ηr)− sin (ηr) cos (ηr)
] [rs 00 rp
] [cos (ηi) sin (ηi)− sin (ηi) cos (ηi)
](2.25)
is a Jones matrix.
19
The Jones matrix in (2.24) can be used to construct the Fresnel reflectance
Mueller matrix M. The formulas required to construct the Mueller matrix from
the Jones matrix are given by [15, 28]:
m00 =
(|Tss|2 + |Tsp|2 + |Tps|2 + |Tpp|2)
2(2.26)
m01 =
(|Tss|2 + |Tsp|2 − |Tps|2 − |Tpp|2)
2(2.27)
m02 =
(TssT
∗ps + T ∗
ssTps + TspT∗pp + T ∗
spTpp
)2
(2.28)
m10 =
(|Tss|2 − |Tsp|2 + |Tps|2 − |Tpp|2)
2(2.29)
m11 =
(|Tss|2 − |Tsp|2 − |Tps|2 + |Tpp|2)
2(2.30)
m12 =
(TssT
∗ps + T ∗
ssTps −(TspT
∗pp + T ∗
spTpp
))2
(2.31)
m20 =
(TssT
∗sp + T ∗
ssTps + TpsT∗pp + T ∗
psTpp
)2
(2.32)
m21 =
(TssT
∗sp + T ∗
ssTsp −(TpsT
∗pp + T ∗
psTpp
))2
(2.33)
m22 =
(TssT
∗pp + T ∗
ssTpp + TpsT∗sp + T ∗
psTsp
)2
(2.34)
where the asterisk (∗) in (2.26 - 2.34) denotes the complex conjugation operation
and mjl denotes the element of M in the jth row and lth column.
The expression for the pBRDF Mueller matrix F can be determined using
(2.17) through (2.34) and is thus given by
fjl(θi, θr, φ) =1
(2π)(4σ2)(cos (θ))4
exp[− tan2(θ)
2σ2
]cos(θr) cos(θi)
mjl(θi, θr, φ) (2.35)
20
where fjl again denotes the element of F in the jth row and lth column. The
model defined in (2.35) has three parameters θi, θr, φ for a given target surface.
As a result, the model depends on the complex index of refraction of the target
n+ ik and the surface roughness parameter σ in addition to the angular quantities
defined above. The complex index of refraction occurs in the expressions for the
Fresnel amplitudes rs and rp [16, 30] which are in turn used to construct the
Fresnel reflectance Mueller matrix M as shown in (2.20) through (2.34).
A note on terminology associated with the complex index of refraction. n
is called the real index of refraction while k is referred to as the extinction co-
efficient [18]. The extinction coefficient and consequently the complex index of
refraction arises in the cases of conductors as they exhibit a complex dielectric
constant [30] as explained in appendix A. The complex dielectric constant is de-
pendent on both the usual dielectric constant as well as the conductivity of the
medium. The textbook by Born and Wolf [30] ( pp. 611-613) derives expressions
for the complex index of refraction for a particular medium as a function of its
dielectric constant, permeability and conductivity.
21
3 PROPOSED PARAMETER ESTIMATION METHODOLOGY
In this chapter we develop a method to extract parameters that are invari-
ant to illumination conditions and viewpoints from multiple passive polarimetric
measurements. These parameters can then be used to design remote sensing appli-
cations that are robust to both illumination and imaging conditions. We consider
two scenarios for making multiple polarimetric measurements in this chapter. In
this first case, it is assumed that the position of the receiver or camera remains
fixed while that of the illumination source moves between measurements. In prac-
tice this would occur when measurements of a target scene are acquired at different
times in a day. In the second scenario, we assume that the position of the source
remains fixed while that of the camera changes between each measurement. This
scenario occurs in practice when multiple polarimetric images of a target of in-
terest on the ground are gathered from moving platforms like unmanned aerial
vehicles (UAVs).
Early work on the utility of passive polarimetry for remote sensing applica-
tions was published by Wolff [8]. He showed that it was possible to distinguish
between metals and dielectrics by recording the polarization state of specularly
reflected light using a passive polarimeter. His method involves a threshold-based
discrimination procedure that utilizes the polarization Fresnel ratio [8], which is
22
the ratio of perpendicular to parallel polarization state components. His method,
however, does not directly incorporate the fundamental parameters that describe
the material such as the refractive index.
Miyazaki et al. developed a method to estimate the surface normal of trans-
parent dielectric objects by recording the polarization state of specularly reflected
light [11]. Their main contribution is to use two views of the object to solve the
problem of mapping the degree of polarization to the reflection angle since this
correspondence is not one-to-one. Atkinson and Hancock utilize both specular and
diffuse polarization for recovering the surface orientation of dielectric objects [13].
Unfortunately, their methods assume the index of refraction of an object is known
a priori. Morel et al. have proposed a method to estimate the shape of specular
metallic objects from polarization information [31, 32]. They use an approxima-
tion for the complex index of refraction to simplify the relationship between the
degree of polarization and the index of refraction. This approximation enables the
three dimensional surface reconstruction of metals from only a single view of the
object. Specifically a ”pseudo-refractive index” for the object is estimated, which
is in turn used to estimate the zenith angle needed for surface reconstruction.
Their method, however, is applicable only to shiny or highly specular metallic
objects which limits its utility for remote sensing applications.
Fetrow et al. [29] have developed a method to determine the index of refraction
of an object from long wave infrared (LWIR) polarization measurements. They
23
have modified the well known Torrance-Sparrow [22] scattering model to predict
the polarization emitted from a roughened surface but their results are only ap-
plicable to measurements made in the LWIR whereas the work presented here is
applied to measurements collected in the visible range of the spectrum. In addi-
tion, our method also recovers the reflection angle — a problem not addressed by
Fetrow et al. [29].
Sadjadi [33] has proposed a method to estimate the surface normal vector
from a single infrared passive polarization measurement. His method estimates
the azimuthal angle from the angle of polarization while the reflected zenith or
the reflection angle is recovered by solving an equation derived from Fresnel’s
equations. This method, however, requires either a priori knowledge or a perfect
estimate of the index of refraction in order to accurately recover the reflection
angle. In contrast, the method we present here accurately recovers the reflection
angle without making any assumptions about the index of refraction. Further-
more, the equations developed by Sadjadi [33] assume that the objects emitting
radiation have a real index of refraction while our method can also account for ob-
jects that have a complex index of refraction. Photometric stereo [34] refers to the
technique that recovers the surface orientation and hence the three-dimensional
shape of an object from multiple intensity [34] or color [35] images wherein the
viewpoint is fixed and the direction of the illumination source is varied between
successive measurements. The proposed method utilizes a similar set-up to ac-
24
curately estimate the reflection angle from a set of passive polarimetric images.
Our method is particularly useful in scenarios where polarimetric imagery exhibits
higher contrast compared to intensity and color image data [1].
In this chapter we present a method to jointly estimate the complex index of
refraction and the reflection angle of specular targets from a set of passive po-
larization measurements made in the visible range of the spectrum for the fixed
receiver (camera) scenario. The term reflection angle refers to the angle between
the observer, or a camera, and the object surface normal. This work general-
izes our previous work [36, 37, 38] and is applicable to a wide class of objects
including dielectrics and metals. The object is modeled using the polarized mi-
crofacet bidirectional reflectance distribution function (pBRDF) model [15] where
it is assumed that polarization is caused by specular scattering. Furthermore, it
is assumed that the position of the illumination source changes between measure-
ments in the fixed receiver (camera) scenario while that of the observer changes
in the fixed illumination source scenario. It is also assumed that the position
of the object is fixed and that scattering occurs in the plane of incidence. The
parameters of interest are recovered using the well-known Levenberg-Marquardt
method [39], a nonlinear least squares estimation algorithm.
We adapt the above method to estimate parameters for the second case where
the position of the source is fixed while the position of the camera changes between
measurements. In this scenario we estimate the complex index of refraction and
25
the illumination direction (angle of incidence) from multiple polarization measure-
ments. The term illumination direction refers to the angle between the source and
the object surface normal.
The chapter is organized as follows: section 3.1 describes the idea of creating
geometry invariant feature vectors for image processing and section 3.2 describes
the assumptions used to model the problem. An expression for the degree of
polarization is derived in section 3.3 and the parameter estimation algorithm
is described in section 3.4. We present experimental results in section 3.5 and
conclude the chapter in section 3.6.
3.1 Feature Vector Creation for Classification
A pattern classification system forms a feature vector that characterizes an object
and is used by the classification algorithm to assign that object to an appropriate
category. The classification algorithm relies on a priori knowledge of the object
categories, often statistical in nature, in order to classify the target. Outdoor
remote sensing systems that utilize imaging sensors may record intensity, spectral
(multispectral and hypersepctral) and/or polarization information. This informa-
tion can be used directly to form feature vectors as part of the classifier. However,
the intensity, spectral and polarization information obtained from target reflec-
tions depend on a variety of factors including the position of the illumination
source (typically the Sun in passive systems) and the viewing geometry. Conse-
26
quently, the information contained in a reflection may change dramatically due
to varying imaging conditions which, in turn, will likely impair the performance
of the classification system. Therefore, it is desirable to create feature vectors
that describe an object well but that are also invariant to the various imaging
conditions commonly encountered in remote sensing. In the following sections, we
demonstrate that the index of refraction as estimated using the proposed nonlin-
ear optimization approach is largely invariant to the position of the source and the
reflection angle, thus making it a potentially useful feature vector for designing
remote sensing systems that are robust to changing illumination conditions and
imaging conditions.
We note here that accurately estimating the true index of refraction, while
desirable, is not strictly necessary for classification or segmentation since illumi-
nation invariance and consistency are the keys here. In fact, our experimental
results with laboratory data indicate that the estimates for the index of refraction
for certain samples considered in this work do not match with published values.
Consequently, it might be more accurate to state that we estimate here an effec-
tive or apparent index of refraction, and that this is then utilized to recover the
reflection angle. Experimental results clearly show, however, that the recovered
apparent index of refraction is sufficient for accurate reflection angle estimation.
27
Surface normal (z)
Camera
Object with a rough surface
Source
r
i1
i2
sc1
sc2
r is the reflection angle
sc1 and sc2 are the phase angles
i1 and i2 are the incident zenith angles
Figure 3.1: Geometry assumed for the fixed receiver (camera) case. The positionof the illumination source changes between measurements
3.2 Problem Description
Figures 3.1 and 3.2 illustrate the observational geometry for the two scenarios
considered in our work for invariant parameter estimation. Specifically the fixed
receiver (camera) and fixed illumination source scenario are described in the fol-
lowing.
3.2.1 Fixed Receiver (Camera) Scenario
Figure 3.1 illustrates the observational geometry assumed for the fixed receiver
(camera) case. The object is assumed to be planar with a rough surface modeled
as a collection of microfacets - a model referred to as the microfacet model in
the literature [15]. θi1 and θi2 denote the incident zenith angles with respect to
28
Surface normal (z)
Object with a rough surface
Source
i
r1
r2
sc1
sc2
i is the incident zenith angle
sc1 and sc2 are the phase angles
r1 and r2 are the reflected zenith angles
Camera
Figure 3.2: Geometry assumed for the fixed source scenario. The position of theobserver (camera) changes between measurements
the surface normal z corresponding to two different positions of the illumination
source. θr is the reflection angle or the reflected zenith angle with respect to the
surface normal z while θsc1 and θsc2 are the angles between the source and the
camera corresponding to the two different locations of the source. We assume
perfect knowledge of the source position with respect to the camera. In other
words, it is assumed that θsc1 and θsc2 are known at the receiver. Furthermore, it
is also assumed that the source is unpolarized and that the camera is fixed and in
the plane of incidence. These assumptions facilitate the development of our basic
method to recover the index of refraction and the reflection angle of the object
under observation using multiple passively-collected polarimetric images.
29
3.2.2 Fixed Illumination Source Scenario
Figure 3.2 illustrates the geometry assumed for the fixed source and moving cam-
era scenario. In this case, θi denotes the illumination direction or incident zenith
angle with respect to the surface normal z, θr1 and θr2 denote the reflected zenith
angle with respect to the surface normal z corresponding to two different posi-
tions of the camera while θsc1 and θsc2 are the angles between the source and the
camera corresponding to the two different locations of the camera. In this case
we recover the index of refraction and the illumination direction from multiple
passive polarimetric images.
3.3 Degree of Polarization for In-Plane Scattering
The pBRDF equations for the microfacet model are presented in chapter 2. In this
section we present the pBRDF equations for scattering in the plane of incidence
where φr − φi = 180◦. For this geometry, it can be shown that ηi and ηr defined
in (2.21) and (2.22) are both 0◦. Thus, from (2.25) we have that
[Tss Tps
Tsp Tpp
]=
[rs 00 rp
]. (3.1)
The Fresnel reflectance Mueller matrix can be simplified using (2.26) - (2.34) and
is given by
M =
⎛⎝ m00 m10 0m10 m00 00 0 m22
⎞⎠ (3.2)
30
where we have dropped the arguments for the individual elements of M for the sake
of readability. Note that element m11 = m00 in this case. The pBRDF Mueller
matrix F is therefore defined by (2.35) using (3.2). We provide expressions for
the elements of M for completeness below:
m00 =
(|rs|2 + |rp|2)
2(3.3)
m10 =
(|rs|2 − |rp|2)
2(3.4)
m22 =
(rsr
∗p + r∗srp
)2
. (3.5)
We now derive an expression for the degree of polarization (DOP) for the case
of scattering in the plane of incidence. The illumination source is assumed to be
unpolarized which is appropriate for passive remote sensing systems. Thus, the
input Stokes vector is given by [1 0 0]T where t denotes the vector transposition
operator. In the following, we drop the arguments for the individual elements fjl
for clarity. Assuming that the zenith angles for the source and the camera are θi
and θr, respectively, the observed Stokes vector at the camera is given by⎛⎝ sr0
sr1
sr2
⎞⎠ =
⎛⎝ f00 f10 0f10 f00 00 0 f22
⎞⎠⎛⎝ 100
⎞⎠ (3.6)
where the superscript r denotes the Stokes vector components for reflected light.
The matrix-vector multiplication results in⎛⎝ sr0
sr1
sr2
⎞⎠ =
⎛⎝ f00
f10
0
⎞⎠ (3.7)
31
and the degree of polarization of the observed Stokes vector is given by
P =f10
f00
(3.8)
implying that
P =m10
m00
(3.9)
which follows from (2.35). This can be simplified using (3.3) and (3.4) which
implies that the DOP is given by
P =
(|rs|2 − |rp|2)(|rs|2 + |rp|2) (3.10)
which can be rewritten in terms of Fresnel reflectance as follows:
P =Rs − Rp
Rs + Rp
(3.11)
where Rs and Rp are respectively the s-plane and p-plane Fresnel reflectance [28,
18]. We note here that the Fresnel reflectances are functions of the complex
index of refraction and angle β. These parameters, in turn, are functions of the
observational geometry θi, θr, φi and φr as defined in (2.19).
The degree of polarization given by (3.11) can be further simplified using Fres-
nel’s equations. Using the notation adopted in the Nonconventional Exploitation
Factors (NEF) Modeling [40] and Shell [28], the Fresnel reflectances Rs and Rp
are given by
32
Rs =(A (n, k, β) − cos (β))2 + B2 (n, k, β)
(A (n, k, β) + cos (β))2 + B2 (n, k, β)(3.12)
Rp = Rs
[(A (n, k, β) − sin (β) tan (β))2 + B2 (n, k, β)
(A (n, k, β) + sin (β) tan (β))2 + B2 (n, k, β)2
]. (3.13)
The quantities A (n, k, β) and B (n, k, β) are defined as:
A (n, k, β) =
√√C (n, k, β) + D (n, k, β)
2(3.14)
and
B (n, k, β) =
√√C (n, k, β) − D (n, k, β)
2(3.15)
where
C (n, k, β) = 4n2k2 + D2 (3.16)
and
D (n, k, β) = n2 − k2 − sin2(β) (3.17)
with n and k being the real and imaginary parts of the complex index of refraction
and β being given by
β =θi + θr
2=
θsc
2(3.18)
from Figure 3.1. Note that 0◦ ≤ β ≤ 90◦ because 0◦ ≤ θi, θr ≤ 90◦. We observe
from (3.12)- (3.17) that Rs ≥ 0 and Rp ≥ 0. Furthermore, it is easily seen that
0 ≤[
(A (n, k, β) − sin (β) tan (β))2 + B2 (n, k, β)
(A (n, k, β) + sin (β) tan (β))2 + B2 (n, k, β)
]≤ 1. (3.19)
33
Consequently, for these definitions Rs ≥ Rp which in turn guarantees that the
degree of polarization given by (3.11) is non-negative. The degree of polarization,
simplified using (3.12)- (3.17), is given by
P (n, k, β) =2A (n, k, β) sin2(β) cos(β)
A2 (n, k, β) cos2(β) + sin4(β) + B2 (n, k, β) cos2(β). (3.20)
3.4 Estimation Approach
This section describes the estimation methodology utilized to recover the param-
eters of interest for the two scenarios described in section 3.2. We first describe
the estimation approach used to recover the complex index of refraction and the
reflection angle and then adapt this method to recover the parameters of interest
for the second scenario.
3.4.1 Index of Refraction and Reflection Angle Estimation
Our objective is to recover the index of refraction and the reflection angle of
an object from a set of DOP measurements. A two step approach is adopted
for parameter estimation. The first step involves estimating the complex index
of refraction while the reflection angle is determined in the second step. We
observe from (3.20) that the DOP is a function of n, k, θi and θr. Note that
the dependence on θiand θr comes from the β variable. If θsc is known, then it
follows from (3.20) that the DOP is a function of only the index of refraction. The
assumption that the phase angle θsc is known at the camera is not unreasonable
34
in many remote sensing applications given that the imaging platform is likely to
have a GPS on board and is thus able estimate the position of the illumination
source, for example the Sun, relative to its own position with high accuracy. Thus,
the complex index of refraction can be estimated using (3.20) by making multiple
DOP measurements with the source at different positions which results in a system
of nonlinear equations given by
Pj (n, k) =2Aj (n, k, β) sin2(βj) cos(βj)
A2j (n, k, β) cos2(βj) + sin4(βj) + B2
j (n, k, β) cos2(βj)(3.21)
where βj = θij + θr and j ∈ {1, 2, · · ·T} indicates the measurement number. If
T ≥ 3, then (3.21) corresponds to an overdetermined system of nonlinear equa-
tions. Thus, the system of nonlinear equations can be recast as a nonlinear least
squares problem, and can be solved using the well-known Levenberg-Marquardt
method [39]. We refer the interested reader to appendix B for an explanation as
well as the implementation details.
The number of unknowns in (3.20) reduces to two once we have an estimate
for the complex index of refraction. Consequently (3.20) can be rewritten as
P (θi, θr) =2A (n, k, β) sin2
(θi+θr
2
)cos(
θi+θr
2
)A2 (n, k, β) cos2
(θi+θr
2
)+ sin4
(θi+θr
2
)+ B2 (n, k, β) cos2
(θi+θr
2
)(3.22)
where A and B are given by (3.14)-(3.17). The reflection angle or the reflected
zenith angle is estimated from (3.22) again by using multiple measurements with
the source at different positions. The result is the system of nonlinear equations
35
given by
Pj (θij , θr) =2Aj (n, k, β) sin2
(θij+θr
2
)cos(
θij+θr
2
)A2
j (n, k, β) cos2(
θij+θr
2
)+ sin4
(θij+θr
2
)+ B2
j (n, k, β) cos2(
θij+θr
2
)(3.23)
where j ∈ {1, 2, · · ·T}. The system of equations given by (3.23) corresponds to
an overdetermined system of nonlinear equations in two unknowns namely θi1
and θr. This is seen by writing θij , j ∈ {2, 3, · · ·T} as θij = θi1 + Δθj1 where
Δθj1 = θscj − θsc1. The reflection angle θr is recovered by again solving (3.23)
using the Levenberg-Marquardt method [39].
3.4.2 Index of Refraction and Illumination Direction Estimation
We now describe the estimation method used to recover the complex index of
refraction and the incident direction (angle of incidence) from multiple passive
polarization measurements where the position of the camera changes between
each measurement while the illumination source position remains fixed. We again
use (3.20) as the basis for our estimation methodology. Specifically, the complex
index of refraction is estimated as before by using (3.20) by utilizing multiple
measurements made with the camera in different positions. In this case, we obtain
a system of nonlinear equations given by
Pj (n, k) =2Aj (n, k, β) sin2(βj) cos(βj)
A2j (n, k, β) cos2(βj) + sin4(βj) + B2
j (n, k, β) cos2(βj)(3.24)
36
where βj =θi+θrj
2and j ∈ {1, 2, · · ·T} indicates the measurement number. As
before βj represents the arithmetic mean of the angle of incidence and reflection
angle for the jth measurement. This quantity is assumed to be known as it can be
estimated using a GPS system as in the previous scenario. Assuming T ≥ 3 the
system of nonlinear equations can be recast as a nonlinear least squares problem,
and is solved once more using the Levenberg-Marquardt method [39].
The incident direction or the angle of incidence is estimated by solving the
following system of nonlinear equations:
Pj (θi, θrj) =2Aj (n, k, β) sin2
(θi+θrj
2
)cos(
θi+θrj
2
)A2
j (n, k, β) cos2(
θi+θrj
2
)+ sin4
(θi+θrj
2
)+ B2
j (n, k, β) cos2(
θi+θrj
2
)(3.25)
where j ∈ {1, 2, · · ·T}. The system of equations given by (3.25) can be shown to
correspond to an overdetermined system of nonlinear equations in two unknowns
namely θi and θrj . This is seen by writing θrj , j ∈ {2, 3, · · ·T} as θrj = θr1 +Δθj1
where Δθj1 = θscj − θsc1. The angle of incidence θi is recovered by solving (3.25)
using the Levenberg-Marquardt method [39].
3.5 Experimental Results
A two-fold approach is adopted to validate the estimation method presented in
section 3.4. First, the sensitivity of the proposed method to the number of input
measurements and to measurement noise is studied using Monte Carlo simula-
tions. The simulation study presented below assumes a fixed camera scenario,
37
Table 3.1: Simulation results for a dielectric surface for ”low-noise” scenario. Thetrue index of refraction (n) is 1.5 and the true reflection angle is 60◦. The rangeof the angle of incidence is noted below and is varied in steps of 5◦. The varianceof the Gaussian noise is 0.1% of the maximum DOP value which is 1.0 for thiscase. The results are obtained from 500 Monte Carlo trials.
Number of n RMSEn θr RMSEθr
measurements (◦)3 (55◦ − 65◦) 1.486 ± 0.007 0.076 54.982 ± 0.413 6.8775 (50◦ − 70◦) 1.482 ± 0.005 0.057 60.468 ± 0.421 4.82210 (35◦ − 80◦) 1.482 ± 0.003 0.039 61.482 ± 0.487 5.743
however, the results from this study directly extend to the fixed illumination
source scenario due to the symmetric nature of the problem which can be seen
from (3.20). Next, the effectiveness of the estimation method for the two scenarios
outlined in section 3.2 is demonstrated for real DOP measurements collected in
the laboratory.
3.5.1 Sensitivity Analysis
Synthetic data sets for the simulation study are generated using (3.20) with spe-
cific input values for n and k and a fixed value of θr, but a sequence of values
for θi as described in section 3.4. The sequence for θi is generated in fixed steps
of 5◦. We chose this step size value to be consistent with our choice of step size
for the laboratory experiments. We use 3, 5 and 10 measurements for the anal-
ysis, which implies that the maximum range for the angles of incidence is 45◦
corresponding to the case of 10 measurements. We chose this range because our
38
experimental results as presented in the next subsection indicate that the range of
the specular lobe is about 40◦ for the materials used here. Our experiments with
ideal synthetic data (no noise) indicate that the proposed estimation approach
can recover the parameters of interest to a high degree of accuracy in all the cases
considered in our experiments. Thus, the algorithm is not affected by the number
of measurements for the noiseless case.
The synthetic data sets are then perturbed with additive white Gaussian noise
in order to analyze the robustness of the proposed algorithm to non-ideal or noisy
inputs. Specifically, the noise is added to the DOP values (3.20) that are then
input to the Levenberg-Marquardt solver. The analysis is performed for both a
dielectric surface and a metallic surface. We consider two levels of measurement
noise in our analysis: (i) a ”low-noise” scenario where the DOP values are per-
turbed with Gaussian noise of variance equal to 0.1% of the maximum DOP value
and (ii) ”high-noise” scenario in which the DOP values are perturbed with Gaus-
sian noise of variance equal to 1% of the maximum DOP value. The maximum
DOP values for the dielectric and metallic surfaces are 1 and 0.082 respectively.
We note here that for these noisy cases, the Levenberg-Marquardt algorithm oc-
casionally converges to physically unrealizable estimates due to its unconstrained
nature. Consequently, we discard such estimates and restart the optimization al-
gorithm for the cases when either n or k is negative or the reflection angle is less
than 0◦ or greater than 90◦.
39
Table 3.2: Simulation results for a dielectric surface for ”high-noise” scenario. Thetrue index of refraction (n) is 1.5 and the true reflection angle is 60◦. The rangeof the angle of incidence is noted below and is varied in steps of 5◦. The varianceof the Gaussian noise is 1% of the maximum DOP value which is 1.0 for this case.The results are obtained from 200 Monte Carlo trials.
Number of n RMSEn θr RMSEθr
measurements (◦)3 (55◦ − 65◦) 1.413 ± 0.019 0.158 52.087 ± 0.812 9.8365 (50◦ − 70◦) 1.428 ± 0.017 0.140 57.653 ± 0.881 6.75910 (35◦ − 80◦) 1.444 ± 0.013 0.107 58.870 ± 0.569 4.251
Tables 3.1 and 3.2 show the average index of refraction n and reflection angle
(θr) estimates for the case of a simple dielectric surface. Table 3.1 lists the sample
mean and its 95% confidence interval for the ”low-noise” scenario. We observe
that the index of refraction is estimated to a high degree of accuracy for the
three cases considered. In addition, we observe that the root mean square value
(RMSE) for the estimates of the index of refraction improves with an increasing
number of measurements. We also note that there is a dramatic improvement in
the reflection angle estimate when the number of measurements is increased from
three to five. It is also clear from table 3.1, however, that no further improvement
is achieved beyond 5 measurements for the reflection angle because of a noise
floor introduced by the additive Gaussian noise. This trend is also observed in
the RMSE values for the reflection angle. Thus, we see that the accuracy of the
proposed estimation method improves as we collect more measurements up to
some saturation point. Table 3.2 shows the results for the ”high-noise” scenario.
40
The sample mean estimates as well as the RMSE values indicate that the estima-
tion performance for the index of refraction improves with an increasing number
of measurements. The reflection angle estimation performance consistently im-
proves with increasing measurements, unlike the results presented in table 3.1 for
the ”low-noise” scenario. As might be expected, the estimates for the higher noise
scenario are less accurate compared with those for the ”low-noise” case.
Tables 3.3 and 3.4 lists the results for a metallic surface, assumed to be copper,
with the complex index of refraction given by 0.314 + i3.554 [41]. Our simula-
tions indicate that accurate estimation of this complex index of refraction value
is generally more difficult than for the real-valued dielectric case. The results in
table 3.3 correspond to the ”low-noise” case. We observe from the table that the
accuracy of the index of refraction estimates again improve with the number of
measurements and that, in fact, the estimate we obtain with 10 measurements is
close to the true index. This is clear from the reduction in the RMSE values as
well as the sample mean estimates for the complex index of refraction. We note
that the reflection angle is estimated to a high degree of accuracy in all three
cases considered in our experiments. This is somewhat surprising but can be
understood by considering the parameter estimation algorithm as a curve fitting
method. Recall that the proposed estimation method first computes the index
of refraction and then uses the computed value to estimate the reflection angle.
Therefore, accurate reflection angle estimation requires only an apparent index of
41
Tab
le3.
3:Sim
ula
tion
resu
lts
for
am
etal
lic
surf
ace
for
”low
-noi
se”
scen
ario
.T
he
true
index
ofre
frac
tion
isn
=0.
314
and
k=
3.54
4w
hile
the
true
reflec
tion
angl
eis
60◦ .
The
range
ofth
ean
gle
ofin
ciden
ceis
not
edbel
owan
dis
vari
edin
step
sof
5◦.
The
vari
ance
ofth
eG
auss
ian
noi
seis
0.1%
ofth
em
axim
um
DO
Pva
lue
whic
his
0.08
2fo
rth
isca
se.
The
resu
lts
are
obta
ined
from
500
Mon
teC
arlo
tria
ls.
Num
ber
ofn
RM
SE
nk
RM
SE
kθ r
RM
SE
θ r
mea
sure
men
ts(◦
)3
(55◦
−65
◦ )2.
479±
0.30
14.
052
7.83
7±
0.59
58.
254
60.0
39±
0.00
80.
098
5(5
0◦−
70◦ )
1.47
4±
0.25
43.
117
5.73
4±
0.48
25.
911
59.8
23±
0.12
01.
382
10(3
5◦−
80◦ )
0.42
1±
0.07
50.
866
3.76
8±
0.16
41.
882
59.6
95±
0.13
01.
508
42
refraction that optimally fits, in the least square sense, the polarization mea-
surements with the model given by (3.20). This phenomenon has been used by
Morel et al. [32] to recover the reflected zenith angle for shiny or highly specular
surfaces. However, their work focused on recovering geometric information from
highly specular surfaces while the current work is applicable to rough surfaces.
We also observe from table 3.3 that the RMSE values for the reflection angle esti-
mates do not improve with more measurements. Although the RMSE appears to
be increasing with the number of measurements, we note that the reflection angle
is estimated accurately in all the three cases considered in our experiments. This
indicates that the algorithm performance is limited by a noise floor introduced by
the additive Gaussian noise.
Table 3.4 summarizes the simulation results for the metallic surface for the
”high-noise” case. We observe from the table that the accuracy of the index of
refraction estimation again improves with the number of measurements. However,
the estimate of the index of refraction obtained for 10 measurements is still far
from the true value. As a result, we consider a fourth case where we almost double
the number of measurements from 10 to 19 in order to more precisely study the
behavior of the proposed estimation method. In this case, we are forced to reduce
the step size of the angles of incidence from 5◦ to 2.5◦ in order to satisfy our
assumption about the width of the specular lobe. We observe from table 3.4 that
the estimated index of refraction moves even closer to the true index. This
43
Tab
le3.
4:Sim
ula
tion
resu
lts
for
am
etal
lic
surf
ace
for”h
igh-n
oise
”sc
enar
io.
The
true
index
ofre
frac
tion
isn
=0.
314
and
k=
3.54
4w
hile
the
true
reflec
tion
angl
eis
60◦ .
The
range
ofth
ean
gle
ofin
ciden
ceis
not
edbel
owan
dis
vari
edin
step
sof
5◦.
Not
eth
atth
est
epsi
zere
duce
sto
2.5◦
for
the
case
of19
mea
sure
men
ts(s
eete
xt
for
expla
nat
ion).
The
vari
ance
ofth
eG
auss
ian
noi
seis
1%of
the
max
imum
DO
Pva
lue
whic
his
0.08
2fo
rth
isca
se.
The
resu
lts
are
obta
ined
from
200
Mon
teC
arlo
tria
ls.
Num
ber
ofn
RM
SE
nk
RM
SE
kθ r
RM
SE
θ r
mea
sure
men
ts(◦
)3
(55◦
−65
◦ )7.
865±
0.82
69.
610
15.6
69±
1.35
315
.545
59.9
04±
0.19
51.
409
5(5
0◦−
70◦ )
6.91
8±
0.96
49.
580
13.3
21±
1.36
113
.832
59.7
01±
0.33
92.
461
10(3
5◦−
80◦ )
3.31
9±
0.82
26.
637
8.03
1±
1.17
79.
579
59.8
15±
0.21
01.
525
19(3
5◦−
80◦ )
1.31
0±
0.44
93.
378
5.37
3±
0.71
95.
486
59.2
81±
0.77
65.
629
44
indicates that the performance of the estimation algorithm gets more precise with
the number of measurements and that the algorithm appears to be converging
towards the true value as we increase the number of measurements. This trend
is also confirmed by the RMSE values for the index of refraction which decrease
with the increasing number of measurements. We also observe from table 3.4 that
the reflection angle is estimated to a high degree of accuracy as is the case for
the ”low-noise” scenario. This is clear from the sample mean of the reflection
angle and its 95% confidence intervals. No clear trend, however, emerges from the
RMSE values as we vary the number of measurements. In fact, the RMSE values
indicate that no further improvement in the accuracy of the reflection angle is
possible beyond 10 measurements.
The above results suggest that the estimation accuracy of the nonlinear esti-
mation approach improves with an increasing number of measurements up to some
saturation point. Moreover, the results for the metallic surface suggests that the
proposed method can recover the reflection angle accurately even with inaccurate
estimates of the index of refraction. The results also indicate that the accuracy
of our proposed estimation approach depends on the level of measurement noise.
Although not noted in the above discussion, we should mention that the
Levenberg-Marquardt solver requires a good initial estimate for accurate reflection
angle estimation which is generally the case with most iterative descent algorithms.
We caution, however, that the presented results are from a limited study and a
45
more comprehensive analysis of the proposed algorithm is a part of future work
at New Mexico State University.
3.5.2 Results with Laboratory Data
Polarization image samples of several materials have been collected with an imag-
ining polarimeter developed by the Electro-Optics Research Lab (EORL) at New
Mexico State University [42]. The EORL polarimeter consists of a linear polarizer
mounted in a computer-controlled rotation stage, a spectral filter (center wave-
length of 650nm and a bandwidth of 80 nm for the results shown here) and a
scientific-grade camera. A tungsten filament lamp is used as the source of illu-
mination in our experiments. A single Stokes vector image is formed from ten
images of a scene collected with the polarizer rotated in steps of 15◦ for each mea-
surement. The interested reader is referred to Damarla [42] for additional details
regarding the EORL polarimeter.
The target sample is mounted on a micrometer driven rotation stage and the
positioning of the source and the camera mounts is done with the aid of a large
protractor device. The incidence and reflection angles can be set to a precision of
approximately one degree or slightly better in the laboratory testbed. The target
samples in our experiments include Styrofoam pieces coated with flat green paint
and flat black paint as well as pieces of sandblasted copper and aluminum. The
root-mean square surface roughness of the samples was measured using a Surfcom
46
120A stylus profilometer to be 2.58 microns for flat green paint, 4.92 microns for
copper, 2.99 microns for flat black paint and 3.3 microns for aluminum. Thus, the
RMS surface roughness in all cases is greater than the wavelength of interest (650
nm), which satisfies the assumption made by the Torrance-Sparrow model [22]
that underlies the pBRDF model employed in our work. The DOP values needed
for parameter estimation were computed by averaging at least 100 x 100 pixels in
the Stokes vector images.
3.5.3 Fixed Camera Scenario
In this section, we present experimental results for the fixed camera case where
the camera position remains fixed while the position of the source (i.e., the angle
of incidence) changes between each measurement. The range of the angles of in-
cidence and the angle step sizes are not the same for all the cases considered in
our experiments. One of the reasons for this is the specular assumption inherent
in our modeling procedure. The camera needs to be in a region where the spec-
ular component clearly dominates the diffuse reflection for the incident angles of
interest. We could roughly identify this region as a relatively bright area of reflec-
tion centered about the mirror reflection angle. However, we were forced in some
cases to adjust the incident angle range when the measured degree of polarization
became relatively small, suggesting we were out of the specular reflection lobe.
In addition, the sensitivity analysis suggests that we need 5-10 measurements in
47
40 45 50 55 60 65 70 75 800
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
angle of incidence
DO
P
Reflection angle is 60 degrees
EstimateExperimentReference
Figure 3.3: DOP plots derived from model and experimental data for green paint.
order to obtain a reasonable estimate for the parameters of interest. Thus, the
example results presented in this section involve several ranges of angles of inci-
dence and step sizes. In general, we have used either 5◦ or 10◦ degree incident
angle steps and have found that our sample materials produce usable specular
patterns that extend over at least a 40 degree range.
Table 3.5 shows the results for the green paint target. For clarity, we mention
again that the angle of incidence refers to the angle between the source and the
object surface normal while the reflection angle refers to the angle between the
camera and the object surface normal. We observe that the estimates for the index
of refraction values agree reasonably well with values published in literature [28].
More importantly, however, the extracted index of refraction values are also largely
48
Table 3.5: Estimation results for green paint target for the fixed camera scenario.The angle of incidence was varied in steps of 10◦. We used n = 1.39 and k = 0.34as a reference [28] for the index of refraction.
Angle of Reflection n k θr
Incidence (◦) Angle (◦) (◦)40-80 60 1.47 0.47 59.8940-80 55 1.46 0.42 54.5940-80 50 1.39 0.41 50.1640-80 45 1.39 0.46 45.48
invariant to the reflection angle over our measurement range. This is critical if
the index is to be used for identifying and classifying materials based on their
polarimetric signatures. In addition, we see that the reflection angle is recovered
accurately for the various cases considered in our experiments. Figure 3.3 shows
the DOP as a function of the angle of incidence for a reflection angle of 60◦ and
illustrates the effectiveness of the Levenberg-Maruquardt method for estimating
the index of refraction for the green paint target. We observe that the DOP curve
corresponding to the experimental data (labeled ”Experiment”) matches well with
the DOP curve (labeled ”Estimate”) that was generated from the pBRDF model
using the estimated index of refraction as the input parameter. We also include
the DOP plot (labeled ”Reference”) generated from the pBRDF model using the
published index of refraction value [28] for comparison. A similar trend is noted
for experiments with the roughened copper piece as shown in figure 3.4. Table 3.6
summarizes the results for the roughened copper piece.
49
35 40 45 50 55 60 65 700
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
angle of incidence
DO
P
Reflection angle is 60 degrees
EstimateExperimentReference
Figure 3.4: DOP plots derived from model and experimental data for copper.
Table 3.7 and figure 3.5 shows the results for the black paint sample. Again,
we observe that the estimates for the index of refraction values deviate slightly
form those published in literature [28]. We note that the differences are not sur-
prising as there are no standard index of refraction values that we are aware of for
paints. Figure 3.5 also illustrates this difference where the reference DOP values
Table 3.6: Estimation results for roughened copper for the fixed camera scenario.The angle of incidence was varied in steps of 5◦. We used n = 0.4 and k = 2.95as a reference [28] for the index of refraction.
Angle of Reflection n k θr
Incidence (◦) Angle (◦) (◦)35-70 60 0.54 3.19 59.5135-70 55 0.53 3.26 54.6335-70 50 0.53 3.32 49.6235-70 45 0.51 3.37 44.85
50
10 20 30 40 50 60 70 800
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
angle of incidence
DO
P
Reflection angle is 60 degrees
EstimateExperimentReference
Figure 3.5: DOP plots derived from model and experimental data for black paint.
are much larger than the measured values. However, the DOP curve obtained us-
ing the estimated index values falls very close to the measurements, which suggests
consistency in our measurements and approach. In addition, table 3.7 shows that
the estimates obtained for the apparent index of refraction are still largely invari-
ant to observational geometry, and we see that the reflection angles are estimated
to a high degree of accuracy in this case as well.
Table 3.8 shows the results for the case of the roughened aluminum piece. We
see that, as with the black painted surface, the estimates do not agree with the
reference index of refraction value [43]. Figure 3.6 shows the DOP values predicted
by the model with the reference inputs are significantly lower than the values
measured in the laboratory. As a result, the estimates differ from the published
51
Table 3.7: Estimation results for black paint target for the fixed camera scenario.The angle of incidence was varied in steps of 10◦. We used n = 1.405 and k =0.2289 as a reference [28] for the index of refraction.
Angle of Reflection n k θr
Incidence (◦) Angle (◦) (◦)10-80 60 1.32 0.63 60.2510-80 55 1.32 0.61 54.9310-80 50 1.33 0.60 50.1010-80 45 1.32 0.61 44.88
index of refraction. The differences may be related to the surface characteristics of
aluminum which can vary depending on oxidation rates. We have noticed changes
in the estimates over time, although we have not done a controlled study of this
phenomenon. More importantly, the estimates obtained for the index of refraction
are still reasonably invariant to observational geometry and the reflection angles
are estimated to a high degree of accuracy in this case as well.
Table 3.8: Estimation results for roughened aluminum for the fixed camera sce-nario. The angle of incidence was varied in steps of 5◦. We used n = 1.24 andk = 6.60 as a reference [43] for the index of refraction.
Angle of Reflection n k θr
Incidence (◦) Angle (◦) (◦)35-80 60 1.37 3.97 59.9935-80 55 1.42 4.02 54.9835-80 50 1.60 4.34 49.9835-80 45 1.81 4.63 44.98
52
35 40 45 50 55 60 65 70 75 800
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
angle of incidence
DO
P
Reflection angle is 60 degrees
EstimateExperimentReference
Figure 3.6: DOP plots derived from model and experimental data for aluminum.
3.5.4 Fixed Illumination Source/Moving Camera Scenario
In this section we present experimental results for the fixed illumination source
case in which the position of the source remains fixed while the position of the
receiver or the angle of reflection varies between each measurement. We present
results for the flat green paint and flat black paint samples in our experiments for
the fixed illumination source scenario. The range of the angles of reflection and
the step sizes are chosen to be consistent with the range of the angles of incidence
employed in the fixed receiver case.
Table 3.9 shows the results for the green paint target. We observe that the
estimates for the index of refraction values agree reasonably well with values pub-
lished in literature [28]. More importantly, however, the extracted index of refrac-
53
Table 3.9: Results for green paint for the fixed source scenario. The angle ofreflection was varied in steps of 10◦. We used n = 1.39 and k = 0.34 as areference [28] for the index of refraction.
Angle of Angle of n k θi
Reflection (◦) Incidence(◦) (◦)40-80 60 1.47 0.39 59.7940-80 55 1.44 0.36 54.6740-80 50 1.41 0.37 49.7340-80 45 1.31 0.44 45.02
tion values are again largely invariant to the angle of incidence or illumination
source direction over our measurement range. To reiterate, this is critical if the
extracted index is to be used to design target discrimination and classification
algorithms that utilize passive polarimetric data. In addition, we see that the
angle of incidence or the illumination source direction is recovered accurately for
all the cases considered in our experiments. Finally, we observe by comparing
tables 3.5 and 3.9 that the estimates for the index of refraction are consistent for
the two scenarios considered in our experiments. This is to be expected as (3.20)
indicates that the DOP observed for a material depends on its index of refraction
and the phase angle. A similar trend is noted for the flat black target. Table 3.10
summarizes the results for the flat black paint sample.
3.6 Summary
In this chapter we present a method that extracts illumination and viewpoint
invariant parameters from multiple passive polarimetric measurements. We con-
54
Table 3.10: Results for black paint for the fixed source scenario. The angle ofreflection was varied in steps of 10◦. We used n = 1.405 and k = 0.2289 as areference [28] for the index of refraction.
Angle of Angle of n k θi
Reflection (◦) Incidence(◦) (◦)10-80 60 1.34 0.65 60.2510-80 55 1.34 0.61 54.1810-80 50 1.35 0.60 50.1110-80 45 1.34 0.59 44.85
sider two scenarios for our measurement set-up: (i) a fixed receiver or camera
scenario and (ii) a fixed illumination source scenario. The estimation algorithm
for both scenarios utilizes an iterative, model-based method to recover the pa-
rameters of interest of a target. The model is built on a microfacet polarimetric
bidirectional reflectance distribution function. The parameters of interest are ex-
tracted using the Levenberg-Marquardt method. A computer simulation-based
sensitivity analysis suggests that the accuracy of the proposed method improves
with an increasing number of measurements. Furthermore, experimental results
with laboratory data indicate that the proposed estimation approach is highly
effective and that the estimates for the index of refraction are largely invariant to
the observational geometry for all the cases considered in our experiments.
55
4 APPLICATIONS OF PROPOSED ESTIMATION APPROACH
In this chapter we develop two image processing applications namely image
segmentation and material classification that utilize the estimation framework pre-
sented in chapter 3. Both applications employ the extracted index of refraction
which is an illumination-invariant parameter as a feature vector for segmenta-
tion and discrimination/classification. We present the segmentation algorithm in
section 4.1 and the material classification method in section 4.2
4.1 Segmentation
Image segmentation refers to the process of grouping pixels in the imaged scene
based on certain coherence attributes such as intensity or color. Scene segmenta-
tion in uncontrolled environments is an important task in many applications such
as target recognition/classification and higher level computer tasks like scene un-
derstanding. The utility of passive polarimetry for scene segmentation was first
demonstrated by Wollf and Boult [9]. They showed that it was possible to segment
a scene based on the material composition of its constituent objects by recording
the polarization state of reflected light using a passive polarimeter. Their method
involves a threshold-based discrimination procedure that relies on the polarization
Fresnel ratio [9], the ratio of perpendicular to parallel polarization state compo-
56
nents. Sadjadi and Chun [44] developed a joint segmentation and classification
algorithm that relied on a statistical similarity criterion called the Fisher criterion
for passive infrared polarimetric imagery. More recently, Goudail et al. [5] have
developed a segmentation method to extract two-dimensional shapes from passive
polarimetric imagery. Their method is based on polygonal active contours and
the minimum description length principle (MDL) [5] and is capable of extracting
complex two-dimensional shapes from polarimetric imagery.
Passive systems in the visible-to-near-infrared (VNIR) use the sun as the
source of illumination. VNIR sensors have potential for widespread use in the
future. However, the observed polarimetric signatures in the VNIR are strongly
dependent on the source, target and imaging geometry. This aspect has not been
considered by the previously proposed scene segmentation methods in literature.
In this section we present a segmentation algorithm for specular targets in passive
polarimetric imagery that is based on our prior work [45] and is robust to illumi-
nation conditions and viewpoint by employing the index of refraction estimated
using the algorithm described in the previous chapter.
4.1.1 Segmentation Methodology
A clustering approach based on the classic c-means algorithm [46] is used to seg-
ment the scene of interest. We note here that the c-means algorithm is more
commonly known as the k-means algorithm in signal processing literature [47].
57
DOP data smoothing
Cluster validation usingsilhouette index
Evaluate segmentationperformance using the
probabilistic Rand index
C-means clustering
Degree of polarization (DOP) data (images)
Clusters
Estimate index of refractionfor all pixels
Figure 4.1: Clustering-based segmentation method.
58
However, the symbol k is used to denote the extinction coefficient in this disser-
tation and use c to denote the number of classes. Figure 4.1 illustrates the steps
utilized for scene segmentation. The DOP imagery (data) is smoothed with a
moving average filter of size 21 pixels by 21 pixels. Our experience with labora-
tory data indicates that this step improves the quality of the DOP estimates. The
smoothed DOP data is then input to the algorithm described in section 3.4 in
order to extract the index of refraction for all pixels in the scene. These estimates
are then clustered by the classic c-means algorithm. The number of clusters c is
determined using a cluster validity index called the Silhouette index [48]. Finally,
the efficacy of the proposed segmentation method is evaluated using the proba-
bilistic Rand index (PRI) metric proposed by Unnikrishnan et al [49, 50]. We
provide a detailed description of each step in the following.
4.1.2 c-means Clustering Method
The extracted index of refraction estimates are clustered using the c-means algo-
rithm with the squared Euclidean distance metric. The c-means algorithm opti-
mally partitions the dataset into c clusters by minimizing the following squared-
error function or the intra-cluster variance [46]:
c∑l=1
∑xq∈Vl
‖xq − μl‖2 (4.1)
59
where xq = [nq kq]t, q denotes the sample index in cluster l and μl denotes
the centroid of cluster l. Note that the subscript l used here is not the same as
the subscript used to describe the elements of the pBRDF matrix in (2.35). The
above minimization is performed by using the well-known Linde-Buzo-Gray algo-
rithm [47]. The c-means algorithm for the squared-error criterion is summarized
below [46, 51]:
1. Assuming there are c classes of interest, initialize cluster centers μ1, μ2, · · · , μc
2. Assign a sample xq to the nearest cluster center using the following rule -
xq belongs to class Vl if
‖xq − μl‖ ≤ ‖xq − μt‖ ; q = 1, 2, · · · , Ns, l = t, t = 1, 2, · · · , c (4.2)
where Ns denotes the number of samples to be clustered and ‖x‖ denotes
the Euclidean norm of x. This rule is just the well-known nearest neighbor
rule [46]
3. Recompute the cluster centers based on the partition obtained in step 2.
4. Repeat steps 2 and 3 until there is little or no change in cluster centers.
The issue of choosing the initial partitions or equivalently the cluster centers
has to be addressed while using the c-means clustering method. There is no known
optimal method of choosing the initial cluster centers. This initialization is done
60
here by selecting vectors randomly from the data set and running the algorithms
many times and observing the robustness of the resulting solutions [46]. Finally,
we note that the c-means algorithm is not guaranteed to converge to the global
optimum [46]. In order to guarantee convergence to the optimal solution, it is
necessary to resort to a stochastic technique such as simulated annealing [46]
which comes at the price of increased complexity. This topic is not addressed in
our work and is left as a part of future work.
4.1.3 Silhouette Index
The c-means algorithm assumes that the number of clusters c is known which
may not be the case in most scene segmentation applications. Consequently, it
is necessary to determine the number of clusters from the dataset in addition
to partitioning the data in an optimal manner. This problem may be solved by
utilizing one of several cluster validation techniques proposed in literature [48,
52, 53]. A cluster validation index runs the clustering algorithm several times
and computes an index that indicates the quality of each resulting partitions.
The objective is to identify the partition for which the considered cluster validity
index is optimal [54].
In this paper, we use a cluster validity index called the Silhouette index [48]
in order to determine the number of clusters. We now review the Silhouette
index in the following. For a given cluster Vl (l = 1, 2, · · · , c), the silhouette
61
index method assigns a quality measure called the silhouette width denoted by
s(r) (r = 1, 2, · · · , m). The Silhouette width indicates the confidence of assigning
the rth sample in cluster Vl and is defined as:
s(r) =b(r) − a(r)
max(b(r), a(r)), (4.3)
where a(r) is the average distance between the rth sample and all the other sam-
ples in Vl, b(r) is the minimum average distance between the rth sample and all
the other samples in Vp (p = 1, 2, · · · , c; p = l) and max denotes the maximum
operator. It is clear from (4.3) that −1 ≤ s(r) ≤ 1. The silhouette width is used
to define a measure called the cluster Silhouette index Sl for cluster Vl which is
given by
Sl =1
m
m∑r=1
s(r), (4.4)
where m is the number of samples in cluster Vl. Finally, the global silhouette value
or the silhouette index for a partition U of the given dataset is given by [48, 54]
GSu =1
c
c∑l=1
Sl, , (4.5)
where the subscript u denotes that the index is computed for a partition U .
Furthermore, it has been shown that the global Silhouette index can be used to
estimate the number of clusters for a given dataset [54]. This is done by choosing
the partition with the maximum silhouette index value.
62
4.1.4 Probabilistic Rand Index (PRI)
The performance of the proposed segmentation method is evaluated by using the
probabilistic Rand index (PRI) metric proposed by Unnikrishnan et al [49, 50].
Their metric assumes the availability of a set of ground truth segmentations of
an image instead of an unique ground truth segmentation to compare with the
output of a segmentation method. In general, two human ”segmenters” do not
agree with the location of boundaries between objects in a scene. Furthermore, a
practical imaging system suffers from effects such as noise and diffraction which
also leads to uncertainty in the location of the boundary pixels. The ambiguity in
the location of edges in an image can be described by using multiple images for
ground truth data. The probabilistic Rand index is robust to small shifts in the
location of the boundary pixels and utilizes all the images in the manually labeled
dataset to evaluate the performance of a segmentation algorithm. The PRI ranges
between 0 and 1 where a higher score indicates better segmentation performance.
We provide a brief description of the PRI and refer the interested reader to the
papers by Unnikrishnan et al [49, 50] for both theoretical and implementation
details of this measure.
The probabilistic Rand index is based on the Rand index [55] which is a mea-
sure of similarity that compares two partitions of a given dataset. This similarity
measure converts the problem of comparing two partitions into a problem of com-
63
paring pairwise label relationships. The term label refers to the name given to a
category or class in a dataset. Note that the Rand index does not assume that
the number of classes in the two partitions are equal. Suppose we have two valid
label assignments U and U′with corresponding labels {li} and
{l′i
}of Ns points
X = {x1, x2, · · · , xi, · · · , xNs}. The Rand index RI is given by the ratio of the
number of pairs of points having compatible label relationship in U and U′. This
is mathematically given by
RI(U, U
′)
=1(N2
)∑i,ji�=j
[I(li = lj ∧ l
′i = l
′j
)+ I(li = lj ∧ l
′i = l
′j
)](4.6)
where I denotes the identity function and(
N2
)is the number of possible unique
pairs among Ns data points. The Rand index ranges from 0 when the two parti-
tions are completely dissimilar to 1 when the segmentations are identical.
The probabilistic Rand index (PRI) generalizes the Rand index to the case
where multiple ground truth segmentations are available instead of a single ground
truth segmentation. Let {U1, U2, · · · , UM} denote the set of ground truth data
available for an input dataset or image X = {x1, x2, · · · , xNs}. Let Utest be the
candidate segmentation that needs to be compared with the ground truth set. Let
lUtesti and lUm
i denote the label of point or pixel xi in segmentation Utest and Um
respectively.
Unnikrishnan and Herbert [49] and Unnikrishnan et al. [50] hypothesize that
64
the label relationships for each pixel pair is modeled by an unknown underlying dis-
tribution. This may be visualized as the scenario where each ”segmenter” provides
information about segmentation Um in the form of binary numbers I(lUmi = lUm
j
)for each pixel pair (xi, xj). Thus, the set of all valid segmentations defines a
Bernoulli distribution over this number with expected value pij. The probablistic
Rand index (PRI) is defined as :
PRI (Utest, {Um}) =1(N2
)∑i,ji<j
[cijpij + (1 − cij) (1 − pij)] (4.7)
where cij denotes the event that pixel pair xi and xj have the same label in Utest
or cij = I(lUtesti = lUtest
j
). This measure ranges from 0 which corresponds to the
case where Utest and {U1, U2, · · · , UM} have no similarities to 1 which implies that
all segmentations are identical. Unnikrishnan et al. [50] also provide an efficient
means to compute pij . The proposed estimator computes the sample mean of the
corresponding underlying Bernoulli distribution and is given by
pij =1
M
M∑m=1
I(lUmi = lUm
j
). (4.8)
The resulting PRI computation reduces to computing the mean of the Rand index
between each pair of images (Utest, Um) as shown by Unnikrishnan et al. [50]. Fig-
ure 4.2 illustrates PRI computation for the case where there is a single boundary
between two objects. We assume that the two objects are separated by a horizon-
tal line and that the uncertainty in location of the boundary is three pixels wide.
As a result, the ground truth set has three images each corresponding to one of
65
Uncertainty in location of
boundary pixels
Ground truth dataset
Candidatesegmentation
RI1RI2 RI3
Figure 4.2: PRI computation when there is an ambiguity in boundary location.RIi, i = 1, 2, 3 refers to the Rand index computed between ground truth segmen-tation i and the candidate segmentation.
the boundary pixel locations. In this example, the PRI is given by the mean of
the Rand index computed between the candidate segmentation and each of the
image in the set of ground truth data. Unnikrishnan et al. have shown that the
computational complexity of the PRI is linear in Ns or the number of pixels in
the test image (or ground truth set) which makes it attractive to implement in
practice. The interested reader is referred to Unnikrishnan and Herbert [49] and
Unnikrishnan et al. [50] for further details on the proposed measure.
66
4.2 Material Classification
Material classification refers to the group of algorithms that classify objects based
on the material composition. Material classification has been shown to be useful
for many tasks including industrial applications such as printed circuit board
inspection [8, 56, 57], cartographic feature extraction [58], target recognition and
geospatial database construction [59].
As with scene segmentation, material classification and recognition in uncon-
trolled environments is an important task in remote sensing applications such
as target recognition, image segmentation and shape extraction as well as in in-
dustrial inspection. The utility of passive polarimetry for material classification
was first demonstrated by Wolff [8]. He showed it was possible to distinguish
between metals and dielectrics by recording the polarization state of specularly
reflected light using a passive polarimeter. His method involves a threshold-based
discrimination procedure that relies on the polarization Fresnel ratio [8], the ratio
of perpendicular to parallel polarization state components. More recently, Zal-
lat, Grabbling and Takakura [60] have proposed a clustering-based approach for
material classification using passive polarimetric imagery.
Passive polarization-based material classification methods that are currently
available in literature do not consider the dependency of the observed signatures
on source, imaging and target geometry. Consequently, these methods are likely to
67
fail in outdoor remote sensing scenarios where one has no control over the illumi-
nation source and the target geometry. In this section, we present a methodology
for material classification that is an extension of our prior work [61] and is robust
to illumination conditions.
4.2.1 Classification Algorithm
We employ the framework developed in chapter 3 to extract feature vectors for
classifying target objects based on their material properties. In particular, the
proposed material classification method utilizes the extracted index of refraction.
This feature vector is a fundamental parameter that describes a material and is
thus naturally suited for material classification. We note here that the proposed
classifier is robust to illumination conditions as it employs the extracted index of
refraction which is useful in certain passive remote sensing applications such as
discriminating between targets and decoys. The classification method is based on
the well-known nearest neighbor method [46] with the Euclidean distance metric
for classification. This method classifies a test vector by assigning it to the class
associated with the nearest prototype in the training set.
4.3 Experimental Results
We validate the proposed segmentation and material classification algorithms with
passive polarimetric data collected under laboratory conditions. The polarimetric
68
5 10 15 20 25 30 35 40 45 50
10
20
30
40
50
60
70
80
90
100
(a) Intensity image for black paint target andbackground (θi = 60◦, θr = 60◦)
5 10 15 20 25 30 35 40 45 50
10
20
30
40
50
60
70
80
90
100
(b) Segmented Image
Figure 4.3: Segmentation results for black paint sample (θr = 60◦).
image data is collected with the EORL polarimeter described in section 3.5 of
chapter 3. We first present the results for the segmentation algorithm and then
those for the material classification results in section 4.3.2.
4.3.1 Segmentation Results
The target samples in our experiments for validating the proposed segmentation
algorithm include Styrofoam pieces coated with flat black paint and flat green
paint and a sandblasted aluminum piece. The root-mean square surface roughness
of the samples was measured using a Surfcom 120A stylus profilometer to be 2.99
microns for flat black paint, 2.58 micros for flat green paint and 3.3 microns for
aluminum. Thus, the RMS surface roughness in all cases is greater than the
wavelength of interest (650 nm), which satisfies the assumption made by the
69
5 10 15 20 25 30 35 40 45 50
10
20
30
40
50
60
70
80
90
100
(a) Intensity image for black paint target andbackground (θi = 50◦, θr = 50◦)
5 10 15 20 25 30 35 40 45 50
10
20
30
40
50
60
70
80
90
100
(b) Segmented Image
Figure 4.4: Segmentation results for black paint sample (θr = 50◦).
Torrance-Sparrow model [22] that underlies the pBRDF model employed in our
work.
Figure 4.3 illustrates the segmentation results obtained for the black paint
sample. The polarimetric data needed to analyze this sample is collected by
fixing the reflection angle θr to 60◦ and varying the angle of incidence from 10◦
to 80◦ in steps of 10◦. The imaged scene consists of a black paint sample in
the foreground affixed to a black cardboard piece that acts as the background as
illustrated in figure 4.3(a). We consider a 100 × 50 section of the imaged scene
in order to improve the execution speed and figure 4.3(b) illustrates the resulting
segmented image. We observe from the figure that the proposed algorithm is able
to identify the target and background areas to a high degree of accuracy. This fact
is supported by a PRI score of 0.9583 for the black sample as listed in table 4.1.
70
Table 4.1: PRI scores for the samples used in experiments.
Sample Reflection angle PRIBlack paint 60◦ 0.9583Black paint 50◦ 0.9669Green paint 60◦ 0.7987Aluminum 60◦ 0.9575
Scene 60◦ 0.9226
10 20 30 40 50 60 70 80
10
20
30
40
50
60
70
80
(a) Intensity image for green paint sample andbackground (θi = 60◦, θr = 60◦)
10 20 30 40 50 60 70 80
10
20
30
40
50
60
70
80
(b) Segmented Image
Figure 4.5: Segmentation results for green paint sample.
We repeat the above experiment with the reflection angle now fixed at 50◦.
Figure 4.4 illustrates the segmentation results for this experiment. We observe
from figure 4.4(b) that the proposed algorithm is able to distinguish between the
painted surface and background to a high degree of accuracy in this case well.
The accuracy is quantified by the high PRI score of 0.9699 as noted in table 4.1.
Figure 4.5 shows the segmentation results obtained for the green paint sample.
The polarimetric data is collected with the reflection angle fixed at 60◦ while the
71
angle of incidence is varied from 40◦ to 80◦ in steps of 10◦. The range for the
angle of incidence is chosen due to reasons described in chapter 3. The imaged
scene consists of the Styrofoam piece coated with green paint affixed on a black
cardboard piece similar to those used in the experiments for black paint. We
consider a 80 × 80 pixel cross-section in this analysis and figure 4.5(b) shows the
results of the segmentation algorithm. In this case we observe from figures 4.5(a)
and 4.5(b) that the segmentation algorithm incorrectly identifies portions of the
green paint target as background. Note that the contrast is poor in the intensity
image (figure 4.5(a)) as we have reduced the image depth from 12 bits to 8 bits in
order to display the images with a gray scale map. We note here that the sample
does not have an uniform coat of green paint which causes inconsistencies in the
measured DOP values. However, the segmentation algorithm performs reasonably
well in this example as well which is supported by a PRI score of 0.7987.
Figure 4.6 illustrates the performance of the segmentation method for the
aluminum sample. The aluminum sample is affixed to a black cardboard piece
similar to the piece used in the previous examples. In this case, the reflection
angle is fixed at 60◦ and the angle of incidence is varied from 35◦ to 80◦ in steps
of 5◦. We observe from figure 4.6(b) that once again the segmentation method
identifies the foreground and the background to a high degree of accuracy. This
is supported by a high PRI score of 0.9575 as shown in table 4.1. These results
indicate that the apparent index of refraction is indeed an effective feature vector
72
5 10 15 20 25 30 35 40 45 50
20
40
60
80
100
120
140
160
180
200
(a) Intensity image for aluminum sample andbackground (θi = 60◦, θr = 60◦)
5 10 15 20 25 30 35 40 45 50
20
40
60
80
100
120
140
160
180
200
(b) Segmented Image
Figure 4.6: Segmentation results for aluminum sample.
for illumination-invariant scene segmentation.
Figure 4.7 illustrates the performance of the proposed algorithm for a scene
consisting of a black painted target affixed on black ground. The polarimetric
data needed to analyze this sample is collected by fixing the reflection angle θr
at 60◦ and varying the angle of incidence from 10◦ to 80◦ in steps of 10◦. We
observe from figures 4.7(a) and 4.7(b) that the proposed method is able to isolate
the object from the background in this case well. Note from figure 4.7(a) that
there is some variability in the intensity image due to non-uniformity of the target
surface, specular highlights and shadows. These effects can cause a segmentation
algorithm that employs intensity data to fail whereas the proposed method is still
able to identify the target and the background reasonably well. This is supported
a high PRI score of 0.9226 as indicated in table 4.1.
73
S0 image: θ
i = θ
r = 60°
(a) Intensity image for a scene consisting of ablack paint target and background.
Segmented scene
(b) Scene segmentation result.
Figure 4.7: Scene segmentation results for black paint sample.
Table 4.2 shows the effective or apparent index of refraction values derived by
computing the centroid of the clusters corresponding to black paint, green paint
and aluminum targets. Nominal values for the true index of refraction obtained
from literature are also included for comparison [28, 43]. We observe from table 4.2
that the estimated index of refraction for the black paint sample agrees reasonably
well with published values for both viewpoints (reflection angles) considered in
our experiments. A similar observation is made for the green paint sample. The
estimate for the aluminum sample is somewhat lower than the corresponding value
obtained from literature. However, this is not a major drawback as we can still
design classification algorithms that are robust to illumination source position
which is the ultimate goal of our work. The utility of the index of refraction
estimates for classification are demonstrated in the next section.
74
Table 4.2: Index of refraction values computed by segmentation method. Thesevalues are estimated by computed the centroid of clusters corresponding to thetargets listed below. n, k denote the reference index of refraction values obtainedfrom [28, 43] while n and k denote the estimates.
Material Angle of Reflection n k n kIncidence (◦) Angle (◦)
Black paint 10-80 60 1.41 0.23 1.3148 0.6303Black paint 10-80 50 1.41 0.23 1.3132 0.6013Green paint 40-80 60 1.39 0.34 1.4836 0.4273Aluminum 35-80 60 1.24 6.60 0.9403 3.8359
We observe from table 4.2 that the index of refraction estimates for black paint
are almost identical for the two reflection angles considered in our experiments
which confirms that extracted index is indeed reasonably invariant to illumina-
tion and observer position. This is a desirable attribute in many remote sensing
applications. Our estimation algorithm also provides us with an estimate of the
reflection angle in addition to the index of refraction. The reflection angle can also
be used as a feature for angle-based scene segmentation. Thus one may able to
segment a scene that contains an object that has two facets oriented in different
directions. However, the estimation algorithm presented in section 3.4 cannot be
directly used in scenes that contains two different reflection angles. Instead we
provide results that indicate that the reflection angle can also be used a feature for
scene segmentation. In this experiment, we make two measurements of the black
paint sample on black cardboard background with reflection angles at 60◦ and 50◦
respectively. Tables 4.2 and 4.3 show the results for this experiment. We see from
75
Table 4.3: Reflection angle estimates for black paint target. The values presentedbelow are obtained by averaging the reflection of all pixels that were classified asblack paint.
Sample True θr Mean θr Variance θr
(◦) (◦) (◦)Black paint 60 60.1261 0.0125Black paint 50 49.8872 0.0262
table 4.2 that the extracted index of refraction for both measurements are almost
identical and hence cannot be used for discrimination. It is clear, however, from
table 4.3 that the estimated reflection angle can be used for scene segmentation
in this scenario and in fact provide accurate estimates for the reflection angle.
4.3.2 Material Classification Results
We now present the results obtained for the material classification algorithm de-
scribed in section 4.2. A Styrofoam piece coated with flat green paint, a Styrofoam
piece coated with flat black paint, a piece of roughened aluminum and a piece of
black cardboard that was used as the background for the segmentation algorithm
is used as the targets in our experiments.
The degree of polarization (DOP) values needed for estimation were computed
by averaging at least 100 by 100 pixels in the Stokes vector images. The index
of refraction estimates are obtained for the green paint and aluminum target
with reflection angle fixed at 60◦, the black target and the background class with
reflection angle fixed at 60◦ and 50◦.
76
Table 4.4: Prototype index of refraction values used by proposed classifier. Thereflection angle is 60◦. n, k denote the reference index of refraction values obtainedfrom [28, 43] while n and k denote the estimates. Note that no reference index isavailable for the black cardboard piece used in our experiments.
Material Angle of n k n kIncidence (◦)
Green paint 40-80 1.39 0.34 1.4836 0.4273Aluminum 35-80 1.24 6.60 0.9403 3.8359Black paint 10-80 1.41 0.23 1.3148 0.6303Background 10-80 - - 1.1831 0.9896
Table 4.4 shows the index of refraction values that are used as prototype
vectors by the nearest neighbor algorithm. These values correspond to the centroid
of the clusters as estimated by the segmentation algorithm in section 4.1. The
measurements for estimating the prototype index of refraction values are made
with the reflection angle fixed at 60◦. The index of refraction values published
in literature [28, 43] have also been included in table 4.4 for comparison. Note
that no refractive index is available for the black cardboard piece used in our
experiments. The choice for the ranges of the angle of incidence for the samples
in the training and test sets are selected to be consistent with the those used in
chapter 3 (Refer to section 3.5 for a detailed explanation).
Figure 4.8 shows a scatter plot of the apparent index of refraction estimates
used in our experiments for classification. The plot also shows the apparent index
of refraction estimates used as prototype vectors by the nearest neighbor classifier
(shown with a bold marker). We observe from figure 4.8 that the clusters corre-
77
0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
n
k
AluminumBlack paintBackgroundGreen paint
Figure 4.8: Target clusters and their respective centroids.
sponding to aluminum and green paint target appears to be well separated from
those corresponding to black paint target and the background while the clusters
corresponding to the black paint target and the background appear to have some
overlap. The classification results are tabulated in table 4.5 in a matrix format
and presented in figure 4.9 as a scatter plot with the misclassified points being in-
dicated by black dots. The rows and columns of the confusion matrix corresponds
to the different target classes. The entries along the diagonal of this matrix cor-
responds to the classification accuracy while the off-diagonal elements indicates
the percentage of misclassified points or pixels. We observe from table 4.5 that
the points corresponding to the aluminum and black paint target are all classi-
fied accurately. In addition, we note that about 2% of the background pixels are
misclassified as black paint which is expected given the lack of separation in the
scatter plots. We also see, however, that about 8% of pixels corresponding to the
78
Table 4.5: Classification result using the nearest neighbor classifier. The resultsare presented as a confusion matrix. The first column corresponds to the trueclass labels while the first row corresponds to the labels assigned by the classifier.
Class label Aluminum Black paint Background Green paintAluminum 1 0 0 0Black paint 0 1 0 0Background 0 0.0201 0.9799 0Green Paint 0 0.0839 0 0.9161
green paint target are misclassified as black paint. This is somewhat unexpected
given that the clusters corresponding to the black paint and green paint target
are visibly separated. This result indicates that some of the points in the green
paint cluster are in fact closer (or ”similar”) in the Euclidean distance sense to
the prototype vector corresponding to the black paint target than to that corre-
sponding to the green paint target. This fact is illustrated in figure 4.10 where we
plot the decision boundary - i.e., the dividing line between the black and green
paint target classes. The region corresponding to black paint is indicated by dark
circles while the region corresponding to the green paint is shown with no mark-
ers in figure 4.10, and the decision boundary itself is shown by a thick split line.
The misclassified points are clearly seen in figure 4.10 (indicated with a black dot
marker). Finally, we note that the overall accuracy of the proposed classifier is
about 98%. Note that this accuracy cannot be directly inferred from table 4.5
since the number of pixels or test vectors used for each class is not the same.
79
0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
n
k
AluminumBlack paintBackgroundMisclassified background pixelGreen paint
Figure 4.9: Result obtained by using the nearest neighbor classifier.
4.4 Summary
In this chapter we present two image processing applications, namely image seg-
mentation and material classification, that utilizes the framework presented in
chapter 3 to extract illumination-invariant parameters. The applications utilize
the iterative, model-based method described in chapter 3 to extract the index of
refraction of specular objects from multiple passive polarization measurements for
purposes of segmentation and discrimination/classification.
The proposed segmentation method first extracts the index of refraction on
a pixel-by-pixel basis and then clusters pixels by using the well-known c-means
algorithm. The number of clusters is automatically determined by utilizing the
silhouette index and the performance of the segmentation method is evaluated
using the probabilistic Rand index metric. Experimental results with laboratory
data indicate that the proposed method is highly effective for segmenting various
80
1 1.2 1.4 1.6 1.8 20
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
n
k
Figure 4.10: Decision boundary for black and green paint targets. We can alsoobserve the boundary regions for the two targets.
targets of interest. Moreover, the proposed algorithm is also capable of extracting
two-dimensional shapes. Most importantly, the proposed segmentation method is
robust to a wide range of illumination conditions considered in our experiments.
The material classification method also utilizes the extracted index of refrac-
tion as a feature vector for discriminating between objects of interest. The ex-
tracted feature vector is used as an input for the nearest neighbor method to
discriminate between objects of interest. Experimental results with laboratory
data indicate that the classification approach is highly effective for distinguishing
between various targets of interest. Finally we note that the presented classifica-
tion method can be combined with the segmentation method in a straight-forward
manner to perform joint segmentation and classification from passive polarimetric
imagery which is of interest in remote sensing.
81
5 ESTIMATION FRAMEWORK FOR GENERALIZED SCATTER-
ING MODEL
The pBRDF model considered thus far in this dissertation accounts only for
the specular component of optical scattering and ignores the contribution from
volumetric scattering. Thus the model presented in chapter 2 assumes that light
undergoes a single reflection from the surface under observation before it reaches
the receiver. In reality, light often undergoes multiple reflections as well as refrac-
tion from medium (assumed to be air) into the object which gets refracted back
into the medium before it reaches the observer. The latter effects are collectively
termed volumetric scattering, and ideally they need to be accounted for by a scat-
tering model. In this chapter, we lay the groundwork for such a model. We use
the so called Maxwell-Beard model proposed by Maxwell et al. [24] to account
for volumetric scattering. In this model, the volumetric component of scattering
is assumed to be completely depolarizing. The Maxwell-Beard model contains
two parameters, namely the diffuse scattering parameter ρD and the volumetric
scattering parameter ρV . This nomenclature is adopted from the nonconventional
exploitation factors (NEF) document [40]. The adopted nomenclature is confusing
as the two parameters are in fact both used to explain volumetric scattering. We
observe that the depolarizing assumption for the volumetric scattering component
is not true in general as was observed by Ellis [62]. Germer [63] has developed a
82
physics-based pBRDF model for explaining polarization due to volumetric scat-
tering which to our knowledge is the only work to date in this area. However,
Germer’s model relies on Monte Carlo simulations to predict polarization which
limits its utility in our work.
5.1 pBRDF Model for Specular and Volumetric Scattering
The pBRDF matrix F is generalized as
F = μFspec + (1 − μ)Fvol (5.1)
where Fspec is the pBRDF Mueller matrix for specular scattering described in
chapter 2, Fvol pBRDF Mueller matrix for volumetric scattering and μ is the
mixing parameter. According to this model, both specular and volumetric scat-
tering contribute to the observed signal at the camera. The amount of each
component is dictated by the mixing paratmeter μ where 0 ≤ μ ≤ 1. If μ is 1,
then the above model reduces to the model described in chapter 2. The model is
(5.1) is reasonable since the energy collected by a given pixel is averaged over a
region of the material surface so that it will contain a linear mixture of specular
and volumetric components.
The volumetric scattering component is assumed to be completely depolarizing
83
as described before and consequently Fv given by
Fvol =
⎛⎝ f vol00 0 00 0 00 0 0
⎞⎠ . (5.2)
The functional form for f vol00 is adopted from the Maxwell-Beard model [24]
and is given by
f vol00 = ρD +
2ρV
cos (θi) + cos (θr)(5.3)
where ρD is the diffuse or Lambertian component, ρV is the volumetric scattering
parameter, θi and θr are the angle of incidence and the reflection angle respectively.
Observe that (5.3) reduces to a simple Lambertian model when ρV = 0.
The goal of the work in this chapter is to estimate these parameters using
passive polarimetric measurements. These estimates can then again be used to
design target discrimination and classification algorithms. Consequently, it is
desirable to estimate the volumetric scattering parameters described above as
this would provide additional information that could be used as feature vectors for
building image processing algorithms. However, the estimation problem gets more
complicated due to added degree of non-linearity introduced here. Instead, we here
first explore the idea of estimating the mixing parameter from passive polarimetric
data assuming complete knowledge of other parameters including the complex
index of refraction, surface roughness, angle of incidence, angle of reflection, diffuse
and volumetric scattering parameter. This situation is highly idealized and very
unlikely to be encountered in practice. However, this study provides an answer
84
about the potential viability of the proposed method for parameter estimation.
The problem of estimating the other parameters is left as a part of future work.
5.2 Mixing Parameter Estimation
The objective here is to recover the mixing parameter μ from passive polariza-
tion measurements. As before, we assume that scattering occurs in the plane of
incidence. The observed Stokes vector at the camera is given by⎛⎝ sr0
sr1
sr2
⎞⎠ =
⎛⎝μ
⎛⎝ f spec00 f spec
10 0f spec
10 f spec00 0
0 0 f spec22
⎞⎠+ (1 − μ)
⎛⎝ f vol00 0 00 0 00 0 0
⎞⎠⎞⎠⎛⎝ 100
⎞⎠(5.4)
and that the observed Stokes vector is given by⎛⎝ sr0
sr1
sr2
⎞⎠ =
⎛⎝ μf spec00 + (1 − μ) f vol
00
μf spec10
0
⎞⎠ (5.5)
which implies that the degree of polarization is
P (n, k, σ, ρD, ρV , θi, θr, μ) =μf spec
10
μf spec00 + (1 − μ) f vol
00
. (5.6)
Note that the DOP is a function of the complex index of refraction (n, k), surface
roughness σ, angle of incidence θi, reflection angle θr, diffuse scattering parameter
ρD, volumetric scattering term ρV and the mixing parameter μ.
The objective here is to estimate the mixing parameter μ from passive polari-
metric measurements assuming complete knowledge of the other parameters. We
adopt a nonlinear least squares method for parameter estimation. The mixing
85
Table 5.1: Simulation results for mixing parameter estimation. The angle ofincidence is varied from 10◦ to 80◦ in steps of 10◦. ρV is assumed to be zero inour simulation.
n k σ θr ρD μ μ(◦)
1.52 0.11 0.0079 60 0.0032 0.5 0.50231.24 6.60 0.1 60 0.0056 1 11.24 6.60 0.1 60 0.0056 0.75 0.751.24 6.60 0.1 60 0.0056 0.25 0.251.24 6.60 0.1 60 0.0056 0 -2.14e-015 (≈ 0)
parameter is estimated from multiple passive polarimetric measurements where it
is assumed that the source moves between measurements. We use the Levenberg-
Marquardt method which is available in the Matlab Optimization Toolbox [64] for
parameter estimation. The following constraint that pertains to the value of μ can
be used to verify the estimate provided by the Levenberq-Marquardt algorithm:
0 ≤ μ ≤ 1. (5.7)
5.3 Simulation Results and Discussion
The performance of the proposed method is analyzed via computer simulations.
We present results for synthetic datasets, i.e., the ground truth is generated using
(5.6). Table 5.1 shows the results that are representative of the various cases
considered in our experiments. The first example corresponds to the case where
the object is assumed to be a painted surface. The other examples assume that the
target of interest is a metal (aluminum). We observe that the mixing parameter
86
μ is accurately recovered in all the cases considered in the experiments. Observe
that the second example corresponds to a purely scattering case as μ = 1. We see
that the mixing parameter is estimated accurately to be equal to one for this case.
Finally, we note that the Levenberg-Marquardt method compute μ correctly even
for the purely volumetric scattering case where the target is purely Lambertian
body. Although at first sight this appears to be a surprising result, however, in
fact this result is trivial. In this case, the Levenberg-Marquardt algorithm solves
for a system of nonlinear equations that evaluate to zero which implies that the
only solution to this set of equations is the trivial solution μ = 0. This case
corresponds to the scenario where polarization provides no information and hence
we will have to resort to intensity information to make any inference about the
underlying parameters. In fact, for this example, we can only compute the diffuse
scattering parameter ρD which is equal to s0.
In closing, we have presented an iterative method to recover the mixing pa-
rameter from multiple passive polarimetric measurements. This method assumes
complete knowledge about the material properties of the target as well as the
geometry and estimates the mixing parameter using the Levenberg-Marquardt
algorithm. We observe that the algorithm works well for most cases considered in
our experiments. However, the proposed method becomes numerically unstable
for very low value surface roughness parameter value (about σ = 0.0001). This is
related to the fact that the surface roughness parameter appears in the denomina-
87
tor of (2.35) as a result of which we run into numerical instability issues. In fact,
a thorough sensitivity analysis needs to be conducted to better understand the
limitations of this algorithm. This is left as a topic to be explored in the future.
88
6 CONCLUSION AND FUTURE WORK
In this dissertation, we have developed a framework for estimating illumi-
nation and viewpoint invariant parameters from multi-look passive polarimetric
imagery. The term multi-look refers to multiple polarization measurements where
either the position of the illumination source (typically the Sun in passive sys-
tems) or the camera changes between measurements. The proposed estimation
algorithm relies on a microfacet polarimetric bidirectional reflectance distribution
function (pBRDF) that models the specular component of optical scattering from
surfaces. The extracted parameter has been used to design an image segmenta-
tion and material classification method that is robust to illumination conditions.
Finally, we briefly explore the topic of using an extended model that considers
volumetric scattering in addition to specular scattering for parameter estimation.
The estimation algorithm is developed for two measurement scenarios: (i)a
fixed receiver or camera scenario and (ii) a fixed illumination source scenario. The
proposed approach for both scenarios utilizes an iterative, model-based method to
recover the parameters of interest of a target. The model is built on the microfacet
pBRDF representation proposed by Priest and Meier [15] and the parameters
of interest are extracted using the Levenberg-Marquardt method. A computer
simulation-based sensitivity analysis suggests that the accuracy of the proposed
method improves with an increasing number of measurements. Furthermore, ex-
89
perimental results with laboratory data indicate that the proposed estimation
approach is highly effective and that the estimates for the index of refraction are
largely invariant to the observational geometry for all the cases considered in our
experiments.
We apply the proposed estimation framework to develop two image processing
applications namely image segmentation and material classification. Both ap-
plications extract the index of refraction on a pixel-by-pixel basis from a set of
passive polarimetric imagery recorded for the recorded scene. These applications
are highly robust to illumination conditions since they utilize the extracted in-
dex of refraction as a feature vector. We also discuss the problem of angle-based
segmentation based on the estimated reflection angle and how the proposed ap-
proach might be applied to solve this problem. This discussion highlights the
potential utility of passive polarimetric imagery for extracting two-dimensional
and eventually three-dimensional shape information.
Finally, we explore the idea of utilizing an extended pBRDF model that ac-
counts for both specular and volumetric scattering for parameter estimation. The
extended model assumes that the the observed Stokes vector contains a linear
mixture of specular and volumetric scattering components. We study the prob-
lem of estimating the mixing parameter value assuming that we have complete
knowledge of all the other model parameters. A computer-based simulation study
indicates that the proposed algorithm shows promise for estimating the mixing
90
parameter.
There are several avenues of research to pursue for future work. The focus of
this project has been to develop image processing algorithms that employ passive
polarimetric imagery. Polarization is a geometry-dependent quantity and conse-
quently can be exploited for extracting shape information from scenes. In this dis-
sertation, we have assumed that the imaged scene contains a single planar surface
as well as in-plane scattering. The proposed estimation method needs to be ex-
tended to accommodate a multi-faceted object in order to extract two-dimensional
shape information. Note that it is possible to extract only two-dimensional shape
information for the considered geometry. The next line of inquiry to pursue would
be to extend the estimation framework to the more general case where the az-
imuthal angle is no longer 180◦. We have already started work on extending the
estimation framework to the ”three-dimensional” case [65]. This extended frame-
work can be used to extract three-dimensional shape information which is useful
in many applications such as surface inspection and scene reconstruction.
Another useful line of inquiry to pursue is related to the extended pBRDF
that models both specular and volumetric scattering. We have proposed a method
to estimate the mixing parameter by assuming complete knowledge of all other
model parameters. Hence, this method has limited practical utility and has to be
adapted to estimate other model parameters as well in order to be useful in more
realistic scenarios. A sensitivity analysis also needs to be performed in order to
91
understand the limitations of the extended model for parameter estimation.
Thus far, the estimation method developed in our work has been applied only
to passive remote sensing systems. It is straight-forward, however, to generalize
the estimation framework to the case of active illumination. In this case, the
scene of interest is illuminated with a polarization-controlled high-intensity source,
possibly a laser in an actual application. We would therefore be able to capture
the entire pBRDF matrix (16 elements) as opposed to only a maximum of three
elements that can be captured in passive systems. Consequently, we would have
more observable data which would likely assist us in the parameter estimation
process.
92
APPENDICES
APPENDIX A
NOTE ON THE COMPLEX INDEX OF REFRACTION
We noted in chapter 2 that the complex index of refraction arises in conducting
media due to the complex nature of their dielectric constant. In this appendix,
we provide a brief derivation of the complex nature of both the dielectric constant
and the refractive index starting from Maxwell’s equations [30]. We adopt the
naming convention used in Born and Wolff [30]. Note that the convention used to
denote symbols here only applies to this appendix. The electromagnetic field is
represented by two vectors �E and �B, called the electric vector and the magnetic
induction respectively. Additionally, let �j, �D, and �H denote the electric current
density, electric displacement and magnetic vector respectively, which are needed
to describe the effect of the electromagnetic field on objects.
Consider a homogeneous isotropic medium of dielectric constant ε, magnetic pe-
meability μ, and specific conductivity σ. We introduce the following equations
that are also referred to as the material equations [30]:
�j = σ�E (A-1)
�D = ε�E (A-2)
�B = μ�H. (A-3)
94
The Maxwell’s equations in this case can be shown to be [30]
curl �H − ε
c�E =
4π
cσ�E (A-4)
curl �E − μ
c�H = 0 (A-5)
div �E =4π
ερ (A-6)
div �H = 0 (A-7)
where the symbol dot denotes the operation of taking derivative with respect to
time (t), div and curl are the usual differential vector operators (for example Reitz
et al. [66] have a definition and of these operators), ρ is the electric charge density
and c is the velocity of light in vacuum. It is possible to shown that ρ = 0 for
metals, which implies that
div �E = 0. (A-8)
Using (A-8), �H from (A-1) and (A-2) we have that
∇2 �E =με
c2�E +
4πμσ
c2�E (A-9)
where ∇2 denotes the Laplacian operator [66].
For the case of plane waves propagating in media, i.e., if �E has the form E0 exp (−iωt),
(A-9) can be simplified to [30]
∇2 �E + kw2�E = 0, (A-10)
where
kw2
=ω2μ
c2
(ε + i
4πσ
ω
). (A-11)
95
We pause here to note that the equation corresponding to (A-10) for non-conducting
media (for e.g. dielectrics) is given by [30]
∇2 �E + k2w�E = 0, (A-12)
and
k2w =
ω2με
c2. (A-13)
where kw is the wave number. This quantity is usually denoted by k in litera-
ture [30] but we use the subscript w as we prefer to use k to denote the imaginary
part of the complex refractive index. The refractive index, which is the ratio of
the velocity of light in vacuum to that in a (non-conducting) medium, is given by
n =c
v=
√με (A-14)
and is related to kw as follows:
n =c
ωkw. (A-15)
We observe that (A-10) and (A-11) are identical to (A-12) and (A-13) if the
dielectric constant is replaced by
ε = ε + i4πσ
ω. (A-16)
The above equation represents the complex nature of dielectric constant in con-
ducting media such as metals mentioned in chapter 2. The term kw in (A-11)
denotes the complex analog of the wave number in (A-13). We can now define
96
the complex index of refraction analogous to (A-14) as
n =√
με. (A-17)
The complex index of refraction is often denoted by n = n (1 + iκ) although we
prefer the representation given by n = n + ik in this dissertation where the two
representations have a straight-forward relationship. The expressions for n and
κ can be found several optics textbooks, for example, Born and Wolf [30]. We
present the equations below and refer interested readers to Born and Wolf [30] for
additional details:
n2 =1
2
{√μ2ε2 +
(2π) 2 (4μ2σ2)
ω2+ με
}(A-18)
and
n2κ2 =1
2
⎧⎨⎩√
μ2ε2 +(2π)
2
(4μ2σ2)
ω2− με
⎫⎬⎭ (A-19)
where κ is called the attenuation index and the positive value of the square root
is taken to be the desired solution.
97
APPENDIX B
THE LEVENBERG-MARQUARDT ALGORITHM
We have used the Levenberg-Marquardt algorithm [39] to design the estimation
algorithms presented in chapter 3. In this appendix, we describe the Levenberg-
Marquardt method and also briefly discuss the procedure used to initialize this
algorithm in our framework. The naming convention and notation used for vari-
ables applies only to this appendix.
The Levenberg-Marquardt method is a standard approach employed to solve non-
linear least-squares problems. This approach can be thought of as a damped ver-
sion of the well-known Gauss-Newton [39] method. Due the damping parameter,
the method behaves like the steepest-descent method [39] when the current esti-
mate or iterate is far from the solution and like the Gauss-Newton method when
the current iterate is close to the solution. Given a vector function f : Rn → Rm
with m ≥ n, this algorithm minimizes the following cost function
F (x) =1
2
m∑i=1
(fi (x))2
=1
2‖f (x)‖2
=1
2f (x)T f (x) (B-1)
where T denotes the transpose operator. The pseduo-code for the Levenberg-
Marquardt method as presented by Madsen [39] is given below:
98
• Levenberg-Marquardt method
• begin
– l = 0; ν = 2; x =x0;A = J (x)TJ (x); g = J (x)T f (x);
–– found = (‖g‖∞ ≤ ε1); μ = τ ∗ max {aii}
• while (not found) and (k ≤ kmax)
– l = l + 1; Solve (A + μI)hlm = −g
– if ‖hlm‖ ≤ ε2 (‖x‖ + ε2)
∗ found = true.
– else
∗ xnew = x + hlm
∗ ρ = (F (x)−F (xnew))(L(0)−L(hlm))
∗ if ρ > 0
· x = xnew
· A = J(x)T J (x); g = J (x)T f (x)found =(‖g‖∞ ≤ ε1)
·· μ = μ ∗ max{
13, 1 − (2ρ − 1)3}; ν = 2
∗ else
· μ = μ ∗ ν; ν = 2 ∗ ν
• end
99
where J is the Jacobian of f , μ is the damping parameter, aii is a diagonal element
of A (ith row and ith column) and ε1, ε2, τand kmax are user-defined parameters.
Algorithm Initialization
The solution that the Levenberg-Marquardt method converges to is dependent on
the starting “guess” used to initialize the algorithm. Recall from chapter 3 that the
objective is to estimate the complex index of refraction and reflection angle from
multiple polarization measurements in the fixed camera scenario. We first discuss
the initialization used for index of refraction estimation and then discuss the case
for reflection angle estimation. The extinction coefficient which is the imaginary
part of the index of refraction is in general a large value for metals in the visible
portion of the spectrum. This suggests that the extinction coefficient should
be set to a large value to initialize the Levenberg-Marquardt method. We use
[1.0 1.0]T as the starting iterate for experiments with laboratory measurements.
Our results in chapter 3 indicates that this value suffices for parameter estimation
with laboratory data. Finally, we remark that as with any iterative algorithm, the
accuracy of the solution provided by the Levenberg-Marquardt algorithm depends
on its initial iterates. The algorithm performs better when the starting value
is closer to the actual solution and converges to undesirable values when the
starting value is far away from the solution. Consequently, any available a priori
information should be utilized to solve the estimation problem. For example, this
100
approach is adopted to study the behavior of the proposed algorithm in simulation
studies. The reader is referred to Madsen [39] for further details.
We use the following rule of thumb for the reflection angle estimation problem
— the sum of the starting estimates angle of incidence and the reflection angle
for the first measurement should not be equal to the phase angle for the first
measurement. In other words
θsc1 = θ(start)i1 + θr(start) (B-2)
where the superscript start indicates these are the starting guesses for θi1 and
θr. The rationale behind this rule of thumb is that the starting guesses would
themselves be a solution to the angle estimation problem if the above were true.
We prefer to choose a value of θ(start)i1 and θ
(start)r that is lesser than θsc1. Our
experience with the proposed method indicates that the algorithm converges to
reasonable estimates of θr when the initial iterates θi1 and θr are within 20◦ of the
actual solution. A thorough analysis of the robustness of the proposed method to
various starting iterates is a part of future work at New Mexico State University.
101
REFERENCES
[1] J. S. Tyo, D. L. Goldstein, D. B. Chenault, and J. A. Shaw, “Review ofpassive imaging polarimetry for remote sensing applications,” Appl. Opt.,vol. 45(22), pp. 5453–5469, Aug. 2006.
[2] IEEE Signal Processing Mag., Special issue on exploiting hyperspectral im-agery, Jan. 2002.
[3] F. A. Sadjadi and C. S. L. Chun, “Polarimetric IR target classification,”IEEE Trans. Aerosp. Electron. Syst., vol. 37(2), pp. 740–751, Oct. 2001.
[4] ——, “Remote sensing using passive infrared stokes parameters,” Opt. Eng.,vol. 43(10), pp. 2283–2291, Oct. 2004.
[5] F. Goudail, P. Terrier, Y. Takakura, L. Bigu, F. Galland, and V. DeVlaminck,“Target detection with a liquid crystal-based passive stokes polarimeter,”Appl. Opt., vol. 43(2), pp. 274–282, Jan. 2004.
[6] V. Thilak, J. Saini, D. G. Voelz, and C. D. Creusere, “Pattern recognitionfor passive polarimetric data using nonparamteric classifiers,” in PolarizationScience and Remote Sensing II, J. A. Shaw, J. S. Tyo, eds., Proc. SPIE, vol.5888, San Diego, CA, Sept. 2005, pp. 337–344.
[7] A. M. E. Saba, “Potential application of visible passive imaging polarimetryin the discimination of real targets and decoys,” in Optical Pattern Recog-nition XVI. D. P. Casasent, T.-H. Chao, eds., Proc. SPIE, vol. 5816, SanDiego, CA, Sept. 2005, pp. 186–194.
[8] L. B. Wolff, “A polarization-based material classification from specular re-flection,” vol. 12, pp. 1059–1071, Nov. 1990.
[9] L. B. Wolff and T. E. Boult, “Constraining object features using a polariza-tion reflection model,” vol. 13, pp. 635–657, 1991.
[10] L. B. Wolff, “Using polarization components to separate reflection compo-nents,” in Proc. IEEE Computer Society Conference on Computer Visionand Pattern Recognition, San Diego, CA, June 1989, pp. 363–369.
[11] D. Miyazaki, M. Kagesawa, and K. Ikeuchi, “Transparent surface modelingfrom a pair of polarization images,” vol. 26, pp. 73–82, Nov. 2004.
[12] G. A. Atkinson and E. R. Hancock, “Shape from diffuse polarisation,” inProc. British Machine Vision Conference, Sept. 2004, pp. 919–928.
102
[13] ——, “Recovery of surface orientation from diffuse polarization,” vol. 15, pp.1653–1664, June 2006.
[14] ——, “Shape estimation using polarization and shading from two views,”vol. 29, pp. 2001–2017, Nov. 2007.
[15] R. G. Priest and S. R. Meier, “Polarimetric microfacet scattering theory withapplications to absorptive and reflective surfaces,” Opt. Eng., vol. 41(5), pp.988–993, May 2002.
[16] E. Hecht, Optics. Reading, MA: Addison-Wesley, 2005.
[17] E. Collett, A Field Guide to Polarization. Bellingham, WA: SPIE - TheSociety of Photo-Optical Instrumentation Engineers Press, 2005.
[18] M. Bass, Handbook of Optics vol. 1. New York: McGraw Hill, 1995.
[19] K. L. Coulson, Polarization and Intensity of Light in the Atmosphere. ADEEPAK Publishing, 1988.
[20] D. S. Flynn and C. Alexander, “Polarized surface scattering expressed interms of a bidirectional reflectance distribution function matrix,” Opt. Eng.,vol. 34(6), pp. 1646–1650, June 1995.
[21] R. G. Priest and T. A. Germer, “Polarimetric brdf in the microfacet model:Theory and measurements,” in Proceedings of the 2000 Meeting of the Mili-tary Sensing Symposia Specialty Group on Passive Sensors.
[22] K. E. Torrance and E. M. Sparrow, “Theory of off-specular reflection fromroughened surfaces,” J. Opt. Soc. Am., vol. 57(9), pp. 1105–1114, Sept. 1967.
[23] J. P. Meyers, “Modeling polarimetric imaging using dirsig,” Ph.D.dissertation, Rochester Institute of Technology, Rochester, NY, 2002.[Online]. Available: http://dirsig.cis.rit.edu/docs/
[24] J. R. Maxwell, J. Beard, S. Weiner, D. Ladd, and S. Ladd, “Bidirectionalreflectance model validation and utilization,” Technical report AFAL-TR-73-303, Environmental Research Institute of Michigan (ERIM), Oct. 1973.
[25] J. A. Conant and F. J. Iannarilli(Jr.), “Development of a combined bidirec-tional reflectance and directional emittance model for polarization model-ing,” in Polarization Analysis and Measurement IV, D. H. Goldstein, D. B.Chenault, W. G. Egan, M. J. Duggin, eds., Proc. SPIE, vol. 4481, Jan. 2002,pp. 206–215.
103
[26] D. D. Duncan, D. V. Hahn, and M. E. Thomas, “Physics-based polarimet-ric brdf models,” in Optical Diagnostic Methods for Inorganic Materials III,Leonard M. Hanssen, ed., Proc. SPIE, vol. 5192, Nov. 2003, pp. 129–140.
[27] B. P. Sanford and D. C. Robertson, “Infrared reflectance properties of aircraftpaints,” in Proc. IRIS Targets, Backgrounds and Discrimination, vol. 4481,1985.
[28] J. R. Shell, “Polarimetric remote sensing in the visible to near infrared,”Ph.D. dissertation, Rochester Institute of Technology, Rochester, NY, Nov.2005. [Online]. Available: http://dirsig.cis.rit.edu/docs/
[29] M. P. Fetrow, D. L. Wellems, S. H. Sposato, K. P. Bishop, T. R. Caudill, M. L.Davis, and E. R. Simrell, “Results of a new polarization simulation,” in Po-larization: Measurement, Analysis, and Remote Sensing IV, D. H. Goldstein,D. B. Chenault, W. G. Egan, M. J. Duggin, eds., Proc. SPIE, vol. 4481, Jan.2002, pp. 149–162.
[30] M. Born and E. Wolf, Principles of Optics - 6th ed. (with corrections). Cam-bridge, United Kingdom: Cambridge University Press, 1998.
[31] O. Morel, F. Meriaudeau, C. Stolz, and P. Gorria, “Polarization imagingapplied to 3d reconstruction of specular metallic surfaces,” in Machine VisionApplications in Industrial Inspection XIII, J. R. Price, F. Meriaudeau, eds.,Proc. SPIE), vol. 5679, Feb. 2005, pp. 178–186.
[32] O. Morel, C. Stolz, F. Meriaudeau, and P. Gorria, “Active lighting applied tothree-dimensional reconstruction of specular metallic surfaces by polarizationimaging,” Appl. Opt., vol. 45(2), pp. 4062–4068, Jan. 2006.
[33] F. A. Sadjadi, “Passive three-dimensional imaging using polarimetric diver-sity,” Opt. Lett., vol. 32(3), pp. 229–231, Oct. 2007.
[34] R. J. Woodham, “Photometric method for determining surface orientationfrom multiple images,” Opt. Eng., vol. 19(5), pp. 139–144, Jan. 1980.
[35] P. H. Christensen and L. G. Shapiro, “Three-dimensional shape from colorphotometric stereo,” Int’l J. Computer Vision, vol. 13(2), pp. 213–227, 1994.
[36] V. Thilak, D. G. Voelz, C. D. Creusere, and S. Damarla, “Estimating therefractive index and reflected zenith angle of a target using multiple polariza-tion measurements,” in Polarization: Measurement, Analysis, and RemoteSensing VII, D. H. Goldstein, D. B. Chenault, eds., Proc. SPIE, vol. 6240,Orlando, FL, May 2006, pp. 35–42.
104
[37] V. Thilak, C. D. Creusere, and D. G. Voelz, “Estimating the complex indexof refraction and view angle of an object using multiple polarization mea-surements,” in Proceedings of the Fortieth Asilomar Conference on Signals,Systems, and Computers, Pacific Grove, CA, Nov. 2006, pp. 1067–1071.
[38] V. Thilak, D. G. Voelz, and C. D. Creusere, “Polarization-based index ofrefraction and reflection angle estimation for remote sensing applications,”Appl. Opt., vol. 46(30), pp. 7527–7536, Oct. 2007.
[39] K. Madsen, H. Nielsen, and O. Tingleff, “Methods for non-linear least squaresproblems,” Technical University of Denmark, 2004.
[40] NMA-NEFMod9.5, “Nonconventional exploitation factors (nef) modeling v9.5,” 2005.
[41] M. Bass, Handbook of Optics vol. 2. New York: McGraw Hill, 1995.
[42] S. Damarla, “Automation and testing of an imaging polarimeter,” Master’sthesis, New Mexico State University, Las Cruces, NM, 2006.
[43] G. Boisset. (2007) Luxpop - thin film and bulk index of refraction andphotonics calculations. [Online]. Available: http://www.luxpop.com/
[44] F. Sadjadi and C. S. Chun, “Application of a passive polarimetric infraredsensor in improved detection and classification of targets,” InternationalJournal of Infrared and Millimeter Waves, vol. 19, pp. 1541–1559, 1998.
[45] V. Thilak, D. G. Voelz, and C. D. Creusere, “Image segmentation from multi-look passive polarimetric imagery,” in Polarization Science and Remote Sens-ing III, J. A. Shaw, J. S. Tyo, eds., Proc. SPIE, vol. 6682, San Diego, CA,Sept. 2007, p. 668206.
[46] R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification. New York:Wiley Interscience, 2001.
[47] A. Gersho and R. M. Gray, Vector Quantization and Signal Compression.Norwell, MA: Kluwer Academic Press, 1992.
[48] P. J. Rousseeuw, “Silhouettes: a graphical aid to the interpretation and val-idation of cluster analysis,” J. Comp App. Math, vol. 20, pp. 53–65, 1987.
[49] R. Unnikrishnan and M. Hebert, “Measures of similarity,” in Proc. IEEEWorkshop on Computer Vision Applications (WACV/MOTION’05), Breck-enridge, Colorado, Jan. 2005, pp. 394–400.
105
[50] R. Unnikrishnan, C. Pantofaru, and M. Hebert, “Toward objective evaluationof image segmentation algorithms,” vol. 29, pp. 929–944, 2007.
[51] X. Rui and D. W. II, “Survey of clustering algorithms,” vol. 16, pp. 645–678,Mar. 2005.
[52] J. Dunn, “Well separated clusters and optimal fuzzy partitions,”J.Cybernetics, vol. 4, pp. 95–104, 1974.
[53] D. Davies and D. Bouldin, “A cluster separation measure,” vol. 1, pp. 224–227, 1979.
[54] N. Bolshakova and F. Azuaje, “Cluster validation techniques for genomeexpression data,” Signal Processing, vol. 83, pp. 825–833, 2003.
[55] W. M. Rand, “objective criteria for the evaluation of clustering methods,” J.Am. Statistical Assoc., vol. 66(336), pp. 846–850, 1971.
[56] S. Tominaga and S. Okamoto, “Reflectance-based material classification forprinted circuit boards,” in Proc. 12th Conference on Image Analysis and Pro-cessing, 2003, pp. 238–244.
[57] H. Chen and L. B. Wolff, “Polarization phase-based method for maaterialclassification and object recognition in computer vision,” in Proc. IEEE Com-puter Society Conference on Computer Vision and Pattern Recognition, June1996, pp. 128–135.
[58] D. M. McKeown, S. D. Cochran, S. J. Ford, J. C. McGlone, J. A. Shufelt,and D. A. Yocum, “Fusion of hydice hyperspectral data with panchromaticimagery for catographic feature extraction,” vol. 37, pp. 1261–1277, Mar.1999.
[59] D. Slater and G. Healey, “Material classification for 3d objects in aerial hyper-spectral images,” in Proc. IEEE Computer Society Conference on ComputerVision and Pattern Recognition, June 1999, p. 2268.
[60] J. Zallat, P. Grabbling, and Y. V. Takakura, “Using polarimetric imaging formaterial classification,” in Proc. IEEE International Conference on ImageProcessing, Barcelona, Spain, Sept. 2003, pp. 827–830.
[61] V. Thilak, C. D. Creusere, and D. G. Voelz, “Material classification frompassive polarimetric imagery,” in Proc. IEEE Int’l Conf. Image Proc., vol. IV,Sept. 2007, pp. 121–124.
106
[62] K. K. Ellis, “Polarimetric bidirectional reflectance distribution function ofglossy coatings,” J. Opt. Soc. Am. A, vol. 13(8), pp. 1758–1762, Aug. 1996.
[63] T. A. Germer, “Polarized light scattered under smooth and rough interfaces,”in Polarization Science and Remote Sensing, J. A. Shaw, J. S. Tyo, eds.,Proc. SPIE, vol. 5158, Dec. 2003, pp. 193–204.
[64] Mathworks, “Optimization toolbox user’s guide 3,” Technical report AFAL-TR-73-303, Environmental Research Institute of Michigan (ERIM), 2007.
[65] A. Pamba, V. Thilak, D. G. Voelz, and C. D. Creusere, “Estimation of inci-dence and reflection angles from passive polarimetric imagery: extension toout-of-plane scattering,” in Polarization Science and Remote Sensing III, J.A. Shaw, J. S. Tyo, eds., Proc. SPIE, vol. 6682, San Diego, CA, Sept. 2007,p. 668200.
[66] J. R. Reitz, F. . J. Milford, and R. W. Christy, Foundations of ElectromagneticTheory - 3rd ed. Reading, MA: Addison-Wesley, 1980.
107
