Matrix Information Geometry and Applications to Signal ...

CMCAA 2018

Matrix Information Geometry and Applications to pp

Signal DetectionSignal Detection

Yongqiang Cheng

National University of Defense Technology

June 2018June 2018

Outline

Concepts of Matrix Information Geometry11

Mathematics of Matrix Information GeometryMathematics of Matrix Information Geometry12

Geometry of Hypothesis Testing 13

Applications to Signal Detection14

Applications to Signal Detection 14

2

1. Concepts of Matrix Information Geometry

What is information geometry?

Samples Distributions Statistical manifold

Data processing Statistics Information geometry

3


• Information geometry is the study of intrinsic properties ofif ld f b bilit di t ib ti b f diff ti lmanifolds of probability distributions by way of differential

geometry.Th i t t f i f ti t i th t• The main tenet of information geometry is that manyimportant structures in information theory and statistics canbe treated as structures in differential geometry by regardingbe treated as structures in differential geometry by regardinga space of probabilities as a differentiable manifold endowedwith a Riemannian metric and a family of affine connections.with a Riemannian metric and a family of affine connections.

Information Theory

StatisticsProbability

Th

Information Geometry

yTheory

Diff ti lPhysics

Relationships with other subjects y

Differential Geometry

Ri iS t

Physicsother subjects

4

RiemannianGeometry

Systems Theory


Matrix information geometry arose from the study of geometry of manifolds of matrices a branch of information geometry: nonrandom caseg y

geometry of covariances Lie group

1,1 1,2 1,

2,1 2,2 2,

n

nH n

i i

x x xx x x x x x x

Lie group

1

,1 ,2 ,

i

n n n nx x x

5

1. Concepts of Matrix Information GeometryMatrix manifold

theory Signal processing Other field

Signal detection STAP Signal filtering Medical signal Information Signal detection STAP Signal filtering ganalysis encoding

Barbaresco (Non-stationary signal detection 15) Balaji (Riemannian

Tompkins(Stiefel Moraga(Anisotropic Suvrit(Generalizdetection, 15)

Wong (Sonar signal detection, 15)Nielsen (Random signal detection 17)

Balaji (Riemannian STAP, 12)Aubry(GeometricBarycenter data selection 12)

bayesian filtering, 08)Bonnabel(Geometric EKF, 09)

Moraga(Anisotropic filtering, 07)Li(Sleepstate decision, 11)Malek(Bhattacharyya

Suvrit(Generalized dictionary learning, 11)Harandi(Steinkernel code 12)

Absil(Grassmann manifold, 04)Boyer(Stiefel

if ld 96)

signal detection, 17)Hua(Manifold filtering matrix CFAR, 17)Hua (Information

selection, 12)Aubry(GeometricMedian data selection, 13)

)Quentin(Grassmann particle filtering, 10)Bourmaud(Continuous-discrete EKF 13)

Malek(Bhattacharyya mean denoising, 13)Malek(Bhattacharyyamedian denoising, 13)

kernel code, 12)Harandi(Bregman sparse coding, 14)

manifold, 96)Sun(geometry of SPD manifold, 14)Prajjwal(Recurrent

divergence matrix CFAR, 17)

us discrete EKF, 13)

Computational information geometry

Computer vision and image processing Others

Manifold, 14)g y p g

Suvrit(S divergence, 12)Moakher (Modified Bh tt h 12)

Ando (ALM mean, 04)Dario (BMP mean, 10)Dario (CHEAP mean,

Ovarlez(SAR image classification, 11)Cherian (Vedio

Sadeep(Manifold kernel sparse, 14)

Barachant(Brain analysis, 13)D d (B iBhattacharyya, 12)

Harandi (Infinite Bregman divergence, 14)

( ,11)Moakher(Bhattacharyya mean, 13)Nielsen (Jensen mean

(tracking , 11)Romero(tBD object tracking, 13)Cao(Image retrieval

p , )Alavi Random projection, 14)Horev(Geometric PCA 15)

Dodero (Brain connectivity graph, 15)Nicholas(BCI

6

Hua (tBD divergence, 17)

Nielsen (Jensen mean, 13)

Cao(Image retrieval, 14)

PCA, 15)system, 16)


Sparse encoding based on matrix manifolds

Given a group of bases of matrices 1 2, ,..., nB B B

Given a group of bases of matrices

Seek a representation of data

1 2, ,..., n

X

1minn

i ii

X B

1 2, , , n

st.

Applications: Facial recognition behavior recognition textureApplications: Facial recognition, behavior recognition, texture classification, etc.

7


Classification for brain connectivity graphs

Riemannian manifold of

positive definite matrices

8

Outline






9

2. Maths of Matrix Information Geometry

1） Positive Definite Matrix ManifoldThe family of Hermitian positive definite matrix forms aspace

n nspace,

= , > 0n nX X

n is a differential manifold with non-positive curvature p

a self-dual convex cone

a canonical high order symmetric space

a submanifold of general linear groupg g p

10


2） Metric and geodesic

Inner product and norm

1 21 1, , ,tr A A A

P Q A PA Q P P P

The geodesic between P and Q is formulated as

1 2 1 2 1 2 1 2 0 1t

S t t P P QP P

The geodesic distance between P and Q

, , 0,1S t t P Q P P QP P

The geodesic distance between P and Q

1, logd P Q P Q , gF

Q Q

11


3） Information divergenceα-divergence, Bregman divergence

α-β logarithm determinant divergenceα β logarithm determinant divergence

4） Geometric mean on manifold

For m positive real number , the mean is formed as 1 2, ,..., mx x x

1 m 1 m

F iti d fi it t i t i=1

1= ii

x xm

2

>0 =1

1=arg min -m

ix i

x x xm

P P PFor m positive definite matrix , geometric mean 1 2, , ..., mP P P

21=arg minm

dP P P

=1

=arg min ,in i

dm P

P P P

12

Outline






13

3. Geometry of Hypothesis Testing 1）Start from target detection

60

70

80

90

100

10

20

30

40

50

0( | )P x

-30 -20 -10 0 10 20 30 -30 -20 -10 0 10 20 300

1( | )P x

x

Hypothesis testing

14

3. Geometry of Hypothesis Testing 2）Geometric interpretation of hypothesis testing

1

0

1

1 0

( )( )

Ni

i i

p xLp x

1

00 1

1( || ) ( || )D q p D q pN

Likelihood ratio test “Minimum distance detector”

G i i i0( | )p x 1( | )p x

1 1( ) TP Q Q P

Geometric interpretation

X1 1( ) TD FP Q Q P s s

2 1Td s s SNR

Xx

11( || ) TK D p p s s

d s s SNR

1d( | )p x θ

( | )p x θ0d0 1( || )

2K D p p s s

KL divergence

1( | )p x θ0( | )p x θ

0

15

KL divergencethe decision is made by selecting the model that is “closer” to signal distribution estimates

3. Geometry of Hypothesis Testing 2）Geometric interpretation of hypothesis testing

2

2

1; , exp22x

p x

222

( ) | ( )p x θ θ

Structure of data ( , ) | ( , )p x θ θ

d i

2 2( , )

geodesic

G i1 1( , )

Gaussian manifold

(hyperbolic)(hyperbolic)

16

0( ,0)

Outline






17

4. Applications to Signal Detection1） Classical CFAR detector

ba

c

targets

a

samples

Classical CFAR detectorD i i b i th t t f th

1z iz 1iz MzDz Decision by comparing the content of the

cell under test with an adaptive threshold

given by the arithmetic mean of reference

cells to achieve the desired constant Dz

z

T

18probability of false alarm.

T

4. Applications to Signal Detection2） Matrix CFAR detection on SPD Manifold

R R R R

Matrix CFAR detector1R iR 1iR NR

R R

A generalized CFAR

technique based on the R R

manifold of symmetric positive

definite (SPD) matricesdefinite (SPD) matrices.

It has been proved that the Matrix Information Geometry1R

NR

p

Riemannian distance-based

d t t h b tt d t ti

Matrix Information Geometry

R R

1iR RDRdetector has better detection

performance than the

19

2R iR

classical CFAR detector.

4. Applications to Signal Detection2） Matrix CFAR detection on SPD Manifold

Initial spectra of measurements Mean spectra of measurementsInitial spectra of measurements Mean spectra of measurements

20

Intensity Classical detector Geometric detector

4. Applications to Signal Detection2） Matrix CFAR detection on SPD Manifold • Riemannian distance between two SPD matrices

22 1 2 1 2 2

n

f S

22 1 2 1 2 2

1 2 1 2 11

, ln ln kk

d

R R R R R

• Riemannian center of N SPD matrices

iN

pdR R R

where p=1 denotes the median; p=2 is the mean

1

arg min ,pi i

iw d

RR R R

R Rwhere p 1, denotes the median; p 2, is the mean.

• The matrix CFAR detector

R R

, id R R

21

4. Applications to Signal Detection2） Matrix CFAR detection on SPD Manifold • Riemannian distance between two SPD matrices

22 1 2 1 2 2

n

f S

22 1 2 1 2 2

1 2 1 2 11

, ln ln kk

d

R R R R R

• Riemannian center of N SPD matrices

iN

pdR R R

where p=1 denotes the median; p=2 is the mean

1

arg min ,pi i

iw d

RR R R

R Rwhere p 1, denotes the median; p 2, is the mean.

• The matrix CFAR detector

R R

, id R R

22

4. Applications to Signal Detection3） Extend KL divergence based matrix CFAR

• Bhattacharyya divergence: Jensen-Bregman KL divergence

1R R 1 21 2 1 2

1, = log log2 2Bd

R RR R R R

• sKL divergence: Jeffreys KL divergence

-1 -11 2 1 2 2 1

1, = 22SKLd tr n R R R R R R

1 1

• tKL divergence: total KL divergence

-1 -11 2 2 1

1 2 2

log,

1 log 2logtKL

tr nd

n

R R R RR R

R

23

22

1 log 2log2 log

4 2n

c

R

R

4. Applications to Signal Detection3） Extend KL divergence based matrix CFAR

Mean matrix CFAR Median matrix CFAR

C l i tKL t i CFAR h th b t fConclusion: tKL matrix CFAR has the best performanceProblem: Detectors with different distance measure have

i d t ti f24

various detection performance.

4. Applications to Signal Detection4）Anisotropy and detection potential

1R iR i+1R nR iR RDR

R

1R

2R3R

i+1R

4RnR DRR

3

Matrix CFAR and classification

R R1R

2R

iRi+1R

4RnR DRR

1R

2R

iRi+1R

4RnR DRR

2

3R4

3R

4

Interpretation of Matrix CFAR on matrix manifoldDR DR

25


The anisotropic isosurface of Riemannian distance The anisotropic isosurface of SKL distance

The anisotropic isosurface of KL distanceThe anisotropic isosurface of Bhat divergence

The anisotropy of various measuresreflects the local structure of manifolds

26

reflects the local structure of manifolds.

The anisotropic isosurface of tKL divergence


Anisotropy factor

mind d R I min ,AId d

R I

1R RDetection potential2R

p

,11 2

2

, AIP

AI

dD

dR R

Definition of detection potential

αI,2AI

The smaller the anisotropy is,the better performance thematrix CFAR obtains.

27Detection potential of different divergences

4. Applications to Signal Detection5）tBD based matrix CFAR

The tKL divergence is a special case of the total Bregmandivergence tBD, which is invariant to linear transformation.

( , ) ,BD x y f x f y x y f y

f f f

2

,( , )

1

f x f y x y f ytBD x y

f y

robuster

BD ,x y

tBD ,x y BD ,x y

tBD ,x y

f y

Rotational invariance

Rotational invariance

28


Extend the definition of tBD from convex space to matrix manifold,then the tBD between X and Y is

2

- - -, =f

tr f f f

X Y Y X YX Y

21+

ff Y

When and we have TSL 2=f x x 2tr f trX X

define different tBD divergence with different ,f X Y f x

When , , and we have TSL =f x x tr f trX X

When , and it is TVN logf x x x x logtr f tr X X X X

When , , and we obtain TLD logf x x logtr f X X

29


The shape of anisotropic isosurface of different metric

Riemannian distance TSL divergence

30TLD divergence TVN divergence


Detection performance and detection potential

IPIX Radar Real Data, Pd vs SCR Detection potential, d

Conclusion: the tBD matrix CFAR has better performance thanRi i t i CFAR

p

Riemannian matrix CFAR.31

4. Applications to Signal Detection6）tJBD based matrix CFAR

Suppose f(x) is differential and convex, the tJBD of x and y is

1 f f f

, ,1

x y x y f x f y f xy

f x f y f x f y

,

, 1 1 , 0,1,

f x f y f x f yx y

x y x y

f x f x

f y

1l

f y

1lRotational invariance

f xy f xy

32


1 f f f

Extend tBD to matrix manifold, the tJBD of X and Y is

2

1, ,1

tr f f f

X Y X Y X Y XY

2

2, 1 1 , 0,1F

F

f f

X YX Y

X Y

When , , and we have TJSL 2=f x x 2tr f trX X When , , and it is TJVN logf x x x x logtr f tr X X X X

When , , and we obtain TJLD logf x x logtr f X X

33


The shape of anisotropic isosurface of different metric

Riemannian distance TJSL divergenceRiemannian distance TJSL divergence

TJLD di TJVN di

34

TJLD divergence TJVN divergence


Detection performance and detection potential

IPIX Radar Real Data, Pd vs SCR Detection potential

Conclusion: the tJBD matrix CFAR has better performance thanthe Riemannian matrix CFAR detector.

35

4. Applications to Signal DetectionSummary

Classical CFAR detectorEuclidean space REuclidean distance measure

Matrix CFAR detector

1R

R

NR

RDR Matrix CFAR detectorMatrix manifoldRiemannian distance measure 2R iR

1iR R

Riemannian distance measureKL divergence, etc.

A d d t t h ld

2 i

A good detector should Properly characterize the

1R NR

1iR RRDR

intrinsic structure of themeasurement space

iR2R

36 Maximize the divergence between two hypotheses (clusters)

Thank you for your

attention！attention！

Matrix Information Geometry and Applications to Signal ...

Documents

Transcript of Matrix Information Geometry and Applications to Signal ...