Image Quality Metrics

• Image quality metrics • Mutual information (cross-entropy) metric

• Intuitive definition • Rigorous definition using entropy• Example: two-point resolution problem • Example: confocal microscopy

• Square error metric • Receiver Operator Characteristic (ROC)

• Heterodyne detection

MIT 2.717 Image quality metrics p-1

Linear inversion modelobject

channel

gHf

hardware “physical attributes”

(measurement) field

propagation detection

inversion problem: determine f, given the measurement g = H f

noise-to-signal ratio (NSR) = 1power) signal (average

variance) (noise = σ 2

= σ 2

normalizing signal power to 1 MIT 2.717 Image quality metrics p-2

Mutual information (cross-entropy)object

channel

gHf

hardware “physical attributes”

(measurement) field

propagation detection

2

1 ln = ∑

n1 eigenvalues µkC

+ of H2σ2k =1

MIT 2.717 Image quality metrics p-3

The significance of eigenvalues

n

n-1

1 0

2σ

2µ 2µ 2µ 2µ

(aka

is worth) ∑

=

+=

n

k

k

1 2

2

2 1C

σ µ

...

...

rank of measurement

how many dimensions

the measurement 1 ln

n n−1 2 1

eigenvalues of HMIT 2.717 Image quality metrics p-4

Precision of measurement2 k

µ 2σ

n1 1 ln +

2 2 2 t t 1 noise floor

−µσµC < <

2∑k 1=

= =

2

2

+

+

σµt1 ln

this term

2 tµ

2σ −

2 t 1 2 +

−µσ

1 ln +

+

1 ln +

2+... ...

≈precision of (t-2)th measurement ≤1

E.g. 0.5470839348

these digits worthless if σ ≈10-5

MIT 2.717 Image quality metrics p-5

this term≈0

Formal definition of cross-entropy (1)

EntropyEntropy in thermodynamics (discrete systems): • log2[how many are the possible states of the system?]

E.g. two-state system: fair coin, outcome=heads (H) or tails (T) Entropy=log22=1

Unfair coin: seems more reasonable to “weigh” the two statesaccording to their frequencies of occurence (i.e., probabilities)

)Entropy −= ∑ p( logstate p( state)2 states

MIT 2.717 Image quality metrics p-6

Formal definition of cross-entropy (2)

• Fair coin: p(H)=1/2; p(T)=1/2

bit11Entropy = − 2

1 1 1log 2

− 2

log2 = 2 2

• Unfair coin: p(H)=1/4; p(T)=3/4

1Entropy = − 4

1 3log 4

− 4

log2 3= bits81.02 4

Maximum entropyMaximum entropy ⇔⇔⇔⇔⇔⇔⇔⇔ Maximum uncertaintyMaximum uncertainty

MIT 2.717 Image quality metrics p-7

Formal definition of cross-entropy (3)Joint EntropyJoint Entropylog2[how many are the possible states of a combined variable

obtained from the Cartesian product of two variables?]

EntropyJoint ( YX ) −= ∑ ∑ yx p )log yx p ), ( , ( ,2 states states

Xx Yy∈ ∈

,object EntropyJointE.g. ( GF )= ?

hardware channel

“physical attributes”

(measurement) field

propagation detection gHf MIT 2.717 Image quality metrics p-8

Formal definition of cross-entropy (4)Conditional EntropyConditional Entropylog2[how many are the possible states of a combined variable

given the actual state of one of the two variables?]

EntropyCond. ( Y | X ) −= ∑ ∑ yxp ) log xyp )( , ( |2 states states

Xx Yy∈ ∈

object EntropyCond.E.g. ( G | F )= ?

hardware channel

“physical attributes”

(measurement) field

propagation detection gHf MIT 2.717 Image quality metrics p-9

Formal definition of cross-entropy (5)object

hardware channel

“physical attributes”

(measurement) field

propagation detection gHf

adds uncertainty to the measurement wrt the objectNoise adds uncertainty⇔ eliminates informationeliminates information from the measurement wrt object

MIT 2.717 Image quality metrics p-10

Formal definition of cross-entropy (6)uncertainty added due to noise

representation by Seth Lloyd, 2.100 Entropy Cond. ( G F )|

Entropy( )F ( ),C G F Entropy( )G information information contained contained

in the object in the measurement

Entropy Cond. ( F G ) cross-entropy | (aka mutual information)

information eliminated due to noise

MIT 2.717 Image quality metrics p-11

Formal definition of cross-entropy (7)

( )FEntropy ( )GEntropy

( ),

( )| ( )|

( ),C

G F Entropy Joint

G F Entropy Cond. F G Entropy Cond.

G F

MIT 2.717 Image quality metrics p-12

Formal definition of cross-entropy (8)

FF GGinformationinformation

sourcesource(object)(object)

informationinformationreceiverreceiver

(measurement)(measurement)Corruption source (Noise)Corruption source (Noise)

Physical ChannelPhysical Channel(transform)(transform)

, |C( GF ) = Entropy(F ) − EntropyCond. ( GF ) |= Entropy(G ) − EntropyCond. ( FG )

,= Entropy(F ) + Entropy(G ) − EntropyJoint ( GF )

MIT 2.717 Image quality metrics p-13

Entropy & Differential Entropy

• Discrete objects (can take values among a discrete set of states) – definition of entropy

( )log2 x pEntropy = −∑ x p ( )k k k

– unit: 1 bit (=entropy value of a YES/NO question with 50% uncertainty)

• Continuous objects (can take values from among a continuum) – definition of differential entropy

( )ln x pEntropy Diff. = − x p ( )dx∫ Ω( )X

– unit: 1 nat (=diff. entropy value of a significant digit in the representation of a random number, divided by ln10)

MIT 2.717 Image quality metrics p-14

Image Mutual Information (IMI)object

channel

gHf

hardware “physical attributes”

(measurement) field

propagation detection

Assumptions: (a) F has Gaussian statistics (b) white additive Gaussian noise (waGn) i.e. g=Hf+w where W is a Gaussian random vector with diagonal correlation matrix

Then C G F,( )=n1 ∑ +

1 ln

µσ2

2 k µk of seigenvalue : H

=2 1k

MIT 2.717 Image quality metrics p-15

Mutual information & degrees of freedom

n

n-1

1 0

2σ

2 nµ 2

1−nµ 2 2µ 2

1µ...

...

rank of measurement

mutual

∑ =

+=

n

k

k

1 2

2

2 1C

σ µ

2σ

H σ2

MIT 2.717

1 ln

information As noise increases • one rank of is lost whenever

overcomes a new eigenvalue • the remaining ranks lose precision

Image quality metrics p-16

Example: two-point resolutionFinite-NA imaging system, unit magnification

Two point-sources Two point-detectors

~ A gAf A A (object) (measurement)

intensities x measured ~ B gBfB B

Classical view

noiseless

x Ag Bg

intensities intensity

emitted @detectorplane

MIT 2.717 Image quality metrics p-17

Cross-leaking power

A ~ B ~

ss

BAB

BAA

fsfg sffg

+=

+=

( )xs 2sinc=

MIT 2.717 Image quality metrics p-18

IMI for two-point resolution problem

+ =11 µ1 ss ( ) H − = 2det 1H =

s − = 11 µ 2 ss

1 − s1 H− 1 =

2 − 11− ss

2( 1 ) 2( 1 )1 ln

1 ln +

1

2

− 1 +( G F ) s sC + +=, 2σ 2σ2

MIT 2.717 Image quality metrics p-19

IMI vs source separation

( ) 2

1SNR σ

=

s → 0s → 1MIT 2.717 Image quality metrics p-20

IMI for rectangular matrices (1)

H = =

H

underdeterminedunderdetermined overdeterminedoverdetermined(more unknowns than (more measurements

measurements) than unknowns)

eigenvalues cannot be computed, but instead we compute the singular valuessingular values of the

rectangular matrix

MIT 2.717 Image quality metrics p-21

IMI for rectangular matrices (2)

HT

H

= square matrix

ˆ T 1 Trecall pseudo-inverse f = ( H H )− H g

inversion operation associated with rank of Tseigenvalue ( H H )≡ aluessingular v ( ) H

MIT 2.717 Image quality metrics p-22

IMI for rectangular matrices (3)object

channel

gHf

hardware “physical attributes”

(measurement) field

propagation detection

under/over determined n1 +

τ k2σ

singular valuesof H1 ln = ∑

C 2k =1

MIT 2.717 Image quality metrics p-23

Confocal microscope

Small pinhole:

Intensity

object beam

splitter

pivirtual slice

detector

nhole Depth resolution

Light efficiency

Large pinhole:

Depth resolution

Light efficiency

MIT 2.717 Image quality metrics p-24

Depth “resolution” vs. noise

point sources, Object structure: mutually

incoherent

optical axis

sampling distance

Imaging method

correspondence intensity measurements

CFM

object scanning direction

NA=0.2

MIT 2.717 Image quality metrics p-25

Depth “resolution”vs. noise & pinhole size

units: Rayleigh distance MIT 2.717 Image quality metrics p-26

IMI summary

• It quantifies the number of possible states of the object that the imaging system can successfully discern; this includes – the rank of the system, i.e. the number of object dimensions that

the system can map – the precision available at each rank, i.e. how many significant

digits can be reliably measured at each available dimension • An alternative interpretation of IMI is the game of “20 questions:” how

many questions about the object can be answered reliably based on the image information?

• IMI is intricately linked to image exploitation for applications, e.g. medical diagnosis, target detection & identification, etc.

• Unfortunately, it can be computed in closed form only for additive Gaussian statistics of both object and image; other more realistic models are usually intractable

MIT 2.717 Image quality metrics p-27

Other image quality metrics

Mean Square Error (MSQ) between object and image•• Mean Square Error (MSQ) 2( f ) ofresultˆ E f − f k == ∑

k k inversionobject

samples

– e.g. pseudoinverse minimizes MSQ in an overdetermined problem – obvious problem: most of the time, we don’t know what f is! – more when we deal with Wiener filters and regularization

•• Receiver Operator ChaReceiver Operator Charracteacterriisstictic– measures the performance of a cognitive system (human or

computer program) in a detection or estimation task based on the image data

MIT 2.717 Image quality metrics p-28

Receiver Operator Characteristic

Target detection task

Example: medical diagnosis, • H0 (null hypothesis) = no tumor • H1 = tumor

TP = true positive (i.e. correct identification of tumor) FP = false positive (aka false alarm)

MIT 2.717 Image quality metrics p-29