1 / 35

Seize the Moments for Subdiffraction Incoherent Imaging ∗

Mankei TsangDepartment of Electrical and Computer Engineering

Department of Physics

mankei@nus.edu.sg

http://mankei.tsang.googlepages.com/

Sep 2017

∗This work was supported by the Singapore National Research Foundation under NRF Award No. NRF-NRFF2011-07 and an MOETier 1 grant.

Subdiffraction Incoherent Imaging

2 / 35

Sparse (Good Regime) Subdiffraction (Worst Nightmare)

1. Limits to direct imaging under diffraction and photon shot noise2. Enhancement via farfield optics3. Tsang, NJP 19, 023054 (2017)4. (Semiclassical): Tsang, arXiv:1703.08833 (2017).5. Tsang, Nair, Lu, PRX (2016).

Superresolution Microscopy

3 / 35

■ PALM, STORM, etc.: isolate emitters. Locate cen-troids.

https://cam.facilities.northwestern.edu/588-2/single-molecule-localization-microscopy/

■ Need controllable fluorophores■ slow■ doesn’t work for stars

Compressed Sensing

4 / 35

1. Donoho, “Super-Resolution via Sparsity Constraints”2. Candes, Fernandez-Granda, “Towards a Mathematical Theory of Super-Resolution”3. Zhu et al., “Faster STORM using compressed sensing,” Nature Methods (2012).

Needs SparsityDoesn’t work if the sources are too close (sub-diffraction)

Diffraction Limit

5 / 35

Diffraction-limited

1. Fluorescence microscopy2. Space telescopes (Webb, $8.8 billion)3. Ground-based telescopes:

(a) Large Binocular Telescope (LBT) ($120million)

(b) Giant Magellan Telescope (GMT) (∼$1billion)

(c) Thirty Meter Telescope (TMT) (∼$1.2billion)

(d) European Extremely Large Telescope(E-ELT) (∼$1.2 billion)

LBT Point-Spread Function

6 / 35

■ Esposito et al., “Large Binocular Telescope Adaptive Optics System: New achievements andperspectives in adaptive optics,” Proc. SPIE 8149, 814902 (2011)

■ Strehl ratio > 80% at infrared■ Spatially incoherent sources (paraxial scalar waves, normalized object intensity = f(R), average

photon number = N):

I(r) = N

∫

d2R |ψ(r −R)|2f(R). (1)

■ Perfect reconstruction of f(R) if object size is limited and I(r) is measured exactly [e.g.,Bertero].

Photon Shot Noise

7 / 35

■ EMCCD: efficiency > 90%, extremely low dark counts■ Fluorescent particles (GFP, dye molecules, quantum dots, etc.)

◆ Poisson, antibunching is negligible◆ see, e.g., Pawley ed., Handbook of Biological Confocal Mi-

croscopy, Ram, Ober, Ward (2006), etc.

■ Optical thermal sources (stars, etc.)

◆ Poisson, bunching is negligible at optical frequencies◆ Multiphoton coincidence, Hanbury Brown-Twiss: obsolete for

decades, SNR too poor.◆ see, e.g., Goodman, Statistical Optics, Zmuidzinas, JOSA A

20, 218 (2003) etc.

■

P (mj) =∏

j

exp(−λj)λmj

j

mj !, λj = I(rj)d

2r. (2)

■ Intensity-dependent noise:

E (mj) = λj , E [mj − E(mj)]2 = λj . (3)

Parameter Estimation

8 / 35

■ Assume probability distribution P (m|θ) depends on parameters θ.■ Given measurement m, estimator is θ(m).■ error covariance:

Σµν(θ) = E[

θµ(m)− θµ] [

θν(m)− θν]

. (4)

■ Assume unbiased estimator: E[

θµ(m)]

= θµ. Cramer-Rao bound (standard in moleculeimaging):

Σµµ ≥ CRBµµ, CRB ≡ J−1, Jµν =∑

m

1

P (m|θ)∂P (m|θ)∂θµ

∂P (m|θ)∂θν

. (5)

■ Poisson:

Jµν =∑

j

1

λj

∂λj

∂θµ

∂λj

∂θν→

∫

d2r1

I(r|θ)∂I(r|θ)∂θµ

∂I(r|θ)∂θν

. (6)

■ Sensitivity/variance ratio■ Σ = CRB for maximum-likelihood estimator in an asymptotic limit (N =

∫

d2rI(r) → ∞)■ Bayesian/minimax generalizations of CRB for any biased or unbiased estimator possible [e.g.,

Tsang, arXiv:1605.03799 (2017)].

Sparse Regime

9 / 35

■ Treat as isolated point sources, s = 1, 2, 3, . . .■ Photometry: estimate average photon number Ns of sth

source:

CRB = Ns. (7)

■ Astrometry/localization: estimate locations:

CRB =σ2

Ns, σ ∼ λ

sinφ. (8)

■ Trivial estimators achieve the bounds.■ Paraxial:

◆ Semiclassical: Falconi (1964); Farrell (1966)◆ Quantum: Helstrom (1970)◆ Astronomy: Lindegren (1978)◆ Microscopy: Bobroff (1986)

■ Full EM, full quantum: Tsang, Optica 2, 646 (2015).

Subdiffraction Regime

10 / 35

I(r|θ) = N

∫

d2R |ψ(r −R)|2f(R|θ), Jµν =

∫

d2r1

I(r|θ)∂I(r|θ)∂θµ

∂I(r|θ)∂θν

. (9)

■ What θ for arbitrary object?■ Simple expressions for J and estimator?

Defining Subdiffraction Regime

11 / 35

∆ ≪ 1. (10)

Moment Parameterization [Tsang, NJP (2017)]

12 / 35

■ Assume Gaussian PSF (common in fluorescence microscopy),

|ψ(r −R)|2 =1

2πexp

(

−1

2|r −R|2

)

= |ψ(r)|2

1 +∑

µ

HeµX(x)HeµY

(y)

µX !µY !XµXY µY

,

■ Linear parameterization:

I(r|θ) = N

∫

d2R|ψ(r −R)|2f(R|θ) = N |ψ(r)|2

1 +∑

µ

HeµX(x) HeµY

(y)

µX !µY !θµ

. (11)

(f(R|θ) is normalized object intensity)■ Moments (uniquely determine distribution)

θµ =

∫

d2Rf(R|θ)XµXY µY . (12)

Fundamental Limit to Direct Imaging [Tsang, NJP (2017)]

13 / 35

■ I(r) = |ψ(r)|2[1 +O(∆)], intensity looks almost like centered Gaussian

Jµν =

∫

d2r1

I(r|θ)∂I(r|θ)∂θµ

∂I(r|θ)∂θν

=N

µX !µY ![δµν +O(∆)] , CRBµµ =

µX !µY !

N[1 +O(∆)] .

(13)

◆ Almost orthogonal information matrix (natural parameter set)◆ µX !µY ! increases with higher orders◆ Almost parameter-independent (Bayesian/minimax bounds are similar)

■ Pretty good unbiased estimator: θµ = 1N

∑

j HeµX(rj) HeµY

(rj)mj , quite efficient if ∆ ≪ 1.

Glass Half-Full or Half-Empty?

14 / 35

■ Benchmark for superresolution techniques (metamaterials, superoscillation, multiphotoncoincidence, etc.): can they beat image processing?

■ θµ are small parameters:

|θµ| ≤(

∆

2

)µX+µY

. (14)

■ Fractional error:√

CRBµµ

|θµ|=

√µX !µY !√

NO(∆µX+µY ). (15)

■ Need many photons to make fractional error ≪ 1.

Example: Separation of Two Point Sources

15 / 35

Tsai and Dunn, Lincoln Lab. Tech. Note AD-A073462 (1979); Bettens et al., Ultrami-croscopy 77, 37 (1999); Van Aert et al., J.Struct. Biol. 138, 21 (2002); Ram, Ward,Ober, PNAS 103, 4457 (2006).

(a)

(b)

■ first moment:

J(θ1) ≈ N. (16)

■ second moment θ2 = d2/4,

J ′(d) =

(

∂θ2

∂d

)2

J(θ2) ≈Nd2

8(quadratic).

(17)

d0 2 4 6 8

Fisher

inform

ation/(N

/4)

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5Classical Fisher information

J(direct)11

J(direct)22

Quantum Optics [e.g., Mandel and Wolf]

16 / 35

■ Sudarshan (Glauber):

ρ =

∫

DαΦ(α) |α〉 〈α| , α =

α1

α2

..

.

. (18)

■ |α〉 is multi-spatial-mode coherent state.■ Long observation time means multiple temporal/spectral modes, assume ρ⊗M .■ Thermal state: zero-mean Gaussian

Φ(α) =1

det(πΓ)exp

(

−α†Γ−1α)

, EΦ (αjα∗k) = Γjk. (19)

Γ is the mutual coherence matrix in classical statistical optics.■ Normalized:

gjk =Γ

tr Γ. (20)

■ Photon-counting probability distribution (Mandel):

P (m) =

∫

DαΦ(α) |〈m|α〉|2 . (Gaussian mixed with Poisson) (21)

■ Bunching

Weak-Source Approximation

17 / 35

■ Full model is too complicated.■ Approximation: average photon number per temporal mode

ǫ =∑

j

EΦ

(

|αj |2)

= tr Γ ≪ 1. (22)

good for natural thermal optical sources, fluorophores.■ Gaussian is much sharper than Poisson

ρ = (1− ǫ) |vac〉 〈vac|+ ǫρ1 +O(ǫ2), (23)

ρ1 =∑

j,k

gjk |j〉 〈k| , |j〉 ≡ a†j |vac〉 . (24)

Normalized mutual coherence matrix becomes the one-photon density matrix (g(1)).■ ǫ→ 0, M = N/ǫ → ∞ limit: photon counting is Poisson■ Gottesman, Jennewein, Croke, PRL (2012); Tsang, PRL 107, 270402 (2011); NJP (2017).

Photon Picture

18 / 35

■ Incoherent sources → mixture, diffraction limit → wavefunction spread in ψ(r −R).

ρ1 =

∫

d2Rf(R|θ) |ψR〉 〈ψR| , |ψR〉 =∫

d2rψ(r −R) |r〉 . (25)

■ Each photon comes with a random displacement R drawn from f(R|θ).■ Direct imaging is measurement of photon positions:

I(r|θ) = N 〈r|ρ1|r〉 = N

∫

d2R|ψ(r −R)|2f(R|θ). (26)

■ If object size ∆ ≪ 1, ρ1 ≈ |ψ0〉 〈ψ0| is almost a pure state.

Pixels in Hilbert Space

19 / 35

■ TEM modes:

|q〉 =∫

d2rφq(r) |r〉 , (27)

φq(r) =Heqx(x) Heqy (y)

√

2πqx!qy !exp

(

−1

4|r|2

)

. (28)

■ For Gaussian PSF, |ψR〉 is a displaced TEM00, i.e., acoherent state.

■ Probability of photon in qth mode:

〈q| ρ1 |q〉 = C(q, q)Θ2q , (29)

Θ2q =

∫

d2R e−|R|2/4f(R|θ)X2qxY 2qy . (30)

■ Yang, Simon, Lvovsky et al., Optica (2016); Tsang, NJP(2017).

■ If ∆ ≪ 1, f(R|θ) is much sharper than the Gaussian,Θ2q ≈ θ2q

■ qth TEM mode → 2qth moment of object.

Explanation

20 / 35

■ Each photon comes with a random displacement X drawn from f(X).■ Wavefunction:

■ Conditioned on X,

Probability of photon in first-order mode ∝ X2. (31)

■ Since X is random,

Average probability of photon in first-order mode ∝∫

dXf(X)X2. (32)

■ Higher orders: Expand ψ(x−Xs) in Hermite-Gaussian functions.

Spatial-Mode Demultiplexing (SPADE)

21 / 35

image plane

...

...

image plane

...

...

■ Poisson with mean λq = N 〈q| ρ1 |q〉.■ Many other ways (optical comm.), e.g.,

◆ DAB Miller, Photonics Research 1, 1 (2013); Li et al., “Space-division multiplexing: the next frontier inoptical communication,” Adv. Opt. Photon. 6, 413 (2014); V. A. Soifer, Computer Design of DiffractiveOptics (CISP/Woodhead, Cambridge, 2013).

◆ Universal photonic circuits: Carolan et al., Science 349, 711 (2015); Harris et al., Nature Photon. 11, 447(2017).

Enhancement [Tsang, NJP (2017); arXiv:1703.08833 (2017)]

22 / 35

■ Mean-square error for estimating 2qth moment:

MSE ≈ CRB ≈ θ2q

C(q, q)N=O(∆2qx+2qy )

N. (33)

much smaller than direct-imaging CRB by O(∆2qx+2qy ) = O(∆µX+µY ) for ∆ ≪ 1 (prefactoris also smaller).

■ Fractional error:√MSE

|θµ|=

1√NO(∆(µX+µY )/2)

. (34)

(Coupling into a high-order channel for a small object is inefficient. Need many photons in achannel to make fractional error ≪ 1, but not as many as direct imaging needs.)

Explanation

23 / 35

■ Direct imaging: noise dominated by TEM00 (background)■ SPADE: background noise in lower-order modes removed, only shot noise in the right mode.

Separation of Two Point Sources

24 / 35

■ θ2 = d2/4,

J(SPADE)(d) =

(

∂θ2

∂d

)2

J(SPADE)(θ2) ≈d2

4

N

4θ2=N

4, CRB ≈ 4

N. (35)

θ/σ0 1 2 3 4 5

Fisher

inform

ation/(L

/4σ2)

0

1

Fisher information for separation estimation

J (SPADE)(θ)

J (direct)(θ)

θ2/σ0 0.2 0.4 0.6 0.8 1

Mean-squareerror/(4σ2/N

)0

20

40

60

80

100Cramer-Rao bounds on separation error

Quantum (1/K22)

Direct imaging (1/J(direct)22 )

■ Tsang, Nair, Lu, PRX (2016).

Experiment: SPLICE

25 / 35

■ Tham, Ferretti, Steinberg, Phys. Rev. Lett. 118, 070801 (2017).

Experiment: SLIVER

26 / 35

Estimator

Image

Inversion

■ SuperLocalization via Image-inVERsion interferometry■ Nair and Tsang, Opt. Express 24, 3684 (2016).■ Experiment: Tang, Durak, Ling (CQT), Opt. Express 24, 22004 (2016).

◆ Classical, nowhere near quantum limit.

Experiment: Hologram

27 / 35

■ Paur, Stoklasa, Hradil, Sanchez-Soto, Rehacek, Optica 3, 1144 (2016).

■ Problem: low diffraction efficiency

Experiment: Heterodyne/Homodyne

28 / 35

■ Direct imaging measures intensity, SPADE is sensitive to phase. Can heterodyne/homodynework?Yang, Simon, Lvovsky et al., Optica (2016):

■ No-go: Fisher information smaller by O(ǫ) [Tsang, PRL 107, 270402 (2011); Yang, Nair, Tsang,Simon, Lvovsky, arXiv:1706.08633 (2017)].

■ Reason: vacuum noise for zero-photon events much larger than photon-counting noise (Poissonvariance is small for low photon numbers)

ρ = (1− ǫ) |vac〉 〈vac|+ ǫρ1 + O(ǫ2). (36)

■ Can beat direct imaging for ǫ≫ 1 (Laser sources/incoherent scatterers)

Odd Moments? Quantum State Tomography

29 / 35

■ Density matrix in TEM basis:

〈q| ρ1 |q′〉 = C(q,q′)Θq+q′ . (37)

■ 1D, odd moments are off-diagonal:

Θ0 Θ1/2 Θ2/(4√2) . . .

Θ2/4 Θ3/(8√2) . . .

Θ4/32 . . .

. (38)

■ interferometric TEM (iTEM) projections

|±〉 = 1√2

(

|q〉 ± |q′〉)

, 〈±| ρ1 |±〉 = β(q, q′) ±C(q, q′)Θq+q′ , (39)

β(q, q′) ≡ 1

2

[

C(q,q)Θ2q + C(q′, q′)Θ2q′

]

. (40)

■ choosing neighboring q and q′,

MSEµ ≈ β(q,q′)

2C2(q, q′)N=

{

O(∆µX+µY −1)/N, µX + µY odd,O(∆µX+µY )/N, µX + µY even.

(41)

■ Significant enhancement if µX + µY > 1 (second order or higher).

TEM and iTEM Schemes

30 / 35

■ To get all moments, split photons at most 7 ways.■ At most factor-of-7 penalty in MSE

Mode Functions

31 / 35

Going Down the Rabbit Hole: Quantum Information

32 / 35

■ Is SPADE the best?■ Helstrom (1967), Nagaoka (1989) (Braunstein Caves 1994): For any measurement,

J ≤ K, Kµν =M Re (trLµLνρ) ,∂ρ

∂θµ=

1

2(Lµρ+ ρLµ) . (42)

■ Nagaoka, Hayashi in Asymptotic Theory of Quantum Statistical Inference; Fujiwara JPA (2006):there exists a measurement that achieves the quantum bound for N → ∞.

■ K(ρ) is the quantum Fisher information, the ultimate amount of information in the photons.■ Difficult to compute exactly for mixed states, use looser bounds.

ρ1(X) =∑

s

pse−ikXs |ψ0〉 〈ψ0| eikXs , (43)

K(X) ≤ K(X) ≡ N∑

s

psK(X)

[

e−ikXs |ψ0〉 〈ψ0| eikXs

]

, (convexity) (44)

QCRB(θ)µµ ≥ µ2θ2µ−2

N. (45)

■ Direct imaging/iTEM is near-optimal for first moment.■ TEM is near-optimal for second moment.■ Bound much lower for third and higher moments.■ Open Question: Bound too loose or measurement not good enough?

Quantitative Imaging Beyond Pretty Pictures

33 / 35

■ Astronomy

◆ determine size and shape of stars, planetary systems (exoplanets), clusters, galaxies, etc.

■ Fluorescence microscopy

◆ superresolution techniques (PALM/STORM/STED) are slow◆ Molecule cluster analysis, protein stoichiometry, see, e.g., Nicovich, Owen, Gaus, “Turning

single-molecule localization microscopy into a quantitative bioanalytical tool,” NatureProtocols 12, 453 (2017):

◆ Image reconstruction: scanning + maximum entropy

Open Questions

34 / 35

■ Quantum optimality for third/higher moments?■ Non-Gaussian PSFs? Use orthogonal polynomials [Rehacek et al., OL (2017)]■ Robustness?■ Intermediate regime between Sparse and Subdiffraction?■ 3D?■ Experiments? Multimode, reconfigurable SPADE for broadband sources.

◆ Astrophotonics: Similar proposal ofphotonic circuits in stellar interfer-ometry

◆ Prior work: to improve atmosphericseeing, technical

◆ Our work: Fundamental advantagewith diffraction + photon shot noise

◆ Singapore: Fluorescence mi-croscopy, compete with/complementPALM/STORM/STED etc.

“Dragonfly,” Jovanovic et al., Mon. Not. R. As-tron. Soc. 427, 806 (2012)

Quantum Technology 1.5

35 / 35