3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987...
Transcript of 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987...
![Page 1: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/1.jpg)
3D Microscopy: Deconvolution, Confocal, Multiphoton
![Page 2: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/2.jpg)
Biological systems are inherently 3D!
Biological processes alsooccur on multiple lengthscale
![Page 3: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/3.jpg)
3D Microscopy
Deconvolution: Hiraoka, Science, 1987McNally, Methods, 1999
Confocal Microscopy: Minsky, US Patent, 1961
Two-Photon Microscopy: Sheppard et al., IEEE J of QE, 1978Denk et al., Science, 1990
![Page 4: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/4.jpg)
Understanding Optics: 4 simple rules of tracing light rays
![Page 5: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/5.jpg)
What is a microscope?
Magnification = f2/f1
This is a wide field microscopy
![Page 6: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/6.jpg)
How light focus by a microscopy objective?
Interference & Diffraction Effects are Important at the Focus
![Page 7: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/7.jpg)
Experimentally Measuring the Light Distribution at Focus
McNally, Methods, 1999
What we observe?
(1)Radial resolution--the lateral dimension is NOT
infinitely small
(2) Axial resolution--light is generated above & below the focal plane
![Page 8: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/8.jpg)
Point Spread Function – Image of an Ideal Point
Lateral Dimension: Airy function
21 ])(2[)(kr
krJkrPSF ∝
is the wave number
Axial Dimension : Sinc function2]
)()sin([)(
kzkzkzPSF ∝
λπ2
=k
FWHM2λ
≈ Resolution
![Page 9: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/9.jpg)
Depth discrimination
For a uniform specimen, we can ask how much fluorescence is generated at each z-section above and below the focal plane assuming that negligible amount of light is absorbed throughout.
u (z-axis)
v (r-axis)
∫∞
− ≡0
sec ),(2)( vdvvuPSFuFz π
Ans: Photon number ateach z-section is the same(little absorption)
The amount of light generatedat each z-section is the same!
Constant
There is no depth discrimination!!!
=− )(sec uFz
![Page 10: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/10.jpg)
What is Convolution?Recall the definition of convolution:
∫∞
∞−
−=⊗ τττ dthgthtg )()()()(
Graphical explanation of convolution:
Convolution is a smearing operation
![Page 11: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/11.jpg)
What is the effect of finite side PSF on imaging?
)()()( rPSFrOrI rrr⊗=
The finite size point spread function implies that imagesare “blurred” in 3D!!!
McNally, Methods, 1999
![Page 12: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/12.jpg)
A View of Resolution and Depth DiscriminationIn terms of Spatial Frequency
2D Fourier Transform
Power Spectrum2
)(~)(~ kIkPrr
=
∫ ∫∞
∞−
∞
∞−
++−= dxdydzzkykxkizyxIkI zyx )](2exp[),,()(~ πr
Two dimensional examples
High frequency Low frequency
![Page 13: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/13.jpg)
Convolution Theorem
)(~)(~))(( fhfgfhg =⊗ℑ
∫ ∫∫∞
∞−
−∞
∞−
∞
∞−
− −=⊗ dtedthgdtethg ftifti ππ τττ 22 )()()(
∫ ∫∞
∞−
−−∞
∞−
−
⎟⎟⎠
⎞⎜⎜⎝
⎛−= )(22 )()( τπτπ τττ tfifi etdthegd
∫ ∫∞
∞−
−∞
∞−
−
⎟⎟⎠
⎞⎜⎜⎝
⎛= )'(22 )'(')( tfifi ethdtegd πτπττ
)(~)(~ fhfg=
where dtdttt =−= '' τ
Proof in 1-D
Fourier transform of the convolution of two functions isthe product of the Fourier transforms of two functions
![Page 14: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/14.jpg)
Resolution and Discrimination in Frequency Domain
)()()( rPSFrOrI rrr⊗= Goes from convolution
To simple multiplication
Optical transfer function, OTF, is the Fourier transform of PSF.How does it looks like?
)()(~)(~ kOTFkOkIrrr
⋅=
![Page 15: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/15.jpg)
Effect of OTF on Image – Loss of Frequency Content
Effects: (1) lower amplitude at high frequency(2) completely loss of information at high frequency
Missing all info along kz axis.“Missing cone” is the origin ofno depth discrimination
![Page 16: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/16.jpg)
Deconvolution Microscopy
![Page 17: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/17.jpg)
What is Deconvolution Microscopy?
)()(~)(~ kOTFkOkIrrr
⋅=
1)()(~)(~ −⋅= kOTFkIkOrrr
)](~[)( kOrOrr -1F=
Convolution
Deconvolution
OTF is zero at high frequency…. Divide by 0???
There are many possible “O” given “I” and “OTF”This belongs to a class of “ill posted problem”
The “art” of deconvolution is to find constrains that allow the best estimate of “O”. An example of these constraints is positivity
What is the problem of this procedure?
![Page 18: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/18.jpg)
Application of Deconvolution I
McNally, Methods, 1999
Improves resolution and 3D slicing
Artifacts?
![Page 19: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/19.jpg)
Application of Deconvolution II
Raw images deconvolutedby 3 different methods
Depending on deconvolutionalgorithm chosen different“features” and “artifacts” are seen
McNally, Methods, 1999
![Page 20: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/20.jpg)
Confocal Microscopy
![Page 21: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/21.jpg)
The Invention of Confocal MicroscopyConfocal microscopy is invented by Prof. Melvin Minsky of MIT in about 1950s.
![Page 22: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/22.jpg)
Principle of Confocal Microscopy
Information comes from only a single point. Needs to move the light or move the sample!
![Page 23: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/23.jpg)
0
0.2
0.4
0.6
0.8
1
1.2
0 10 20 30 40
One-Photon
Tw o-Photon
Depth discrimination
OTF
![Page 24: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/24.jpg)
Point Spread Function – Image of an Ideal Point
Lateral Dimension: Airy function
41 ])(2[)(kr
krJkrPSFc ∝
Axial Dimension : Sinc function4]
)()sin([)(
kzkzkzPSFc ∝
The PSF of confocal is the squareof the PSF of wide field microscopy
≠=
=
∫
∫∞
∞
−
0
2
0sec
),(2
),(2)(
vdvvuPSF
vdvvuPSFuF cz
π
π
constant
![Page 25: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/25.jpg)
Early Demonstration of Confocal Microscopy in Biological Imaging
White et al., JCB 1987
![Page 26: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/26.jpg)
Tandem Scanning Confocal Microscope
Utilizes a Nipkow Disk
Holes organize in anArchimedes spiral Petran’s System
![Page 27: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/27.jpg)
A Model Tandem Confocal MicroscopeUtilizing Yokogawa Scan Head
Eliminate light throughputIssue by spinning botha plate of lenslets and nother plate of pinholes
C. Elegans
Calcium events in nerve fiber
![Page 28: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/28.jpg)
Multiphoton Microscopy
![Page 29: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/29.jpg)
Two-Photon Excitation Microscopy
(a) (b)
excitationphoton
emissionphoton
emissionphotonexcitation
photons
first electronicexcited state
electronicground state
one-photonexcitation
fluorescenceemission
fluorescenceemission
two-photonexcitation
vibrationalstates
vibrationalrelaxation
![Page 30: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/30.jpg)
A comparison of two-photon and confocal microscopes
(1) Confocal microscopes have better resolution than two-photon microscopes without confocal detection.
(2) Two-photon microscope results in less photodamage in biological specimens. The seminal work by the White group in U. Wisconsin on the development of c. elegans and hamsters provides some of the best demonstration. After embryos have been continuously imaged for over hours, live specimens are born after implantation.
![Page 31: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/31.jpg)
(3) Two-photon microscope provides better penetration into highly scattering tissue specimen. Infrared light has lower absorption and lower scattering in turbid media.
![Page 32: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/32.jpg)
In collaboration with I. Kochevar, Wellman Labs, MGH and B. Masters
![Page 33: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/33.jpg)
In collaboration with Wei Lee & Elle Nevidi, MIT
Z-Stack, Individual Slices
Reconstructed 3-D View
Computational Model of Dendrite Branches
IN VIVO IMAGING OF NEURONAL DEVELOPMENT
![Page 34: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/34.jpg)
3D Multiple Particle Tracking with Video Rate Two-Photon Microscopy
In collaboration with Ki Hean Kim (MIT)
PMT
Ti-Sa laser
Galvanometer-drivenmirror
(Y-axis scanner)
Polygonal mirror(X-axis scanner)
Objectivelens
Computer-controlledSpecimen stage
Dichroicmirror
Laser diode
Photo diode
Piezoelectrictranslator
Inside of microscope
PMT
Ti-Sa laser
Galvanometer-drivenmirror
(Y-axis scanner)
Polygonal mirror(X-axis scanner)
Objectivelens
Computer-controlledSpecimen stage
Dichroicmirror
Laser diode
Photo diode
Piezoelectrictranslator
Inside of microscope
Imaging of myocyte contraction --
R6G labeled mitochondria
In collaboration with J. Lammerding, H. Huang, K. Kim, R. Kamm, R. Lee(MIT and Brigham & Women’s Hospital)
![Page 35: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/35.jpg)
3D Quantification of Blood Flow in Solid Tumors
In collaboration with Rakesh Jain, MGH
![Page 36: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/36.jpg)
Macroscopic View of Whole Mouse Heart with Microscopic Subcellular Image Resolution
QUANTIFYING AND UNDERSTANDING GENETICALLY INDUCED CARDIAC HYPERTROPY
![Page 37: 3D Microscopy: Deconvolution, Confocal, Multiphoton Microscopy Deconvolution: Hiraoka, Science, 1987 McNally, Methods, 1999 Confocal Microscopy: Minsky, US Patent, 1961 Two-Photon](https://reader030.fdocuments.in/reader030/viewer/2022021501/5ac35fff7f8b9a12608bd214/html5/thumbnails/37.jpg)
A Comparison of The Three 3D Imaging Methods with Wide Field
$$$$$$$$$$$$Cost
++--+Uncertainty
+++---Imaging depth
YesYesYesNo3D
SimilarBetterBetter (depend on SNR)
NAResolution
MultiphotonConfocalDeconvolutionWide field