Digital Holography Ad Digital Image Processing
-
Upload
marvyn-w-inga-c -
Category
Documents
-
view
52 -
download
5
Transcript of Digital Holography Ad Digital Image Processing
DIGITAL HOLOGRAPHYAND DIGITAL IMAGE
PROCESSING
University of Alicante,Valencia, Spain,
Dec. 21, 2004
Digital holography and image processing: twinsborn by the computer era
Digital holography:- computer synthesis, analysis and simulation ofwave fieldsDigital image processing:- digital image formation;- image perfection;- image enhancement for visual analysis;- image measurements and parameter estimation;- image storage & transmission;- image visualization
•Digital reconstruction of of electronicallyrecorded optical holograms
•Target location and tracking
•Multi-component image restoration
•Image re-sampling with discrete sinc-interpolation
•Nonlinear (rank) filters for image restoration,enhancement and segmentation
•3-D visualization
DIGITAL HOLOGRAPHY:numerical reconstruction of
electronically recorded opticalholograms
Laser
Collimator
Beam spatialfilter
Lens
Microscope
Object table
DigitalPhoto-graphiccamera
Computer
One of the main drawbacks ofmicroscopy: the higher is the spatialresolution, the lower is depth of focus.
This problem can be resolved byholography.Holography is capable of recording 3-Dinformation. Optical reconstruction isthen possible with visual 3-D observation.Drawbacks of optical holography:
-Intermediate step(photographic developmentof holograms) is needed.-Quantitative 3-D analysisrequires bringing inadditional facilities
Radical solution: optical holography withhologram recording by electron means(digital photographic cameras) and digitalreconstruction of holograms. This is theprinciple of digital holographicmicroscopy.
Digital Holographic/Interferometric Microscopy
HologramFourier PlaneFirst focal plane Second focal plane
Digital Reconstruction of Holograms (Equivalent optical setup)
Hologramsensor
Preprocessingof digitizedhologram
Imagereconstruction(DFT/DFrT)
Imageprocessing
Hologram
Analog-to-digital
conversion
Outputimage
Computer
Digital Holography: Digital Reconstruction of HologramsM.A. Kronrod, N.S. Merzlyakov, L.P. Yaroslavsky, Reconstruction of a Hologram with a Computer,Soviet Physics-Technical Physics, v. 17, no. 2, 1972, p. 419 - 420
PSF, resolving power and speckle phenomena inreconstruction of electronically recorded optical holograms
Problems:
•How the point spread function of the reconstructionprocess depends on parameters of the optical set-up andrecording camera
•Resolving power of the reconstruction process
•Potential accuracy in measuring phase component of theobject wavefront
•Statistical properties of the reconstruction speckle noise
Discrete representation of optical transforms:Discrete Fourier Transforms
x
a(x) Sampled signal
xΔu
f
( )fαSampled signal spectrum
fΔ
v
( ) ( )( )xukxaxaN
kreconstr_signk Δ+−= ∑
−
=
1
0ϕ
( ) ( )( )fvrffN
kreconstr_spnr Δ+−= ∑
−
=
1
0ϕαα
Continuos signal
Continuous signalspectrum
( ) ( ) ( )dxfxiexpxaf ∫∞
∞−
= πα 2 ( )( )∑−
=⎟⎠⎞
⎜⎝⎛ ++=
1
021 N
kk
v,ur N
vrukiexpaN
πα
Fourier integral Shifted DFT (canonic form)
fx/N ΔΔ= 1
Signal and spectrum sampling
( )∑−
=⎟⎠⎞
⎜⎝⎛ +
⎟⎠⎞
⎜⎝⎛=
1
0
221 N
kk
v,ur N
rukiexpNkviexpa
Nππα ( )∑
−
=⎟⎠⎞
⎜⎝⎛ +
⎟⎠⎞
⎜⎝⎛−=
1
0
221 N
kk
v,ur N
vrkiexpNruiexpa
Nππα
Direct and inverse Shifted DFTs (reduced form)
L.P. Yaroslavsky, Shifted Discrete Fourier Transforms, In: Digital Signal Processing, Ed. by V. Cappellini, and A. G.Constantinides, Avademic Press, London, 1980, p. 69- 74.
Discrete representation of optical transforms:Discrete Fresnel Transforms
( ){ }∑−
=
+−−=1
0
2, //exp1 N
kk
wr Nwrkia
Nκκπα κ
( ){ }∑−
=
+−=1
0
2, //exp1 N
kr
wk Nwrki
Na κκπα κκ
Integral Fresnel Transform:
( ) ( ) ( )∫∞
∞−⎥⎦
⎤⎢⎣
⎡ −−= dxD
fxixaf2
exp πα
Discrete Fresnel Transforms
Signal and its transform discretizationwith shift basis functions
( ) ( )( )[ ]{ }xxukxsincxk ΔΔ+−= /πφ
( ) ( )( )[ ]{ }ffvrfsincfr ΔΔ+−= /πχ
2/1 fDxN ΔΔ= ( ) 2/1/ xf ΔΔ=κ
w is the overall shift w=uκ-v/κ 0 50 100 150 200 2500
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
q=0
q=0.0001
q=0.0002
q=0.0004
q=0.0008
q=0.0016
Absolute values of function for N=256 anddifferent “focusing” parameter q
( )rqNfrinc ;;
( ) ( )∑−
=⎟⎠⎞
⎜⎝⎛−=
1
0
2 2expexp1;;N
k Nkriqki
NrqNfrinc ππ
A pictorial representation of the discrete frinc-function(DFT of the chirp-function). Amplitude is shown in green,phase in red. Horizontal coordinate – argument of thefunction; vertical coordinate – focusing parameter. Binaryrandom noise is added just for fun.
2005
Digital reconstruction of samples of the object wave front amplitude fromsamples of its hologram is treated as a process of sampling the object wave front.Signal sampling is a linear transformation that is fully specified by its pointspread function:
( ) ( )∫=X
k dxkxPSFxaa ,
Wavefront
samples
Object’swavefront
Sampling devicePoint Spread
Function
Point spread function of numerical reconstructionof electronically recorded optical holograms
According to the sampling theorem,ideal sampling PSF is sinc-function
( ) ( )[ ]( )[ ]
( )[ ]xxkxxxkx
xxkxxkPSF
ΔΔπΔΔπ
ΔΔπ
−−=
−=sin
sinc,
For hologram recording in far diffraction zone, wave propagation kernel WP(x,f) is :
)2exp(),(Z
xfifxWPλ
π−=
Assume that, for hologram reconstruction, shifted and scaled DFT is used with the reconstructionkernel: ( )
⎥⎦⎤
⎢⎣⎡ +=
Nvrki
NrkDR T
σπ2exp1),(
where vT and σ are shift and scale parameters
With this reconstruction kernel, point spread function of the reconstruction process PSFFZ(x,k) is
( ) ⎥⎦
⎤⎢⎣
⎡⎟⎠⎞
⎜⎝⎛ −
⎭⎬⎫
⎩⎨⎧
⎥⎦
⎤⎢⎣
⎡⎟⎠⎞
⎜⎝⎛ −+−⎟
⎠⎞
⎜⎝⎛ −+−= xxkxNNvkNv
xxi
ZxkxPSF Tr
dFZ ΔσΔπ
σΔπ
λΦ ,sincd
21
212exp)(,
where ∫∞
∞−
⎟⎠⎞
⎜⎝⎛ −= df
Zxfif
Zx dd )2(exp)()(
λπϕ
λΦ
fNZSZx H ΔλλΔ == ( ) ( )( )NxN
xxNsinsin,sincd =
Define hologram discretization and reconstruction device coordinate system through the objectcoordinate system by choosing vr=vT=(N-1)/2. Then
( ) ⎥⎦
⎤⎢⎣
⎡⎟⎠⎞
⎜⎝⎛ −= xxkxN
ZxkxPSF r
dFZ ΔσΔπ
λΦ ,sincd)(,
is frequency response of the hologram sampling device and
PSF of reconstruction of holograms recorded infar diffraction zone
L. Yaroslavsky, F. Zhang, I. Yamaguchi, Point spread functions of digital reconstruction of digitally recorded holograms,In: Photonics Asia conference on Information Optics and Photonics Technology, 8-12 November 2004, Beijing, China
( ) ( )[ ]xxkxNZxkxPSF r
dFZ ΔΔπλ
Φ −= ,sincd)(,
PSF of reconstruction of holograms recorded in fardiffraction zone (ctnd)
As one can see from the equation,
The point spread function is a periodical function of k:
( ) ( ) ( ) ( );1 1 kPSFNgkPSF FZNgr
FZ r −−=+ σ
(g is integer). It generates σN samples of object wavefront masked by the frequency response of thehologram recording and sampling device, the samples being taken with discretization interval
Δx/σ = λZ/ σSH =λZ/ σNΔf
within the object size So= λZ/ Δf.
The case σ =1 corresponds to a “cardinal” reconstructed object wavefront sampled with discretizationinterval Δx= λZ/ SH =λZ/ NΔf . When σ >1 , reconstructed discrete wavefront is σ -times over-sampled,or σ -times zoomed-in. One can show that in this case the reconstructed object wavefront is a discretesinc-interpolated version of the “cardinal” one.
Discrete sinc-function ( ) ( )( )xNxN
xxxxNΔ
ΔπΔπsin
sin,sincd =
Discrete sinc-function is a discrete analog of the continuous sampling sinc-function, which is a pointspread function of the ideal low-pass filter. As distinct from the sinc-function, discrete sinc-function is aperiodical function with period NΔx or 2NΔx depending on whether N is an odd or an even number andits Fourier spectrum is a sampled version of the frequency response of the ideal low pass filter
N is an oddnumber
N is an evennumber
Continuous (red dots) and discrete (blue line)sinc-functions for odd and even number ofsamples N
NΔx 2NΔx
Frequency response of the ideal low pass filter(red) and Fourier transform of the discretesinc-function (blue)
Reconstructed object areaReconstructed object area
Statistical characterization of speckle noise incoherent imagingL. Yaroslavs ky, A. Shefler, Statistical characterization of speckle noise in coherent imaging systems, in: OpticalMeasurement Systems for Industrial Inspection III, SPIE’s Int. Symposium on Optical Metrology, 23-25 June 2003,Munich, Germany, W. Osten, K. Creath, M. Kujawinska, Eds., SPIE v. 5144, pp. 175-182
2-D array that specifiesamplitude component of
the object wave front
Generating 2-D array ofpseudo-random numbers
that specify the phasecomponent of the object
wave front
Computingobject’s wave
front
Simulating wavefront propagation
(DFT, DFrT)
Introducing signaldistortions:-Array sizelimitation-Dynamic rangelimitation-Quantization
Simulating wavefront
reconstruction(IDFT, IDFrT)
Comparingreconstructed and initialwave fronts; computing
and accumulation ofnoise statistical
parameters
Continueiterations?
Yes
NoOutput data
Illustrative examples of simulated images: a) - originalimage; b) - image reconstructed in far diffraction zone from0.9 of area of the wave front; c) - image reconstructed in fardiffraction zone from 0.5 of area of the wave front; d) -image reconstructed in far diffraction zone after limitation ofthe wave front orthogonal components in the range.
Computer model
a) b)
c) d)
TARGET LOCATION ANDTRACKING
Target localization in clutter in multi-componentimagesL. Yaroslavsky, Optimal target location in color and multi component images, Asian Journal of Physics, Vol. 8, No 3(1999) 355-369
Object tracking in video sequencies: examples
Tracking fetus movements in Ultrasound movie
For details see http://www.eng.tau.ac.il/~yaro
Leonid P. Yaroslavsky, Ben-Zion Shaick Transform Oriented Image Processing Technology for Quantitative Analysisof Fetal Movements in Ultrasound Image Sequences. In: Signal Processing IX. Theories and Applications, Proceedingsof Eusipco-98, Rhodes, Greece, 8-11 Sept., 1998, ed. By S. Theodorisdis, I. Pitas, A. Stouraitis, N. Kalouptsidis,Typorama Editions, 1998, p. 1745-1748
Face detection in complex imagesBen-Zion Shaick, L. Yaroslavsky, Object Localization Using Linear Adaptive Filters, 6th Fall Workshop,Vision, Modeling And Visualization 2001 (Vmv01), November 21-23, 2001, Stuttgart, GermanyStuttgart,Germany, November 21-23, 2001, pp. 11-17
http://www.eng.tau.ac.il/~yaro/Zion/ZionPhdSeminar4Web.pdf
The developed algorithm iscapable of detecting, withhigh reliability, faces ofvarying size from minimumsize of 12 pixels width and15 pixels height to themaximum size of the inputimage.
The face detection capabilityof the developed system wasexperimentally examined ontwo test databases of imagesof high and low quality. Thedetection rates 96% and84% were achieved for thesedatabases, respectively.
Non-face
1st stage
Face
Maybe face
2-nd stage
Face-like-non-face
Non-face -like-face
Face detection: two-stage algorithm
The “non-face” detection algorithm was proved to have “non-face” rejecting rate of ~99% and false alarm rate of1.3% (faces wrongly rejected), thus leaving only 1% of the image area for subsequent thorough analysis by the “multi-template classification” algorithm. The algorithm is fast and requires approximately 200 flops per pixel in an inputimage of 640480 pixels size.
“Multi-templateclassification” algorithms usea very large set of templatesprepared for different targetshapes and varyingillumination conditions.The developed algorithmswere trained using a speciallycreated training databaseobtained by extending four“face” databases to 32,000images and one “non-face”database to one millionimages by means of scalingand rotating database images.In particular, “face”, “non-face”, “faces like clutter” and“clutter like faces” templateswere generated from thesetraining databases.
Face detection: Face-like-non-face and non-face-likeface data bases
MULTI COMPONENTIMAGE RESTORATION
Spatial/temporal adaptive linear filtersL. Yaroslavsky, Image Restoration, enhancement and target location with local adaptive filters, in:International Trends inOptics and Photonics, ICOIV, ed. by T.Asakura, Springer Verlag, 1999, pp. 111-127
3-D Local adaptive spatial-temporal filtering:denoising and deblurring thermal video (ctnd)
3-D Local adaptive spatial-temporal filtering:denoising and deblurring thermal video
Before After(5x5x5 DCT domain filtering)
L. Yaroslavsky, A. Stainman, B. Fishbain, Sh. Gepstein, Processing and Fusion of Thermal and Video Sequences forTerrestrial Long Distance Observation Systems Processing and Fusion of Thermal and Video Sequences forTerrestrial Long Distance Observation Systems, ISIF, Seventh International Conference on Information Fusion(FUSION 2004), Stockholm, Sweden, 28 June - 1 July 2004 .
IMAGE RESAMPLING:discrete sinc-interpolation
Image resampling and geometrical transformations:efficient discrete sinc-interpolation algorithms
Spline interpolationand discrete sinc-interpolation
Spectra of DFT-error (le ft) and Beta11-error (right)
Test random image
Spectrum of DFT-error (red) andBeta11 error (green)
Spectrum of DFT-error (green)and bicubic-error (red)
Discrete sinc-interpolation for image resizing
Image (a) is reduced with reduction factor 77/256 and then magnified with magnification factor256/77. b) – bilinear interpolation; c) – bicubic interpolation; d) – nearest neighbor interpolation;e) – sincd-interpolation 11x11; f) – sincd interpolation 31x31
Discrete sinc-interpolation in DCT domain
Initial signal ( )ka
DCT
DCTrα
{ }DCTrα
DFT of zero-pad
sinc-interpolation
kernel
( ){ }prη
IDST
p-shifted sinc-interpolated signal{ }pka +
IDCT
( ){ }prerη
( ){ }pimrη
+ - Zooming an image fragment (left) by sinc-interpolationin DFT domain (right upper image) and in DCT domain(right bottom image). Oscillations due to boundaryeffects that are clearly seen in DFT-interpolated imagecompletely disappear in DCT-interpolated image.
L. Yaroslavsky, Boundary effect free and adaptive discrete sinc-interpolatioon, Applied Optics, 10 July 2003, v. 42, No.20, p.4166-4175
Sliding window sinc-interpolationDCT domain: Simultaneous imageresampling and restoration/enhancement
Inputsignal
InverseDCT/DST
∑Computing slidingwindow
DCT
Outputsignal
Introducing p-shift
Modification ofthe spectrum for
restoration/enhancement
Sliding window sinc-interpolation in DCTdomain: signal resampling and denoising
Noisy image (a) and a result of the rotation and denoising withsliding window DCT sinc-interpolation and denoising (b).
a)
b)
Outputsignal
Computingsliding window
DCT
Discrete DCT domainsinc-interpolator
Nearest neighborinterpolator
Analysis of localspectrum
Inputsignal
Mixer
Sliding window sinc-interpolationDCT domain: local adaptive interpolation
Adaptive versus non adaptive signal interpolation
Comparison of nearest neighbor, linear, bicubic spline and adaptivesliding window sinc interpolation methods for zooming a digital signal(From left to right, from top to bottom: Continuous signal; initialsampled signal; nearest neighbor -interpolated signal ; linearly -interpolated signal; cubic spline -interpolated signal; sliding windowsinc-interpolated signal ).
Signal (upper plot) shift by non-adaptive (middle plot) and adaptive(bottom plot) sliding window DCT sinc-interpolation. One can noticedisappearance of oscillations at the edges of rectangle impulses wheninterpolation is adaptive.
Image rotation with adaptive and non-adaptivediscrete sinc interpolation
Direct Fourier method for inverse Radon transform:polar-to- Cartesian coordinate spectrum conversion withdiscrete sinc-interpolation
Object
Projections
Projection spectra
Interpolated 2-Dobject spectrum
Reconstructedimage
Fourier method for inverse Radon transform:polar-to- Cartesian coordinate spectrum conversion bymeans of zooming with variable zooming factor(L. P. Yaroslavsky, Y. Chernobrodov, Sinc-interpolation methods for Direct Fourier Tomographic Reconstruction, 3-d Int.Symposium, Image and Signal Processing and Analysis, Sept. 18-20, 2003, Rome, Italy)
Initial Polar Grid
Polar Grid after 2time zooming
Target Cartesian Grid
??
a
b
r1
r2
Initial Polar Grid
Target Cartesian Grid
1-D spectra of projections
radiusangle
S. Gepshtein, A. Shtainman, B. Fishbain, and L.P. Yaroslavsky, Restoration of atmospheric turbulent video containingreal motion using rank filtering and elastic image registration, Eusipco2004, Vienna, Austria, Sept. 2004
Stabilization and restoration of atmosphericturbulent video
Initial videoRestored stabilized video
with moving objectsunaffected
Moving objects
Turbulent atmosphere video
Stabilized turbulent atmosphere video
NONLINEAR (RANK)FILTERS
for image restoration,enhancement and segmentation
Image restoration and enhancement: nonlinear filtersL. Yaroslavsky, Nonlinear Filters for Image Processing in Neuromorphic Parallel Networks, Optical Memory and NeuralNetworks, v. 12, No. 1, 2003B. Hirshl, L. Yaroslavsky, FPGA implementation of sorters for non-linear filters, EUSPCO 2004, Vienna, Austria, Sept. 2004
Noisy image, stdev = 20, Pn=0.15
Iterative SCSigma-filter . Wind.5x5, Evpl=Evmn=15; 5 iterations
Initial image SIZE(Evnbh(Wnbh5x5,2,2))-filter HIST(W-nbh)-filter
Nonlinear filters: Image enhancement
Initial image
RANK(KNV(Wnbh15x15;113))
RANK(Wnbh15x15)
RANK(EV(Wnbh15x15;10,10))
Nonlinear Filters: Image EnhancementLocal histogram equalization: Wnbh, EV-nbh and KNV-nbh
Local P-histogram equalization: color images(blind calibration of CCD-camera images)
3-D VISUALIZATION:improved anaglyph method and
3-D video from 2-D video
Computer synthesis and display of stereoscopic imagesI. Ideses, L. Yaroslavsky, Efficient Compression and Synthesis of Stereoscopic Video, 2nd IASTED Internat.Conference, Visualization, Imaging and Image Processing (VIIP 2002), Sept. 9-12, 2002, Malaga, Spain, Ed. J.J.Villanueva, Acta Press, Anaheim, Calgary, Zurich, 2002, pp. 191-194 (ISBN: 0-88986-354-3; ISSN: 1482-7921)http://www.eng.tau.ac.il/~yaro/Ideses/malca1.html
Bahai garden, Haifa, Israel
Roman Querry, Tarragona, Catalonia
Computer generated stereo from 2-D videoI. Ideses, L. Yaroslavsky, A Method for Generating 3D Video from a Single Video Stream, in: G. Greiner, H.Niemann, T. Ertl, B. Girod, H.-P. Seidel (Eds.), Vision Modeling and Visualization 2002, Proceedings, Nov. 20-22,2002, Erlangen, Germany, Academishe Verlagsgesellschaft Aka GmbH, Berlin, 2002, ISBN 3-89838-034-3, ISBN 1-58603-302-6, p. 435-439
Current projects:
Multi-component image processing:•Restoration and fusion of atmospheric turbulent, thermal and visible range video•Super-resolution from video sequencies•3-D color display and artificial 3-D from video•Moving object detection and tracking
Digital holography:•Point spread functions and resolving power of digital reconstructionof near and far zone holograms
•Theory and new algorithms for fast transforms
L. Yaroslavsky,Ph.D., Dr. Sc. Phys&Math,
ProfessorDept. of Interdisciplinary Studies,Faculty of Engineering, Tel Aviv
University, Tel Aviv, Israelwww.eng.tau.ac.il/~yaro