Department of Electronic Engineering Faculty of Engineering and...
157
Applications of Compressed Sensing to Biomedical Imaging By Jawad Ali Shah Reg. No. 44-FET/PHDEE/F11 A dissertation submitted to I.I.U. in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Electronic Engineering Faculty of Engineering and Technology INTERNATIONAL ISLAMIC UNIVERSITY ISLAMABAD 2015
Transcript of Department of Electronic Engineering Faculty of Engineering and...
Applications of Compressed Sensing to
Biomedical Imaging
By
Jawad Ali Shah
Reg. No. 44-FET/PHDEE/F11
A dissertation submitted to I.I.U. in partial fulfillment of the requirements for the degree of
DOCTOR OF PHILOSOPHY
Department of Electronic Engineering
Faculty of Engineering and Technology
INTERNATIONAL ISLAMIC UNIVERSITY ISLAMABAD
2015
i
Copyright © 2015 by J. Shah
All rights reserved. No part of the material protected by this copyright notice may be
reproduced or utilized in any form or by any means, electronic or mechanical, including
photocopying, recording or by any information storage and retrieval system, without
the permission from the author.
ii
DEDICATED TO
My Teachers,
Students
And
Friends
iii
CERTIFICATE OF APPROVAL
Title of Thesis: Applications of Compressed Sensing to Biomedical Imaging
Name of Student: Jawad Ali Shah
Registration No: 44-FET/PHDEE/F11
Accepted by the Department of Electronic Engineering, Faculty of Engineering
and Technology, International Islamic University, Islamabad, in partial fulfillment of
the requirements for the Doctor of Philosophy degree in Electronic Engineering.
Viva voce committee:
Prof. Dr. Ijaz Mansoor Qureshi (Supervisor)
Department of Electrical Engineering
Air University, Islamabad
Dr. Ihsanul Haq (Co-supervisor)
Assistant Professor, DEE, FET, IIUI
iv
ABSTRACT
The application of compressed sensing (CS) to biomedical imaging is exciting because
it allows a reasonably accurate reconstruction of images from far fewer measurements.
For biomedical imaging, CS can increase the imaging speed and consequently decrease
the radiation dose. While the idea of CS has been used to reduce the acquisition time of
magnetic resonance imaging (MRI), x-ray computed tomography (CT) and microwave
imaging (MWI), unfortunately the computation time of image recovery has increased
as the nonlinear CS reconstruction algorithms are fairly slow. Reconstructing high-
dimensional signals or biomedical images from compressively sampled data is a
fundamental challenge faced by the CS.
In this dissertation, we propose a suite of novel CS recovery methods that can efficiently
recover the Fourier encoded biomedical images (MRI, parallel-beam CT and MWI)
from a small set of randomized measurements. The initial part of the current work
presents CS based reconstruction of sub-sampled biomedical imaging modalities using
projection onto convex sets (POCS) and separable surrogate functional (SSF) methods.
The iterative shrinkage based SSF algorithm incorporates the linear estimate of the error
to improve the reconstruction quality. It does not involve any matrix inversion and is
used to estimate the missing Fourier samples of the original image by applying data
consistency in the frequency domain and soft thresholding in the sparsifying domain.
The idea of using hybrid evolutionary techniques for the sparse signal recovery is
presented next. It proposes how to combine the heuristic techniques such as Differential
evolution (DE), genetic algorithms (GA), and Particle Swarm Optimization (PSO) with
v
iterative shrinkage algorithms to faithfully reconstruct sparse signals from a small
number of measurements. Based on the notion of GA, a modified POCS based
algorithm is developed. This novel CS recovery technique uses two different estimates
for the initialization and iteratively combines them to recover the original Fourier
encoded image.
In the last part, we use hyperbolic tangent function separately to develop a
reconstruction algorithm and a non-linear shrinkage curve for thresholding. As the 𝑙1-
norm penalty is not differentiable, the proposed hyperbolic tangent based function is
used to closely approximate the 𝑙1-norm regularization by a differentiable surrogate
function. Using the method of gradient descent, a simple update rule is developed. The
algorithm is shown to perform well for one dimensional (1-D) sparse signal recovery
as well as CS reconstruction of Fourier encoded biomedical imaging. The idea is further
extended by using hyperbolic tangent based approximations for the soft-thresholding
that provide flexibility in terms of its adjustable parameters. Besides using synthetic
data, the effectiveness of the proposed techniques are also validated using the real data
collected from the MRI and MWI scanners.
vi
LIST OF PUBLICATIONS AND SUBMISSIONS
[1] J. Shah, I. Qureshi, H. Omer, and A. Khaliq, "A modified POCS‐based
reconstruction method for compressively sampled MR imaging," International
Journal of Imaging Systems and Technology, vol. 24, pp. 203-207, 2014, John
Wiley & Sons.
[2] J. Shah, I. Qureshi, H. Omer, A. Khaliq, and Y. Deng, "Compressively sampled
MR image reconstruction using separable surrogate functional method",
Concepts in Magnetic Resonance, Part A, John Wiley & Sons. 2014. DOI:
10.1002/cmr.a.21314.
[3] J. Shah, I. Qureshi, J. Proano, Y. Deng,"Compressively Sampled Fourier
encoded Image Reconstruction using hyperbolic tangent based soft-
thresholding", 'Applied Magnetic resonance'. DOI: 10.1007/s00723-015-0683-
2.
[4] J. Shah, I. Qureshi, Y. Deng, A. Khaliq, "Reconstruction of Sparse Signals and
Compressively Sampled Biomedical Images Based on Smooth 𝑙1-Norm
Approximation", sumbitted to 'Journal of Signal Processing Systems'.
[5] J. Shah, I. Qureshi, A. Khaliq, H. Omer, "Sparse Signal Recovery based on
Hybrid Genetic Algorithm" , Research Journal of Recent Sciences, vol. 3(5),
pp. 86-93, 2014, ISSN 2277-2502.
[6] J. Shah, I. Qureshi, A. Khaliq, H. Omer, "Sparse Signal Reconstruction from
Compressed Measurements using Hybrid Differential Evolution", World
Applied Sciences Journal, vol. 27 (12), pp. 1614-1619, 2013, ISSN 1818-4952.
[7] J. Shah, I. Qureshi, A. Khaliq, M. Iqbal, "Adaptive equalization of a nonlinear
recursive channel using Gaussian RBF based network". IEEE 15th
International Mutlitopic Conference (INMIC) 2012.
vii
[8] H. Haider, J. Shah, U. Ali, " Comparative Analysis of Sparse Signal Recovery
Algorithms based on Minimization Norms". World Congress on Sustainable
Technology 2014.
[9] H. Haider, J. Shah, I.Qureshi, H. Omer, "Compressively Sampled MRI
Recovery using Modified Iterative Reweighted Least Square Method”,
submitted to 'Inverse Problem and Imaging Journal’.
[10] S. Khursheed, A. Khaliq, S. Abdullah, J. Shah, S.Khan “Poisson noise
removal using wavelet transforms”, Science International.
[11] A. Khaliq, I. Qureshi, J. Shah, “Source Extraction from Functional magnetic
resonance imaging (fMRI) data using ICA based on Fourth order and
Exponential contrast functions” , Journal of the Chinese Institute of Engineers,
vol. 36 (6), 730-742, 2013.
[12] A. Khaliq, J. Shah, S. Malik, “De-noising of Functional Magnetic Resonance
Imaging (fMRI) data using Nonlinear Anisotropic 1D and 2D filters”,
International journal of advanced research in computer science, vol. 4 (4), pp.
13-17, 2013.
[13] A. Khaliq, I. Qureshi, Ihsanulhaq, J. Shah, “Detection of brain activity in
functional magnetic resonance imaging data using matrix factorization”,
Research journal of applied sciences, engineering and technology, vol. 5(24),
pp. 8566-8571 , 2013
[14] A. Khaliq, I. Qureshi, J. Shah, S. Malik, “High order Differential Covariance
based source separation of Monkey’s fMRI data”, International Journal of
Engineering Sciences and Research Technology, vol. 2 (5), pp. 1287-1292,
2013
[15] A. Khaliq, I. Qureshi, J. Shah, S. Malik, Ihsanulhaq, “De-Noising Functional
Magnetic Resonance Imaging (fMRI) data using Exponential Gradient filter”,
Middle east journal of scientific research, vol. 18 (9), pp. 1349-1356, 2013
[16] S. Khursheed, A. Khaliq, J. Shah, S. Abdullah, S. Khan, “Third order NLM
filter for Poisson noise removal from medical images”, MAGNT Research
Reviews. vol.2 (7): pp. 482-48
viii
[17] A. Khaliq, I. Qureshi, J. Shah,, “Un-mixing Functional Magnetic Resonance
Imaging (fMRI) data using Matrix Factorization”, International journal of
imaging system and Technology, vol. 22 (4), pp. 195-199, 2012 John Wiley &
Sons.
[18] A. Khaliq, I. Qureshi, J. Shah, “Temporal Correlation based spatial filtering of
Functional MRIs”, Chinese Physics Letters, vol. 29(1), 2012.
ix
Awards and Grants
Award of scholarship by Higher Education Comission (HEC) Pakistan under
“International Research Support Initiative Program (IRSIP)” for six months
research fellowship at Electromagnetic and Accoustic Imaging and Prognostics
(LEAP) lab of the University of Colorado Denver/Anschutz Medical Campus
USA covering:
Stipend (9000 USD)
Bench fee (2500 USD)
Travel grant (150 USD)
Full fee waiver for PhD studies by the International Islamic University Islmabad
(IIUI), Pakistan.
Grant of study leave for nine months with full pay.
x
ACKNOWLEDGEMENTS
This dissertation would have not been written without the consistent support and
encouragement from many colleagues, family members and friends that surround me.
First, I would like to express my gratitude to Dr. Ijaz Mansoor Quershi, my PhD
supervisor, mentor, teacher, brother and friend. He has been an inspiring figure and
role model for me since I joined the teaching profession. I am also grateful to my foreign
supervisor, Dr. Yiming Deng, for his valuable suggestions and ideas that has a great
influence on my research work. I really appreciate the unbounded support of Dr. Julio
Proano for providing useful and valuable comments, reviews and suggestions about my
work. I owe a lot to Dr. Hammad Omer who introduced me to the fascinating area of
MRI and provided access to original MR scanner data.
It has been an absolute pleasure and honor to be the student of Dr. Aqdas Naveed Malik
from whom I learned and benefited a lot. He is one of my ideal teachers. The role and
support of Dr. Ihsanul Haq during my PhD course work was exceptional. His
encouraging approach is always a source of motivation for me.
My sincere thanks also goes to my colleagues: Dr. Muhammad Amir, Dr. Adan Umer,
Dr. Saeed Badshah, Dr. Muhammad Afzal Khan and Dr. Suheel Abdullah. I have been
fortunate to receive their long-standing support during each stage of PhD and
throughout difficult times. I am also grateful to Engr. Hassaan Haider and Engr.
Shahid Ikram for the experience of working with them, guiding them and learning so
much from them in the process.
I would like to acknowledge and express my gratitude to Higher Education Commission
(HEC) Pakistan for providing me the opportunity to conduct my six months research at
the UC Denver, USA. I am also indebted to Pakistani-origin US citizens: Ehsan Gillani,
Tahseen Ibrahim, Faqir Mohammad (FM), Ahmad Zia, Muhammad Imran, Ahsan
Awan, Naseeer and Rukhsana Baji who provided me a home-like environment in
Denver and never realized me that I am away from Pakistan.
I would like to acknowledge the support of International Islamic University Islamabad
Pakistan for providing me nine months study leave and full fee waiver during the PhD
studies. It was indeed a generous award and investment from my university.
xi
The biggest thanks goes to my family members: My father (late), mother, sisters and
brothers. It is only because of their love, never-ending support and prayers that I could
do this or anything else. I owe a lot to my devoted wife for her unending encouragement
and patience. Finally, I express my special graduate to my kids: Ahmad Mujtaba, Abdul
Mohaimin and the newcomer Abdul Mannan for always making me smile and allowing
me to utilize their time in my research against their wishes!
(Jawad Ali Shah)
xii
Table of Contents
Abstract ......................................................................................................................... iv
List of Publications and submissions ............................................................................ vi
Awards and Grants ........................................................................................................ ix
ACKNOWLEDGEMENTS ........................................................................................... x
List of figures .............................................................................................................. xvi
List of Tables .............................................................................................................. xxi
List of abbreviations .................................................................................................. xxii
CHAPTER-1 Introduction ............................................................................................ 1
1.1 Compressed Sensing ....................................................................................... 3
1.2 Main Contributions ......................................................................................... 4
1.3 Thesis Outline ................................................................................................. 6
CHAPTER-2 Sparse Representation and Compressed Sensing .................................... 8
2.1 Sparse Representation ..................................................................................... 8
2.1.1 Sparsity: From Basis Expansion to Sparse Representation ............................. 8
2.2 Compressed Sensing for Biomedical Images ................................................ 11
2.2.1 Sparsifying transforms for biomedical images ...................................... 12
2.2.2 Non-Linear Recovery............................................................................. 25
xiii
2.3 Related Work................................................................................................. 27
2.4 Quality assessment parameters...................................................................... 30
2.5 Summary ....................................................................................................... 31
CHAPTER-3 Biomedical Imaging Modalities and Projection onto Convex Sets Based
CS recovery .................................................................................................................. 32
3.1 Magnetic Resonance Imaging (MRI) ............................................................ 32
3.2 Parallel beam CT ........................................................................................... 38
3.2 Microwave Imaging (MWI) .......................................................................... 40
3.3 POCS based Recovery of Fourier encoded images ....................................... 43
3.4 Summary ....................................................................................................... 49
CHAPTER-4 Compressively sampled Fourier-encoded image reconstruction using
Seperable Surrogate Functional ................................................................................... 50
4.1 Rapid imaging and compressed sensing........................................................ 50
4.2 𝑙1-regularized least square problem for Orthonormal basis .......................... 52
4.3 Proposed Recovery algorithm ....................................................................... 54
4.4 Recovery of MR images using the proposed algorithm ................................ 56
4.5 Recovery of parallel beam CT using the proposed method .......................... 61
4.6 Reconstruction of MWI using SSF based method ....................................... 63
4.7 Summary ....................................................................................................... 65
Chapter-5 Sparse signal reconstruction using hybrid evolutionary algorithms ........... 67
xiv
5.1 Evolutionary algorithms ................................................................................ 67
5.2 The Proposed Hybrid Particle Swarm Optimization ..................................... 70
5.3 Proposed Hybrid Genetic Algorithm ............................................................ 72
5.4 Results and Discussions ................................................................................ 73
5.4 Proposed modified POCS algorithm for biomedical images ........................ 76
5.5 Summary ....................................................................................................... 80
Chapter-6 CS Recovery Based on Smooth 𝑙1-Norm Approximation .......................... 81
6.1 Problem Statement ........................................................................................ 81
6.2 Proposed hyperbolic Tangent based surrogate function ............................... 82
6.3 Proposed Recovery Algorithm ...................................................................... 85
6.4 Results of Simulation with Discussions ....................................................... 87
6.5 Summary ....................................................................................................... 96
Chapter-7 A flexible soft thresholding for Iterative Shrinkage algorithms ................ 98
7. 1 CS Recovery and denoising .......................................................................... 98
7.2 MAP estimator for denoising and proposed thresholding ........................... 102
7.3 CS recovery of biomedical imaging (Denoising approach) ........................ 106
7.4 Experimental results .................................................................................... 107
7.5 Summary ..................................................................................................... 112
CHAPTER-8 Conclusion ........................................................................................... 113
8.1 Summary of thesis ....................................................................................... 113
xv
8.2 Directions for future work ........................................................................... 115
References .................................................................................................................. 117
xvi
LIST OF FIGURES
Figure-2.1: DCT basis 𝜓𝑖,𝑗(𝑙, 𝑝) for 0 ≤ 𝑖, 𝑗 ≤ 7 ........................................................ 13
Figure-2.2: Original and reconstructed microwave image from 10% DCT coefficients
...................................................................................................................................... 14
Figure-2.3: Block diagram of the QMF algorithm for DWT ....................................... 15
Figure-2.4: Block diagram for IDWT using QMF algorithm ...................................... 16
Figure-2.5: Original MR Image of human head .......................................................... 17
Figure 2.6: 7-bands of Daubechies 2-tap filters when applied to brain MRI. ............. 17
Figure-2.7: Sorted coefficients of MRI in pixel and wavelet domain. Only 1000
significant values are shown for comparsion ............................................................... 18
Figure-2.8: Gradient based sparsity of Shepp-Logan image ....................................... 19
Figure-2.9: Aliasing due to linear reconstruction of MRI from its partial Fourier data
...................................................................................................................................... 22
Figure-2.10: Different domains and transformations in CS......................................... 22
Figure-2.11: Behavior of scalar function |𝛼|𝑝 ............................................................. 26
Figure-3.1 Pulse sequence diagram representing various gradients for spatial encoding
...................................................................................................................................... 34
Figure-3.2: Fourier encoded MR image ....................................................................... 36
Figure-3.3: Aliasing due to the uniform undersampling of k-space ............................ 37
Figure-3.4: Effect of undersampling the k-space on reconstructed image ................... 38
xvii
Figure-3.5: Projections in parallel beam CT ................................................................ 39
Figure-3.6: Radial lines used to sample the Fourier transform of an object for CT
imaging ........................................................................................................................ 41
Figure-3.7: Measurement configuration for MWI ....................................................... 42
Figure-3.8: POCS algorithm (with block diagram) for Fourier-encoded image recovery
...................................................................................................................................... 44
Figure-3.9: Recover of compressively sampled MRI using POCS ............................. 46
Figure-3.10: Effect of thresholding parameter on the quality (MSE) of reconstructed
MR image..................................................................................................................... 47
Figure-3.11: Original MWI along with the undersampling pattern ............................. 48
Figure-3.12: POCS based recovery of compressively sampled MWI ......................... 48
Figure-4.1: The proposed SSF based recovery algorithm (with block diagram) ......... 56
Figure-4.2: Original phantom and brain images taken from the MRI scanner. ........... 57
Figure-4.3: Recovery (phantom MRI) using various techniques................................. 58
Figure-4.4: Recovery of phantom image with proposed algorithm. ............................ 59
Figure-4.5: Comparison (phantom MR image) based on Structural similarity (SSIM)
...................................................................................................................................... 59
Figure-4.6: Comparison based on ISNR (brain image) ............................................... 60
Figure-4.7: Comparison on the basis of fitness value (brain image) ........................... 60
Figure-4.8: Difference of the recovered images with the original brain image. .......... 61
Figure-4.9: CT phantom image (a) and corresponding sinogram (b) .......................... 62
xviii
Figure-4.10: (a) undersampling pattern (b) aliased CT image ..................................... 62
Figure-4.11: Final reconstructed CT image with various algorithms .......................... 63
Figure-4.12: fully sampled SAR image of SUT .......................................................... 64
Figure-4.13: Selective raster scanning for MWI acquisition ....................................... 64
Figure-4.14: Under-sampled (top) and recovered microwave image (bottom) ........... 65
Figure-5.1: Flow chart of generic GA.......................................................................... 69
Figure-5.2: The proposed hybrid PSO algorithm ........................................................ 71
Figure-5.3: The proposed hybrid genetic algorithm for sparse signal recovery .......... 72
Figure-5.4: Performance of PSO with varying inertial weights .................................. 73
Figure-5.5 showing fast convergence achieved with the proposed method ................ 74
Figure-5.6: Signal reconstruction through conventional and hybrid PSO ................... 75
Figure-5.7: Comparison of hybrid GA based on mean-square-error ........................... 75
Figure-5.8: Modified POCS based algorithm for CS recovery .................................... 77
Figure-5.9: Final image recovery with modified POCS algorithm .............................. 78
Figure-5.10: Comparison of proposed algorithm with POCS and SSF ....................... 79
Figure-6.1 Comparison of approximation for different values of 𝛾 ............................ 84
Figure-6.2 Mean-square error of the approximation for different bounds of signal .... 84
Figure-6.3: Graphical solution to Eq-6.7 corresponding to 𝛾 = 50 . .......................... 86
Figure-6.4: Proposed hyperbolic tangent based iterative algorithm for (1-D) sparse
recovery........................................................................................................................ 87
xix
Figure-6.5: Compressively sampled sparse signal ....................................................... 88
Figure-6.6: Recovery with the proposed reconstruction algorithm ............................. 89
Figure-6.7: Signal recovery with and without knowledge of sparsity ......................... 90
Figure-6.8: Proposed hyperbolic tangent based algorithm for CS recovery of Fourier
encoded imaging .......................................................................................................... 91
Figure-6.9: Phantom image and the under-sampling pattern used .............................. 92
Figure-6.10: Recovery of phantom image (parallel-beam CT) with proposed algorithm
...................................................................................................................................... 92
Figure:-6.11: Original human head MR image and the corresponding sampling pattern
...................................................................................................................................... 93
Figure-6.12: Recovery of original MR image using proposed algorithm .................... 94
Figure-6.13: SSIM improvement of the reconstruction images in each iteration ........ 94
Figure-6.14: SAR image and corresponding under-sampling pattern used ................. 95
Figure-6.15: Under-sampled and recovered microwave image ................................... 96
Figure-7.1: Transformed based denoising ................................................................... 99
Figure:-7.2: Histogram of error (between original Fourier encoded image and linear
reconstructed image) for (a) parallel beam CT (b) MRI ............................................ 101
Figure-7.3: Reduction in the variance of error during CS recovery .......................... 101
Figure-7.4: Versatility of the hyperbolic-tangent based soft-thresholding ................ 104
Figure-7.5: Approximation of Eq-7.10 using the proposed soft-thresholding (Eq-7.11)
.................................................................................................................................... 104
Figure-7.6: performance of the proposed thresholding function ............................... 105
xx
Figure-7.7: A general denoising based CS recovery algorithm ................................. 107
Figure-7.8: Distribution of pixel values of the original MR image ........................... 108
Figure-7.9: Distribution of pixel values of the original undersampled image ........... 109
Figure-7.10: Distribution of pixel values of the reconstructed image ....................... 109
Figure-7.11: Improvement in correlation using soft (Eq-7.10) and proposed (Eq-7.11)
thresholding function for MR recovery ..................................................................... 110
Figure-7.12: Reconstruction of MR phantom image using thresholding of Eq-7.11 111
Figure-7.13: Comparison based on SSIM using two different thresholding ............. 111
Figure-7.14: SSF based Original MR Image reconstruction using soft-thresholding (Eq-
7.10) and the proposed (Eq-7.11) .............................................................................. 112
xxi
LIST OF TABLES
Table-3.1 Values of MSE and correlations attained by the final MR image ............... 47
Table-4.1: Comparison of MR brain image based on PSNR, AP and correlation....... 61
Table-5.1: Values of various parameters such as correlation, MSE etc. achieved by
different recovery algorithms ....................................................................................... 76
Table-5.2: Comparison of algorithms for MR image reconstruction .......................... 79
Table-6.1: Comparison based on various performance metrics................................... 96
xxii
LIST OF ABBREVIATIONS
AP Artifact Power
BP Basis Pursuit
CoSaMP Compressively Sampled Matching Pursuit
CT Computed Tomography
CS Compressed (or compressive) Sensing (or sampling)
DAQ Data Acquisition
dB Decibel
db2 Daubechies-2
DCT Discrete Cosine Transform
DE Differential Evolution
DFT Discrete Fourier Transform
DWT Discrete Wavelet Transform
FBP Filtered Back Projection
FID Free Induction Decay
FOV Field of View
fMRI Functional Magnetic Resonance Imaging
GA Genetic Algorithms
IDCT Inverse Discrete Cosine Transform
IDFT Inverse Discrete Fourier Transform
IDWT Inverse Discrete Wavelet Transform
IHT Iterative Hard Thresholding
ISNR Improved-Signal-to-Noise Ratio
IST Iterative Shrinkage Thresholding
LMS Least-mean-square
LR Low Resolution
RF Radio Frequency
RIP Restricted Isometry Property
SPECT Single Photon Emission Computed Tomography
SSF Separable Surrogate Functional
SSIM Structural Similarity Index
xxiii
SNR Signal-to-Noise Ratio
SUT Sample Under-test
MAP Maximum a posteriori probability
MSE Mean Square Error
MWI Microwave Imaging
NP Non-deterministic Polynomial-time
OMP Orthogonal Matching pursuit
PCD Parallel Coordinate Descent
PET Positron Emission Tomography
pdf Probability density function
POCS Projection Onto Convex Set
PSO Particle Swarm Optimization
PSNR Peak Signal-to-Noise Ratio
TV Total Variation
ZF Zero-Filling
1
CHAPTER-1
INTRODUCTION
Medical imaging techniques such as Magnetic Resonance Imaging (MRI), x-rays
Computed Tomography (CT) and Microwave Imaging (MWI) etc. are becoming
increasingly important tools in medical decision making that offer useful information
about the medical conditions of the patients.
Unlike CT scanners which generate images by passing x-ray (ionizing) radiations
though the body, MRI is considered to be a nonionizing and non-invasive imaging
modality as it uses magnet and radio waves to produce images. MRI provides excellent
depiction of soft tissue contrast and its variant, known as Functional MRI (fMRI) can
be used to record the functional activities of the brain by measuring the associated
changes in the blood flow. However, MRI is much slower and takes longer acquisition
time than CT.
Microwave imaging is another promising modality that is based on the scattering
phenomena of microwave signals (1GHz to 30 GHz). Microwave radiations can
penetrate inside the human body and can retrieve various structural and functional
information based on the tissue water content. It is a noninvasive imaging modality and
is becoming popular because of its ability to detect breast cancer in the early stages.
The recent technological advancement in the field of biomedical imaging has resulted
in massive clinical data. It is therefore necessary to find methods and tools that can
sparsely represent the biomedical data. This will not only reduce the storage
requirements but will also be beneficial in extracting the useful information efficiently
thereby reducing the diagnosis time. There is another challenging problem related to
2
the data acquisition. The scanners used to acquire these images are generally
claustrophobic, loud, slow, uncomfortable, and may involve exposure of patient to the
harmful radiations. In order to get a high resolution image, the Nyquist rate is very high.
So, it is necessary to reduce the massive amount of the acquired data for the following
purposes:
1. To speed up the long acquisition time and increase the imaging rate to achieve
a full or nearly real-time monitoring.
2. To obtain high resolution in time, and high resolution in 3-dimensional space
of internal body structure for image-guided surgery imaging.
3. To minimize the processing/diagnosis time and less exposure of patient to the
ionizing radiation dose as in the case of CT.
4. To alleviate discomfort of the patients because of the slow acquisition, even if
there are no hazards of EM radiations as in the case of MRI, fMRI and MWI.
5. To keep the dataflow tractable for diagnosis and follow-up of human diseases.
6. To reduce to the storage size and save battery power in case of wireless
applications such as telemedicine.
The accelerated acquisition time can also reduce the motion artifacts due to respiratory
and cardiac cycles which is a common issue in dynamic cardiac imaging. Furthermore,
in biomedical imaging via neutron scattering, limited sensors may be available or
measurements may be extremely expensive. Therefore, reconstructing a high quality
image from reduced number of measurement may be highly cost-effective.
While the underlying principle and physical quantities being imaged by CT, MRI and
MWI are different but their scanners naturally acquire the encoded samples instead of
direct pixel values. So the acquisition process can be modeled by a set of linear
3
measurements of the form 𝐲 = 𝚽𝒙, making these imaging modalities a potential
application for the CS.
1.1 Compressed Sensing
Like natural images taken by digital cameras, biomedical images can also be
compressed using popular compression techniques. However, in the conventional
transform coding such as JPEG and JPGE-2000 etc., the measurements of the image
are acquired first (through Nyquist criteria) and then using a sparsifying transforms e.g.
Discrete Cosine (DCT) and Wavelet Transforms (DWT), most of the small energy
coefficients are discarded to achieve the desired level of compression. This sample-
then-compress framework of conventional compression methods introduces extra
overhead thus making these algorithms inefficient.
Compression using transformed coding is essentially a post-processing operation. In
recent years, a new data acquisition protocol known as compressive (or compressed)
sampling (or sensing) has seen enormous growth and interest in the areas of
signal/image processing, information theory, statistics and neural networks. The theory
of compressed sensing (or sampling) suggests that a sparse or compressible signal can
essentially be recovered using measurement rates below the conventional Nyquist rate
[1-3].
A sparse signal is one that has many zero and few nonzero coefficients. CS uses non-
linear recovery techniques to reconstruct the original sparse signal from small number
of non-adaptive random projections that are proportional to the sparsity level of the
signal, instead of its ambient dimensionality [4-5].
Contrary to the transform coding, CS suggests that the encoding process can be made
efficient by combining the compression step directly with the acquisition thus avoiding
4
the unnecessary information to be captured and processed. CS has enormous practical
applications in biomedical imaging such as MRI, fMRI, CT and MWI which involves
slow acquisition process. Three essential requirements for the application of
compressed sensing to biomedical images are the image sparsity, incoherent sampling
and non-linear reconstruction algorithm [6-8].
The encoder part of CS is relatively simple and consists of the non-adaptive and
incoherent linear measurements of the form 𝐲 = 𝚽𝒙. However, the CS decoder is very
challenging because of the computation cost in the (nonlinear) image reconstruction
process. Many reconstructions algorithms have been proposed for compressed sensing
in the recent past. However, most of these algorithms are general purpose and require
too many iterations making the recovery inefficient, specifically if the images have
large dimensions as in the case of biomedical applications [9-11].
The numerical algorithms used for the sparse signal recovery frequently involve finding
solution to the least squares optimization problem with 𝑙1-norm regularization. As the
𝑙1-norm penalty is not differentiable, so it rules out the possibility of using the efficient
optimization techniques that call for the derivative of the objective function. Devising
an efficient and lower cost CS recovery technique for high dimensional biomedical
images is still considered as one of the fundamental challenging task which is the main
focus of this dissertation.
1.2 Main Contributions
This work is motivated by the desire to propose a number of efficient reconstruction
methods for compressively sampled biomedical imaging modalities such as MRI,
parallel-beam CT and MWI. The proposed recovery algorithms are based on the
iterative shrinkage methods which are well suited for large dimensional signals. These
5
suits of algorithms estimate the original image from its perturbed observation
(measurements) and therefore recast the reconstruction as denoising or interference
cancellation problem.
The dissertation also presents some novel work related to the shrinkage curves used for
soft-thresholding. A flexible hyperbolic tangent function is proposed for approximating
the shrinkage curve. Hyperbolic tangent function is also used in a totally different
context to surrogate the 𝑙1-norm by a differentiable function for developing a gradient
based sparse reconstruction algorithm.
The main contributions of this work are summarized below:
Development of fast and efficient CS recovery techniques for Fourier encoded
biomedical imaging using the family of iterative-thresholding algorithms. By
incorporating data consistency constraint in the Fourier domain, computationally low-
cost sparse recovery algorithms such as separable surrogate functional (SSF) and
parallel coordinate descent (PCD) are shown to perform well in CS recovery of
biomedical images .
Sparse signal reconstruction has been carried out using heuristic techniques such as
differential evolution (DE), genetic algorithm (GA) and particle swarm optimization
(PSO). It has been shown that, by using deterministic algorithm, the convergence speed
of evolutionary algorithm can be improved.
Using the idea of GA, a modified projection onto convex set based algorithm is
developed for the recovery of compressively sampled MR images.
A novel sparse recovery algorithm is proposed to surrogate the 𝑙1-norm by a
differentiable function. The algorithm is shown to efficiently recover MRI, parallel-
beam CT and MWI from sub-Nyquist measurements.
6
A flexible hyperbolic tangent based soft-thresholding is proposed. The novel
thresholding is shown to perform well with iterated-shrinkage algorithms to recover the
biomedical images from compressive measurements. The proposed nonlinear function
can also be used in place of soft-thresholding for imaging denoising.
The proposed techniques are applied to various biomedical imaging modalities using
phantom as well as original images from the scanners. All the results related to MRI
experiments are validated using the real brain images taken from the MRI scanner at
St. Mary's Hospital, London. The microwave imaging uses a phantom image developed
at the Laboratory of Electromagnetic and Acoustic Imaging and Prognostics (LEAP),
University of Colorado Denver, USA.
1.3 Thesis Outline
The rest of the thesis is organized as follows:
Chapter-2 starts with an introduction of sparse representation and compressed sensing
with special focus on biomedical imaging modalities. It also provides a brief literature
review of the related work, followed by the quality assessment metrics used in the CS
reconstruction methods.
Chapter-3 provides the underlying physics and mathematical details to show that the
data acquisition of MRI, parallel CT and MWI scanner can be model as a Fourier
encoded measurement process, making it a potential application for the compressed
sensing. POCS based algorithm is then applied to recover MR, CT and microwave
images from sub-sampled measurements.
Chapter-4 presents the mathematical detail of SSF based CS recovery algorithm. It
incorporates the linear estimate of the residual error in POCS algorithm to further
7
improve the reconstruction quality. The viability of the technique is used to reconstruct
Fourier encoded images from a reduced dataset.
Chapter-5 provides two hybrid sparse recovery algorithms based on evolutionary
techniques. Particle swarm optimization and genetic algorithms are combined with SSF
to speed up the convergence. Based on the idea of GA, a novel CS reconstruction
method for Fourier encoded imaging is also presented. The novel approach uses
multiple initialization and randomly combines two estimates to reconstruct the final
image.
Chapter-6 is based on the differentiable surrogate approximation of 𝑙1-norm. The
proposed smooth function is used to develop an iterative algorithm for the CS
reconstruction using a simple gradient descent for the solution update. The results are
validated to recover 1-D sparse signal, original MR, CT and MW images.
Chpater-7 starts by providing an equivalence between undersampling and denoising. It
presents a novel thresholding scheme based on hyperbolic tangent function. This
flexible thresholding can be used with any of the iterative shrinkage algorithm for the
recovery of biomedical images as well as denoising.
Chapter-8 concludes the work along with the future work.
8
CHAPTER-2
SPARSE REPRESENTATION AND COMPRESSED SENSING
This chapter revisits the basic concepts of sparse representation and compressed
sensing, mainly in the context of biomedical imaging modalities such as MRI, CT and
MWI. It also provides a brief description of the background work related to the
application of compressed sensing to various biomedical imaging. Different quality
assessment parameters used in the experimental work are also discussed.
2.1 Sparse Representation
Sparse representation aims to approximate an image or signal in the most parsimonious
way by representing it as a linear combination of few elementary signals (known as the
basis or atoms) drawn from a fixed collection (called dictionary) [9-10]. The sparse
representation for a signal or image gives us the advantage of fast computation, less
storage requirement and efficient transmission. The recently developed theory of
compressed sensing (CS) has further revolutionized the field of sparse signal
approximation with the claim that a sparse signal can be recovered from far few
measurements that are needed by the conventional Nyquist theorem [1-4].
2.1.1 Sparsity: From Basis Expansion to Sparse Representation
Basis representation breaks up the (discrete and continuous time) signals as a linear
combination of fixed basis signals (atoms). In its general form, the basis expansion of
a signal 𝑥(𝑡) can be expressed as [12]:
9
𝑥(𝑡) = ∑𝛼𝛾𝜓𝛾(𝑡)
𝛾𝜖𝛤
(2.1)
where {𝜓𝛾(𝑡)}𝛾𝜖𝛤 is a fixed set of basis signals, 𝛼𝛾 is a discrete list of numbers called
the transformed coefficients and 𝛤 ∈ ℤ is a discrete index set.
The basis expansion of Eq-2.1 discretizes a signal 𝑥(𝑡) by translating it into discrete
list of numbers (𝛼𝛾) in such a way that the signal can be reconstructed. It is also referred
to as the “atomic decomposition”.
If the basis are orthonormal i.e. ⟨𝜓𝛾 , 𝜓𝛾′⟩ = 𝛿(𝛾 − 𝛾′), the coefficients can be
computed through a simple inner product:
𝛼𝛾 = ⟨𝑥(𝑡), 𝜓𝛾(𝑡)⟩ (2.2)
The well-known examples of orthonormal basis expansion are the continuous and
discrete time Fourier series, discrete Fourier transform (DFT), Sinc interpolation or
reconstruction, convolution, DCT and wavelet.
Mathematically, any real or complex-valued DT signal of length 𝑛 can be treated as
(column) vector in an 𝑛-dimensional space. Similarly, a (gray-scale) biomedical image
𝑋 can be represented by a matrix. However, without loss of generality, the biomedical
images can be brought into a vector form by stacking all its columns into a single vector.
Thus, for the discrete case (𝒙 ∈ ℝ𝑛 ), Eq-2.1 can be written as [13]:
𝒙 = ∑ 𝛼𝑖𝝍𝑖
𝑛−1
𝑖=0
= ∑⟨𝒙,𝝍𝑖⟩𝝍𝑖
𝑛−1
𝑖=0
= 𝚿𝑯𝜶 (2.3)
where 𝛼𝑖 are the expansion coefficients and 𝜶 ∈ ℂ𝑛 is the representation vector. The
operator 𝚿 = [𝝍0 𝝍𝟏 …𝝍𝑛−1] is a unitary matrix (𝑖. 𝑒. 𝚿𝚿𝐇 = 𝐈), known as the basis
matrix or dictionary or sparsifying transform. In the classical example of Fourier series,
10
𝝍𝑖 are harmonically related complex sinusoids, 𝚿 is the normalized Fourier matrix and
𝛼𝑖 are the Fourier coefficients.
Eq-2.3 represents a linear system with the same number of equations as unknowns. As
𝚿 is a square and invertible matrix (𝚿−𝟏 = 𝚿𝐇) having no null-space, the
representation of signal 𝒙 is unique [14].
An orthonormal basis representation of the signal has the following advantages:
i. The transformed coefficients 𝛼𝑖 may carry semantic information such as the
frequency contents (as in the case of Fourier series or DFT).
ii. According to the Parsevel theorem, energy of the original signal is preserved
in its transformed coefficients i.e. ‖𝒙‖22 = ‖𝜶‖2
2, where ‖∙‖2 represents the
Euclidean norm corresponding to 𝑝 = 2 of the general 𝑙𝑝 −norm defined as:
‖𝒙‖𝑝 = (∑|𝑥(𝑖)|𝑝𝑛
𝑖=1
)
1𝑝
(2.4)
iii. The expansion gives a discrete representation even for the continuous time
signals. The coefficients can therefore be processed through digital computers.
iv. The transformation may provide the energy compaction that leads to the sparse
representation of the signal or images.
A simple example of the energy compaction property can be seen in the Fourier series,
where the Fourier coefficients of a signal falls off quickly if it has more derivatives (in
time domain), leading to an implicit compression in the frequency domain.
Energy compaction property has a direct relation with the sparse representation. If a
signal or image 𝒙 can be represented as a superposition of k atoms in the 𝚿-domain,
11
then the column vector 𝜶 is known as the k-sparse representation of 𝒙 and Eq-2.3 can
be re-written as [11]:
𝒙 = ∑𝛼𝑛𝑖𝝍𝑛𝑖
𝑘
𝑖=1
where the k set of indices {𝑛𝑖}𝑖=1𝑘 correspond to the k non-zero entries of the coefficients
of the basis signals. The representation vector 𝜶 ∈ ℝ𝑛 is said to be k-sparse and will
only have 𝑘 < 𝑛 non-zero coefficients. Mathematically speaking, ‖𝜶‖0 = 𝑘 where
‖∙‖0 is the 𝑙0 pseudo-norm defined as:
‖𝒙‖0 = lim𝑝→0
‖𝒙‖𝑝𝑝 = lim
𝑝→0∑|𝑥(𝑖)|𝑝 = #{𝑖: 𝑥(𝑖) ≠ 0}
𝑛
𝑖=1
(2.5)
Where it is assumed that 00 = 0. Thus the 𝑙0 quasi norm is a counting function that
returns the number of non-zero elements (sparsity) of the signal.
Real world signals including biomedical images are seldom sparse in a transformed
domain but are instead compressible. A compressible signal has magnitude of
coefficients that decay according to a power law when sorted in a descending order. A
compressible signal can be closely approximated by a sparse signal by setting all the
small values of its coefficients equal to zero [15, 16].
2.2 Compressed Sensing for Biomedical Images
As the information contents of a sparse signal are much smaller than its bandwidth, one
can design an efficient sampling scheme by taking the number of measurements
proportional to its information contents. The theory of CS claims that for the class of
sparse or compressible signals, the required number of measurements is usually smaller
than the Nyquist limit and is instead proportional to the sparsity level of the signal. CS
12
is based on three important principles, namely sparsity, incoherent sampling or
measurements and non-linear recovery which are discussed briefly below.
2.2.1 Sparsifying transforms for biomedical images
The most popular analytical transforms used for the sparse representation of biomedical
images are discrete cosine transform, total variation and wavelet.
2.2.1.1 Discrete Cosine Transform (DCT)
DCT is an alternative to the Fourier series with the following two main differences that
makes it more attractive for certain applications:
i. The expansion coefficients and the (ortho) basis functions are real-valued.
ii. Each basis has half integer number of cycle.
The (1-D) DCT basis for ℝ𝑛 are defined as [17]:
𝜓𝑖(𝑙) =
{
√1𝑛⁄ , 𝑖 = 0
𝑙 = 0,1…𝑛 − 1 (2.6)
√2𝑛⁄ cos (
𝜋𝑖
𝑛(𝑙 +
1
2)) , 𝑖 = 1…𝑛 − 1
For 2-D biomedical images, one can extend the 1-D DCT basis of Eq-2.6 into separable
bases. Let 𝜓𝑖,𝑗(𝑙, 𝑝) be the 2-D DCT basis for ℝ𝑛 × ℝ𝑛, then:
𝜓𝑖,𝑗(𝑙, 𝑝) = 𝜓𝑖(𝑙)𝜓𝑗(𝑝), 0 ≤ 𝑖, 𝑗 ≤ 𝑛 − 1
As the bases 𝜓𝑖(𝑙) and 𝜓𝑗(𝑝) are orthogonal, the 2-D DCT basis 𝜓𝑖,𝑗(𝑙, 𝑝) are also
orthogonal. As an example, Fig-2.1 shows the 64 DCT basis functions for 𝑛 = 8. The
basis are indexed by two integers 𝑖 and 𝑗 an each basis comprises of 8 × 8 image block.
So, any arbitrary 8 × 8 image patch can be represented as a weighted sum of these 64
different DCT basis functions (image block). The transformed domain (DCT)
13
coefficients can be obtained by a simple inner product of the image patch with each of
these basis functions.
The DCT is widely used for image compression because of its ability to sparsify the
images. The basic idea is that while the energy of an 8 × 8 image block is less evenly
distributed in the pixel domain, the DCT concentrates this energy onto a relatively small
number of transform coefficients. To demonstrate the energy compaction property of
DCT, we apply it to a microwave image. Fig-2.2 shows a 6×1.5 inches near field
microwave image developed at the Electromagnetic and Acoustic Imaging and
prognostics (LEAP) lab of the University of Colorado Denver and Anschutz Medical
Campus, using a customized coaxial tip antenna. The image was acquired by using
raster scanning with excitation frequency =10 𝐺𝐻𝑧. The image reconstructed from its
10% largest DCT coefficients is also shown in the figure. It is clear that by acquiring
the most significant DCT coefficients, the original image can be reconstructed
faithfully.
Figure-2.1: DCT basis 𝜓𝑖,𝑗(𝑙, 𝑝)for 0 ≤ 𝑖, 𝑗 ≤ 7
Sparse representation with DCT is good if the image is smooth. However, it suffers
when transforming a signal that has time varying and/or singularity characteristics.
i
j
14
There is a general issue with the transforms that are based on Fourier basis. The general
Fourier basis uses complex exponentials which are periodic and time-unlimited. Once
the signal is transformed using Fourier basis, it loses time information. So, in certain
cases, it may not be the best solution to use it for the representation of time-limited
signals or if temporal resolution is required. Wavelet transform is a useful tool and
captures both frequency and time information of the signal [18].
Figure-2.2: Original and reconstructed microwave image from 10% DCT coefficients
2.2.1.2 Wavelet Transform
The wavelet consists of a single time-limited basis function (“little wave”) that satisfies
certain properties and is known as the mother wavelet. The entire library of the ortho-
basis is created from it by the operations of time-shift and scaling. The wavelet
representation of a 1-D, continuous-time finite energy signal is given by:
𝑥(𝑡) = ∑ 𝛼𝑖,𝑗𝜓𝑖,𝑗(𝑡)𝑖,𝑗 = ∑ ⟨𝒙,𝝍𝑖𝑗⟩𝑖,𝑗 𝜓𝑖,𝑗(𝑡) (2.7)
where the wavelet basis 𝜓𝑖,𝑗(𝑡) are indexed by two variables. These basis are
reconstructed from a single mother wavelet 𝜓(𝑡). Changing the choice of 𝜓(𝑡) can lead
15
to different types of wavelet transforms. For the case of discrete wavelet transform
(DWT), a typical wavelet is compressed 𝑖-times and shifted 𝑗-times to obtain the
orthonormal basis i.e.
𝜓𝑖,𝑗(𝑡) =1
√2𝑖𝜓(2−𝑖𝑡 − 𝑗), 𝑖, 𝑗 ∈ ℤ (2.8)
For a DT signal 𝒙 ∈ ℝ𝑛, the DWT is computed by passing it through a series of low
and high-pass filters. Using quadrature mirror filter (QMF) algorithm, the impulse
response of the high-pass filter 𝑔(𝑖) is calculated from that of the low-pass filter ℎ(𝑖) by
the relation:
𝑔(𝑖) = ℎ(2𝑛 − 1 − 𝑖), 0 ≤ 𝑖 ≤ 𝑛 − 1 (2.9)
After filtering, half of the samples are retained by down-sampling (decimating) the
outputs of the filters. The schematic diagram of (two-level) wavelet decomposition is
shown in Fig-2.3.
Figure-2.3: Block diagram of the QMF algorithm for DWT
The signal transformation is generally repeated for as many levels as desired by further
decomposing the low-pass version of the signal.
The inverse discrete wavelet transform (IDWT) is computed by a similar multilevel
process as shown in Fig-2.4. The impulse responses of the IDWT filters are calculated
as follows:
16
ℎ1(𝑖) = (−1)1−𝑛ℎ(1 − 𝑖)
𝑔1(𝑖) = (−1)1−𝑛𝑔(2𝑛 − 1 − 𝑖)
Figure-2.4: Block diagram for IDWT using QMF algorithm
Wavelets are extremely useful tools and are widely used for the sparse representation
of biomedical images such as CT and brain MRI [6, 7]. They have the ability to
automatically adapt to the singularities in the images. DWT is also used for
compression in JPEG-2000 [5, 19].
The 2-D wavelet transform can be easily implemented for biomedical images. This is
usually done by treating an 𝑛 × 𝑛 image once as a series of 1-D row signals and once
as a series of 1-D column signals. The 1-D DWT for all the 𝑛-rows of the image is
calculated first and the same process is repeated on the 𝑛-columns after that.
Fig-2.5 shows a real MR image of human head that was acquired at St. Mary’s Hospital
London using 1.5 Tesla GE HDxt scanner. To obtain its Daubechies-2 (db2) wavelet,
the Haar transform is applied by filtering the image with 1-D kernels horizontally and
applying the same filters vertically resulting in four output images. The process is
further continued with the low-low pass image thereby resulting in a total of 7 images.
The resulting 7 bands obtained using the Daubechies 2-taps filter (ℎ = [1 2⁄ ,
12⁄ ], 𝑔 = [1 2⁄ ,−1
2⁄ ]) with two layers of resolution are shown in Fig-2.6.
17
Figure-2.5: Original MR Image of human head
Figure 2.6: 7-bands of Daubechies 2-tap filters when applied to brain MRI.
Furthermore, Fig-2.7 demonstrations the compressibility of the MR image (Fig-2.5) in
the Wavelet domain. The figure is obtained by sorting the pixel values and wavelet
coefficients in descending order. The rapid decay of the wavelet coefficients of the MR
image shows that it can have a nearly sparse representation in the transformed (wavelet)
domain
18
Figure-2.7: Sorted coefficients of MRI in pixel and wavelet domain. Only 1000
significant values are shown for comparsion
The sparse approximation using transform coding may be either adaptive or non-
adaptive. In the non-adaptive (linear) sparse approximation, only the desired number of
transformed coefficients are retained at fixed location. However, in the adaptive (non-
linear) approximation, the biggest number of transformed coefficients are kept and the
rest are thrown away. For DCT, there is little difference between the two. However,
nonlinear wavelet approximations adapt to the singularities and outperform the linear
approximation [5].
2.2.1.3 Total Variation (TV)
TV is a commonly used to sparsely represent piece-wise smooth biomedical images. It
measures the variations in the images and is computed by summing the norms of the
discrete gradient [20]. Let 𝑋 ∈ ℂ𝑛×𝑛 be a discrete space biomedical image with 𝑥𝑖,𝑗
denoting the pixel value at 𝑖th row and 𝑗th column, then its TV is defined as:
Compressible MR image:
The rapid decay of the representation
(wavelet) coefficients.
19
𝑇𝑉(𝑋) = ∑ √|∇1(𝑖,𝑗)𝑋|2+ |∇2(𝑖,𝑗)𝑋|
2𝑖,𝑗 = ∑ ‖∇(𝑖,𝑗)𝑋‖
2𝑖,𝑗 (2.10)
where ∇1(𝑖,𝑗)𝑋 ∈ ℂ𝑛×𝑛 and ∇2(𝑖,𝑗)𝑋 ∈ ℂ𝑛×𝑛 are the horizontal and vertical difference
on image 𝑋 respectively i.e.
∇1(𝑖,𝑗)𝑋 = {𝑥𝑖+1,𝑗 − 𝑥𝑖,𝑗 , 𝑖 < 𝑛
0, 𝑖 = 𝑛
∇2(𝑖,𝑗)𝑋 = {𝑥𝑖,𝑗+1 − 𝑥𝑖,𝑗 , 𝑗 < 𝑛
0, 𝑗 = 𝑛
and ∇(𝑖,𝑗)𝑋 = [∇1(𝑖,𝑗)𝑋 ∇2(𝑖,𝑗)𝑋]𝑇 ∈ ℂ𝑛×𝑛×2 is the discrete gradient of the image. A
biomedical image is said to be k-sparse in the total-variation sense if ‖𝑇𝑉(𝑋)‖0 =
𝑘. Fig-2.8 shows a 256 × 256 standard shepp-Logan phantom image that is used
primarily in the biomedical image reconstruction such as computed tomography (CT).
This image is not directly sparse but has gradient-based sparsity.
Figure-2.8: Gradient based sparsity of Shepp-Logan image
20
Once a signal has sparse representation (direct or in a transformed domain), CS
provides a framework for its recovery from sub-Nyquist incoherent measurements
2.2.2 Incoherent Sampling (measurements)
In the theory of CS, the sensing or sampling mechanism at the encoder can be
represented as a linear transformation of a length 𝑛 signal vector 𝒙 to a length 𝑚 vector
y with 𝑚 ≪ 𝑛:
𝑦𝑖 = ∑𝜑𝑖𝑗
𝑛
𝑗=1
𝑥𝑗 =< 𝝋𝒊, 𝒙 > , 1 ≤ 𝑖 ≤ 𝑚 𝑜𝑟 𝐲 = 𝚽𝒙 (2.11)
where the transformation matrix 𝚽:ℝ𝑛 → ℝ𝑚 is known as the measurement or sensing
matrix and is formed from the sampling/sensing waveforms (or test functions)
𝝋𝟏𝑇 , 𝝋𝟐
𝑇 … 𝝋𝒎𝑇as rows. Thus, the measurement or observation vector 𝐲 is obtained
from the non-adaptive linear projections (transformation) of the original signal 𝒙.
As given in Eq-2.11, the generic notion of the sampling as inner product of the
signal/image 𝒙 against the test functions (𝝋𝒊) allow us to successfully apply CS to
biomedical images where the scanners naturally acquire the encoded samples instead
of direct pixel values. The choice of 𝝋𝒊 varies from one imaging modality to another.
For example, if the sensing waveforms (𝝋𝒊) are sinusoids at different frequencies, then
𝚽 is essentially a Fourier matrix and the measurement vector contains Fourier
coefficients as in the case of magnetic resonance imaging (MRI). Similarly, if the test
functions are delta ridges then 𝚽 is the discrete Radon transform and the measurements
are line integrals as in the case of computed tomography (CT) [5].
For a square (𝑚 = 𝑛) and invertible matrix 𝚽, the transformation is reversible which
means that the input signal can be exactly recovered from the output (𝑖. 𝑒. �̂� = 𝚽−𝟏𝐲 ).
21
However, there are some practical situations where it is preferable to use a rectangular
transformation matrix (𝑚 << 𝑛) having less number of rows than the columns. In
context of linear algebra, this leads to an under-determined system of linear equations.
In the framework of CS, this is equivalent to acquiring less number of measurements
than the ambient dimensionality of the original signal while in the application of
biomedical imaging e.g. MRI and CT, this directly corresponds to a reduction in the
acquisition (scan) time and less exposure to the radiation dose respectively [7,21].
For the under-determined system, the matrix 𝚽 has a null space which means that
different vectors can result in the same measurements after the transformation. Thus,
there are infinitely many solutions that make the recovery process ill-conditioned. In
the MRI case, it means that the final linear reconstructed image from its partial Fourier
data will have aliasing artifacts because of violation of the Nyquist criterion. Figure-
2.9 depicts the issue of aliasing during the final reconstruction of the undersampled MR
image by skipping every other line at the MR scanner (coherent sampling). The folding
over of the image is clearly seen in the final reconstructed image that is obtained using
a linear recovery technique by acquiring 50% equispaced samples of the original MRI
data of Fig-2.5.
Thus, it is not possible to reduce the acquisition time of the biomedical images by
simply reducing the data points collected by the scanners. However, the theory of CS
suggests that if the images are sparse or compressible and the measurements are
incoherent (i.e. the sensing matrix 𝚽 satisfies certain properties), then the recovery
from the partial data is possible using a non-linear reconstruction algorithm.
CS recovery works well when the representation and measurement basis pair (𝚿,𝚽)
are highly uncorrelated. In the context of CS, there are three domains and two different
transformations associated with a biomedical image, as shown in Fig-2.10.
22
Figure-2.9: Aliasing due to linear reconstruction of MRI from its partial Fourier data
Figure-2.10: Different domains and transformations in CS
Assuming the measurement and sparsifying bases to be orthonormal, their mutual
coherence is defined as the maximum value of the inner product between the vectors of
the two bases [4]:
𝜇(𝚿,𝚽) = (√𝑛 max1<𝑖,𝑗<𝑛
|< 𝝍𝑖 , 𝝋𝑗 >|) ∈ [1, √𝑛]
here both 𝚿,𝚽 ∈ ℂ𝒏×𝒏 and √𝑛 is the normalization constant. As an example, the
Fourier basis and canonical basis are maximally incoherent with 𝜇 = 1. High value of
coherence indicates that there are correlated vectors between the two bases which is
23
undesirable in CS. High value of incoherence leads to higher undersampling factors and
specifies that a signal with sparse representation in a given sparsifying domain (𝚿) must
be spread out in the domain where it is acquired. It means that the measurement basis
vectors 𝝋𝑗 needs to be completely unstructured (noise-like) [5]. This is where the
importance of random matrices comes into the theory of CS.
It has been shown that Gaussian random matrices can be used for the acquisition at CS
encoder as they are largely incoherent with any choice of basis 𝚿 [3]. However, purely
random matrices cannot be used for practical problems as they are computationally
inefficient and need a large storage. Other preferable choices of sensing matrices are
Bernoulli matrices with entries ±1, noiselets and random Fourier matrices.
The notion of coherence allows us to choose the domain where the image can be
compressively sampled. In the example of MRI, MWI and parallel-beam CT, 𝚽 is taken
as the Fourier matrix and 𝚿 can be a DCT or DWT matrix.
When 𝚽 ∈ ℂ𝒎×𝒏 and 𝚿 ∈ ℂ𝒏×𝒏, the measurements as CS encoder can be treated as a
dimensionality reduction problem. Using Eq-2.3 an Eq-2.11, the observation vector can
be written as:
𝒚 = 𝚽𝒙 = 𝚽(𝚿𝜶) = 𝐀𝜶 (2.12)
Here 𝐀 = 𝚽𝚿 ∈ ℂ𝒎×𝒏 is the new measurement matrix that maps the sparse
representation of 𝒙 into the measurements. From Eq-2.12, it is clear that the problem of
CS recovery is equivalent to the sparse representation problem. For Matrix 𝐀 to be good
for CS, it has to satisfy the restricted Isometry Property (RIP) [22,23]. RIP is a necessary
condition on matrix 𝐀 that guarantee a stable recovery for k-sparse or compressible
signals. An 𝑚 × 𝑛 measurement matrix 𝐀 is said to satisfy the RIP of order k with
isometry constant 𝛿𝑘 ∈ (0,1) if:
24
(1 − 𝛿𝑘)‖𝜶‖22 ≤ ‖𝐀𝜶‖2
2 ≤ (1 + 𝛿𝑘)‖𝜶‖22 (2.13)
for all k-sparse vector 𝜶 (i.e. ‖𝜶‖0 ≤ 𝑘).
For good measurements matrices the isometry constant 𝛿𝑘 is close to 0 which assures
that k-sparse vectors cannot be in the null space of 𝐀. This is equivalent to saying that
every subset of k columns taken from matrix 𝐀 are nearly orthogonal. It is known that
random Gaussian or Bernoulli matrices as well as matrix formed by randomly selecting
columns from Fourier matrix have small isometry constants.
The philosophy of CS is based on the fact that the number of measurements required
for the reconstruction of a signal is proportional to the compressed size of the signal,
rather than its uncompressed size. The number of compressively sampled
measurements (𝑚) directly depends on the coherence of the sensing matrix (𝐀 = 𝚽𝚿)
and sparsity level (k) [1,4].
𝑚 ≥ 𝐶. 𝜇2(𝚽,𝚿). 𝑘. log 𝑛
here 𝜇(𝚽,𝚿) is the coherence and 𝐶 is a positive constant. Empirical results show that
a k-sparse signal can be recovered from 𝑚 ≥ 4𝑘 incoherent measurements.
For biomedical images such as MRI and MWI the data is acquired by recording the
Fourier coefficients and not the pixels, DCT or wavelet coefficients. At the encoder
(scanner), the acquired measurements can be written in the form of 𝒚 = 𝐅𝒖𝒙 (i.e.
A=𝐅𝒖), where 𝐅𝒖 is the partial Fourier matrix. So, if the random undersampling of
frequency domain (Fourier) data results in incoherent artifacts in the sparsifying domain
such as DCT or wavelet, then the final image can be reconstructed using a nonlinear
recovery algorithm. In MRI, the incoherence between sparsity and sampling bases can
be improved using variable density, spiral or radial sampling. For MWI, a nonuniform
raster scan provides better incoherence with sparsifying basis [24]. However, it is
25
appropriate to consider the prior knowledge of the image class before designing a
sampling pattern.
2.2.2 Non-Linear Recovery
Given the sensing matrix 𝚽 (or 𝐀) and measurement vector 𝒚, the aim of the CS
reconstruction algorithm is to estimate the original signal 𝒙 (or 𝜶). From Eq-2.3, it clear
that the CS recovery (finding 𝛼 given 𝐴 and 𝑦) is a specific type of sparse
approximation problem. Therefore, sparse representation algorithms plays a vital role
in the recovery of compressively sampled biomedical images. Mathematically
speaking, the CS recovery algorithm has to solve an underdetermined systems of linear
equations where the number of equations (acquired data at the scanner) is less than the
number of unknowns (pixel values of the biomedical image).
One possible approach is to find the minimum norm solution by solving the following
least square minimization problem:
�̂� = argmin𝛼
1
2‖𝒚 − 𝐀𝜶‖2
2 (2.14)
However, the final reconstructed image obtained from (2.14) is severely distorted even
if the sampling is incoherent. An improved solution can be obtained by including a
proper regularization term in the objective (cost) function. So, for the general inverse
problem corresponding to Eq-2.12, the recovery algorithm can be formulated by the
following Lagrangian form:
�̂� = argmin𝜶
1
2‖𝒚 − 𝑨𝜶‖2
2 + 𝛽 ℛ(𝜶) (2.15)
where 𝛽 ∈ ℝ+is the Lagrange multiplier that adjusts a trade-off between the
representation error (data fidelity) and the regularizer term ℛ(𝜶). The function ℛ(𝜶)
26
operates element-wise on vector 𝜶. It is selected to promote sparsity and usually takes
the form [3-5]
ℛ(𝜶) = ‖𝚿𝒙‖𝑝𝑝 = ‖𝜶‖𝑝
𝑝 = ∑|𝛼(𝑖)|𝑝
𝑛
𝑖=1
(2.16)
Fig-2.11 shows the scalar function |𝛼|𝑝 that is used in the computation of norm. It is
clear that as 𝑝 approaches zero, the curve becomes an indicator function of the 𝑙0-norm.
For 𝑝 = 2, the problem presented in Eq-2.15 reduces to the classical Tikhonov
regularization with a proper closed form solution. But again, this will not work for the
CS recovery as it uses 𝑙2-norm which does not promote sparsity. Ideally, the 𝑙0-norm
(𝑝 = 0) of an image provide the exact measure of its sparsity. However, its practical
implementation is limited as the problem becomes computationally intractable (NP-
hard). Surprisingly, it has been shown that in many situations of practical interest, the
𝑙1-norm (corresponding to 𝑝 = 1) can be used in place of the 𝑙0-norm to recover the
compressively sampled biomedical images [1-4].
Figure-2.11: Behavior of scalar function |𝛼|𝑝
27
To conclude, the encoder part of CS is relatively simple and consists of the non-adaptive
linear measurements. However, the CS decoder is very challenging because of the
computation cost in the image reconstruction process. Devising an efficient and lower
cost CS recovery technique for high dimensional biomedical images is still considered
as one of the fundamental challenging task which is one of the main goals of this
dissertation.
2.3 Related Work
Before the development of modern CS theory, some of the key ideas of CS were applied
to biomedical imaging. In the earlier work, reweighted least-squares and its variants
were used for sparsity based imaging methods to solve the associated nonlinear
reconstruction problems [25]. The method of combining random sampling with
nonlinear recovery algorithm for the recovery of MRI and tomography was presented
in [26]
The introduction of CS theory immediately found important applications in the diverse
medical imaging modalities. Its first potential application was MRI because of the slow
acquisition and its pressing need to reduce the sampling rate. The subsequent work
includes brain [7], coronary [27], dynamic and cardiac [28, 29], pediatric [30] and
parallel MR imaging [31]. CS has also been successfully applied to optical imaging
modalities including diffusion optical tomography [32] and opto-acoustics tomography
[33]. Because of its ability to shorten the scan time and consequently reduce the
radiation does, CS has seen fast growth in CT over the last decade [34-35]. The
application of CS to imaging modalities like Positron Emission Tomography (PET) or
Single Photon Emission Computed Tomography (SPECT) is relatively few because
28
these modalities are mainly photon-limited instead of sampling-limited [36,37]. In a
relatively recent work, several encouraging results have been reported by using CS for
2D and 3D ultrasound images [38]. CS applications to microwave imaging is relatively
new. In [24], CS is demonstrated to improve the efficiency of microwave imaging. The
new and cutting-edged applications includes motion corrected CS [39], matrix
completion [40], tensor completion [41] and dictionary learning for biomedical imaging
[10, 42].
Because of the transformative potential for preclinical and clinical applications,
algorithm development and system designs, the applications of compressed sensing to
biomedical imaging are enormous. Almost all of these applications strongly rely on a
non-linear recovery algorithm for the image recovery from the undersampled data
which is a challenging and fascinating task. While CS has reduced the acquisition time
and the amount of raw data, unfortunately the computation time of the image recovery
has increased. In the initial work of CS, convex optimization was used to solve the
recovery problem by developing algorithms for the following constrained formulation
[12]:
�̂� = argmin𝜶
‖𝜶‖1 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 𝒚 = 𝑨𝜶 (2.17)
which is known formally as a Basis Pursuit (BP). BP is a natural convex relaxation of
the sparse decomposition problem and is computationally tractable. Other major classes
of CS reconstruction algorithms include greedy pursuit, non-convex optimization and
Bayesian framework [43]. Greedy algorithms such as Orthogonal Matching Pursuit
(OMP) [44], Stagewise OMP [45] and Compressively Sampled Matching Pursuit
(CoSaMP) [46] operate by iteratively selecting the columns of the dictionary while
keeping track of the current approximation and the residual. At each iteration, it chooses
29
the column that is most closely correlated with the residual and possibly involves least
square projections which makes the iterations computationally expensive.
Other popular recovery methods include Bregman iterations [47], Gradient Projection
for Sparse Reconstruction (GPSR) [48] and sparse reconstruction by Separable
Approximation (SpaRSA) [49]. A comparison of three CS reconstruction algorithms
(SPGL1, NESTA and RecPF) for biomedical imaging can be found in [50]. Most of the
CS reconstruction algorithms are general purpose and require too many iterations
making the recovery inefficient, specifically if the images have large dimensions as in
the case of biomedical applications [11,51].
Contrary to the previous algorithms that mostly involve expensive operations e.g.
solving least square and matrix factorization, Iterative Shrinkage-Thresholding (IST)
algorithms utilize only simple operations such as matrix-vector multiplications.
Shrinkage is an appealing sparsity inducing method and is known to be best suited for
the denoising of Gaussian noise [51,52]. The popular types of IST are Iterative Hard
Thresholding (IHT) [53,54], fast iterative shrinkage thresholding algorithm (FISTA)
[55], separable surrogate functional (SSF) and parallel coordinate descent (PCD) [56].
IST algorithms minimize the following Lagrangian formulation to get the sparse signal
approximation [57]:
�̂� = argmin𝜶∈ℂ𝒏
1
2‖𝒚 − 𝑨𝜶‖2
2 + 𝛽‖𝜶‖1 (2.18)
where 𝛽 ≥ 0 is the regularization parameter. For the orthonormal basis (𝑨 = 𝚿), it has
been shown that a closed form solution to the optimization problem of (2.18) is given
by [55-57]:
�̂�𝑖 = 𝑇𝛽(𝛼𝑖) = {𝛼𝑖 − 𝛽, 𝛼𝑖 > 𝛽 𝛼𝑖 + 𝛽, 𝛼𝑖 < −𝛽 0, 𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
(2.19)
30
Where 𝜶 = 𝚿𝒙 = {𝛼𝑖}𝒊=𝟏𝒏 are the transformed coefficients and 𝑇𝛽(∙) is the element-
wise thresholding (shrinkage) operator.
Because of the computational simplicity, near-optimal error guarantee and robustness,
the suite of algorithms presented in this dissertation are mainly based on IST methods
such as SSF, PCD etc. These methods are applied to solve the following optimization
problem for the Fourier encoded image such as MRI, parallel-beam CT and MWI.
�̂� = argmin𝒙
( 1
2‖𝒚 − 𝐅𝐮𝒙‖2
2 + 𝛽‖𝚿𝒙‖1) (2.20)
2.4 Quality assessment parameters
To assess the quality of the final reconstructed image, standard performance metrics
such as correlation, fitness value, peak signal-to-noise ratio (PSNR), structural
similarity index (SSIM) [58], improved-signal-to-noise ratio (ISNR) and artifact power
(AP) are used.
PSNR is one of the widely used quality assessment measure and is considered to be an
approximation to human perception of reconstruction quality. For a (256 × 256)
biomedical image, the standard definition of PSNR is [5]:
𝑃𝑆𝑁𝑅 = 20 ∙ log10 {255 ∙ 256
‖�̂� − 𝒙‖𝟐𝟐} [𝑑𝐵] (2.21)
ISNR is another commonly used metric for quantitative evolution of reconstruction
results and is defined as:
𝐼𝑆𝑁𝑅 = 10 ∙ log10 {‖𝒙 − |𝐅𝐮𝒙|‖2
2
‖�̂� − 𝒙‖𝟐𝟐
} [𝑑𝐵] (2.22)
The higher the value of ISNR, the better the quality of reconstructed image.
AP has been derived from “square difference error” and is calculated as:
31
𝐴𝑃 =(∑ |𝒙(𝑖) − �̂�(𝑖)|2)𝑖
∑ |𝒙(𝑖)|2𝑖 (2.23)
Reconstructed image with a smaller value of AP indicates a better quality.
The computation of SSIM between two images (𝒙 and �̂�) is based on the luminance,
contrast and structure of the images. It is computed on various windows of the
reconstructed image using the relation:
𝑆𝑆𝐼𝑀(𝒙, �̂�) =(2𝜇𝑥𝜇�̂� + 𝐶1)(2𝜎𝑥�̂� + 𝐶2)
(𝜇𝑥2 + 𝜇�̂�
2 + 𝐶1)(𝜎𝑥2 + 𝜎�̂�
2 + 𝐶2) (2.24)
where 𝐶1and 𝐶2 are constants that depends on the dynamic range of the images.
𝜇𝑥 and 𝜇�̂� represent the mean values while 𝜎𝑥2 and 𝜎�̂�
2 denote the variances of the
original and estimated image respectively. 𝜎𝑥�̂� is the covariance between of original
and recovered image. SSIM is a scalar value in the interval [-1, 1]. The maximum value
1 is achieved when both images are exactly identical.
2.5 Summary
This chapter presented an overview of the sparse representation and compressed
sensing with focus on biomedical imaging. The commonly used analytical sparsifying
transforms were reviewed as well. Finally, quality assessment metrics were also
discussed after briefly reviewing the applications of compressed sensing to various
biomedical imaging modalities.
32
CHAPTER-3
BIOMEDICAL IMAGING MODALITIES AND PROJECTION ONTO
CONVEX SETS BASED CS RECOVERY
This chapter presents the physical principles and mathematical descriptions related to
the data acquisition of three biomedical imaging modalities, namely parallel-beam CT,
MRI and MWI. The underlying physics for each of these imaging techniques is
different but they share some common properties. For example, their scanners record
the encoded (Fourier) measurements and the acquisition process can be represented by
a linear model of the form 𝒚 = 𝚽𝒙, making them a potential application of CS. It has
been shown that iterative POCS algorithm can be used to recover these images from
less number of Fourier data.
3.1 Magnetic Resonance Imaging (MRI)
MRI scanners use magnetic field and radio frequencies rather than ionizing radiations
such as x-rays used in CT. Majority of the clinical MRI machines use a superconducting
magnet having magnetic flux density, 𝐵0, of 1.5 or 3 Tesla (T). This field is about
50,000 times the earth magnetic field (0.00003 T).
The human body is composed of 70% water (𝐻2𝑂) which comprises of hydrogen and
oxygen atoms. MRI uses the magnetic properties of hydrogen atom to produce images.
The hydrogen atom has only one proton that yields a magnetic field (called magnetic
moment) due to its spinning. In the absence of an external magnetic field, the net
magnetic moment is zero because of the random orientation of the protons. However,
in the presence of an external magnetic field (𝐵0), a greater proportion of the protons
33
(hydrogen nuclei) align themselves parallel (low energy state) than antiparallel (high
energy state) to the direction of the applied field. This gives rise to a net magnetic
moment, 𝑀0, in the direction of 𝐵0 and is called longitudinal magnetization. The proton
spins around the long axis of the applied magnetic field at a frequency known as Larmor
frequency which is about 63.9 𝑀𝐻𝑧 for 1.5 T clinical scanner.
MRI scanners use three gradient coils, one in each of the cardinal directions to alter the
longitudinal magnetic field. It gives MRI the capacity to image directionally along the
x, y and z-axis. Gradient coils help to excite only a slice of interest in the imaging
volume by varying the precession in the object. The x, y and z-gradients (𝐺𝑥, 𝐺𝑦, 𝐺𝑧) run
along the horizontal, vertical and long axes to produce sagittal, coronal and axial images
respectively.
MRI scanners also use radio frequency (RF) coils that come with different designs for
each body part to produce best possible diagnostic images. The RF coils are used to
transmit a second magnetic field, 𝐵1, or RF pulse (at Larmor frequency) which results
in the disturbance of the proton alignment. This causes some low energy parallel
protons to flip to a high energy state decreasing the longitudinal magnetization and
producing a magnetization component 𝑀𝑥𝑦 that is transverse to 𝑀0. The flip angle (𝛼)
depends on the duration of the pulse and the strength of the magnetic field (𝐵1) which
is usually a few 𝜇T.
When the RF pulse is removed, the transverse magnetization 𝑀𝑥𝑦 experiences an
exponential decay with a time constant 𝑇2 while the longitudinal component 𝑀𝑧
recovers exponentially with a time constant 𝑇1. The 𝑇1 and 𝑇2 relaxation times will vary
depending on the tissue composition and structure. The changing magnetic moment of
the net magnetic vector (sum of longitudinal and transverse magnetization) results in
34
free induction decay (FID) that induces a changing voltage in the receiver coil and is
used for imaging in MRI.
As shown in Figure 3.1, three types of spatial encoding are generally used for MRI.
These are called slice selection, phase encoding and frequency encoding [59].
Magnetic resonance occurs at a particular slice (subvolume) where the transmitted RF
pulse has a frequency close to the Larmor frequency at that slice. Other slices cannot
absorb this RF energy because of different procession frequencies due to gradient fields.
The frequency of the RF pulse is determined by the magnitude of the slice selection
gradients and the slice position (𝐵0, 𝐺𝑧 , 𝑧). The thickness of the slice is controlled by
the range of frequencies (bandwidth) of the applied RF pulse. After the slice selection,
the scanner measures the transverse magnetization with two dimensional distribution
by applying additional gradients that cause spins at different spatial locations to precess
at different rate, so that their individual contributions can be measured.
Figure-3.1 Pulse sequence diagram representing various gradients for spatial
encoding
35
By applying a constant gradient 𝐺𝑦 (in the y direction) to the selected slice, the
precession frequency will change linearly in this direction. The phase encode gradient
is turned on for a brief period of time. When the gradient is turned off, the Larmor
frequency returns to a constant value. The signal at different positions will accumulate
a different phase along the y axis. This process of locating MR signal by changing the
phase of spins is called phase encoding. Spatial resolution directly depends on the
number of phase encoding levels used.
Similarly, by applying a constant gradient 𝐺𝑥 (in the x direction), the Larmor frequency
will vary linearly in that direction. With gradient on, the recorded signal will exhibit
different frequencies along the x axis. This process is known as frequency encoding and
the corresponding gradient is known as read out gradient.
Final MR image is obtained after collecting a series of frames of data involving many
RF excitations and the application of gradient fields in an orderly manner that generates
a map with unique phase-frequency pair at each point in the two spatial dimensions.
During each readout, the samples are stored in a raw matrix know as k-space. To fill a
single line in the k-space, the RF pulse is applied which is followed by phase and
frequency encodings. This process is repeated after every TR (repetition time) seconds
till the entire k-space is filled. It is worth mentioning that the acquisition time of MRI
heavily depends on the number of phase encoding steps as the frequency encoding
process is fast and the samples along the frequency encoding dimension are acquired
instantaneously.
For the conventional MRI using spatial encoding, the complex data collected by the
receiver coils at the scanner takes the form of a volume integral [7, 60]:
𝑦(𝒌) = ∫ 𝑥(𝒓) 𝑒−𝑗𝒌𝒓
𝑠𝑙𝑖𝑐𝑒
𝑑𝒓 =< 𝝋𝑘(𝒓), 𝑥(𝒓) > (3.1)
36
The vector 𝒌, which is the integral of the gradients, is interpreted as the vector of spatial
frequency coordinates and 𝝋𝑘(𝒓) = 𝑒−𝑗𝒌𝒓 are the Fourier Basis. 𝑥(𝒓) represents the
spatial domain image. So the conventional MR gives a Fourier encoded image.
As the data is recorded digitally, so all the measurements are taken at discrete space 𝒓𝑛.
Thus, the discrete version of Eq-2.1 is [7-8]:
𝒚 = [
𝑦1
𝑦2
⋮𝑦𝑚
] = [
< 𝝋1(𝒓𝑛), 𝑥(𝒓𝑛) >
< 𝝋2(𝒓𝑛), 𝑥(𝒓𝑛) >⋮
< 𝝋𝑚(𝒓𝑛), 𝑥(𝒓𝑛) >
] 𝑜𝑟 𝒚 = 𝚽𝒙 (3.2)
Where the matrix 𝚽 = 𝑭 is the Fourier matrix. So the k-space is the ‘raw data space’
with Fourier coefficients of the desired MR image. The original MR image is
reconstructed by taking the inverse Fourier transform (IFFT) of the acquired k-space
data i.e. 𝒙 = 𝚽−𝟏𝒚 = 𝑭−𝟏𝒚.
Fig-3.2 shows the k-space and the original MR image that is extensively used in our
MRI related experiments. It was acquired at St. Mary’s Hospital London using 1.5 Tesla
GE HDxt scanner with an eight-channel head coil and a gradient echo sequence with
the following specifications: TR/TE=55/10 msec, FOV =20 cm, bandwidth=31.25
KHz, slice thickness= 3 mm, flip angle= 90°, matrix size=256×256.
Figure-3.2: Fourier encoded MR image
37
Reducing the scan time of MRI by simply undersampling the k-space (as is done in CS),
results in a smaller field of view (FOV). FOV of the MR image is defined by the gap
between the phase encode lines. If the distance (∆𝑘𝑦) between phase encode lines is
doubled by uniformly undersampling the k-space, FOV will reduce to half of the
original and aliasing will occur [61]. This effect is shown in Fig-3.3.
Instead of uniform undersampling, CS uses other undersampling patterns such as radial
and variable density which take more samples at the center of the k-space than its outer
periphery [47, 62]. It is due to the fact that most of the energy of MRI is concentrated
at the center of the k-space. High frequency information about the image such as edges,
contours etc. are preserved at the outer edges of the k-space. Fig-3.4 shows the
relationship between the image space and k-space. To show how different parts of k-
space contribute in the MR image formation, two different MR images are
reconstructed by sampling the center and outer periphery of the k-space. This
information is quite useful in designing an undersampling pattern for CS acquisition.
Figure-3.3: Aliasing due to the uniform undersampling of k-space
38
Figure-3.4: Effect of undersampling the k-space on reconstructed image
3.2 Parallel beam CT
Unlike MRI which uses magnetic field and RF pulses, CT involves shooting x-rays
through the human body. CT imaging measures the attenuation coefficients 𝑓(𝑥, 𝑦) of
the object being imaged. The scanner acquires the projection data 𝑝𝜃(𝑟) by recording
the intensities of x-ray radiations after it has passed through the object at different
angles. As shown in Fig-3.5, parallel beam CT uses parallel beams of radiation with
angle 𝜃 to form projections. For a single measurement, the x-ray beam travels along a
projection line 𝑟 = 𝑥 cos(𝜃) + 𝑦 sin (𝜃) defined by the space parameters (𝑟, 𝜃). The
final image is reconstructed using the Fourier transforms of these projection functions
(measurements) at various angles.
39
Mathematically, the projection and attenuation functions are related by a line integral
[63]:
𝑝𝜃(𝑟) = ∫ 𝑓(𝑥, 𝑦)𝑑𝑙
𝐿𝜃,𝑟
(3.3)
where 𝐿𝜃,𝑟 represents a line passing through the point (𝑥 cos(𝜃) , 𝑦 sin (𝜃)) and parallel
to the t-axis. Eq-3.3 is often known as the Radon transform. So in CT, the Radom
transform is computed physically by the attenuation of the x-rays as they pass through
the tissues.
Figure-3.5: Projections in parallel beam CT
Using change of variables, 𝑥 = −𝑡 sin 𝜃 + 𝑟 cos 𝜃 and 𝑦 = 𝑡 cos 𝜃 + 𝑟 sin 𝜃, the line
integral of Eq-3.3 can be parametrized in the form [63,64]:
𝑝𝜃(𝑟) = ∫ ∫ 𝑓(𝑥, 𝑦) 𝛿(𝑥 cos 𝜃 + 𝑦 sin 𝜃 − 𝑟)𝑑𝑥𝑑𝑦 (3.4)
∞
−∞
∞
−∞
Here 𝛿(. ) is the Dirac delta or continuous-time impulse. In modern CT, the x-ray beam
is directed and the detector sweeps around the patient to collect thousands of projections
at various angles. To discretize the collected data, a square grid is superimposed on the
image with the assumption that the value of attenuation coefficient is small within each
40
cell of the grid. Thus, for a given set of discrete measurements 𝑝𝑖 (𝑖 = 1,2…𝑚)
corresponding to line integrals at different angles 𝜃𝑖 and offsets 𝑟𝑖, the discrete version
of Eq-3.4 becomes [65]:
𝑝𝑖 = ∫ ∫ 𝑓(𝑥, 𝑦) 𝛿(𝑥 cos 𝑖 + 𝑦 sin 𝜃𝑖 − 𝑟𝑖)𝑑𝑥𝑑𝑦 =< 𝒇,𝝋𝒊 > (3.5)
∞
−∞
∞
−∞
𝒑 = [
𝑝1
𝑝2
⋮𝑝𝑚
] = [
< 𝒇,𝝋𝟏 > < 𝒇,𝝋𝟐 >
⋮< 𝒇, 𝝋𝒎 >
] or 𝒑 = 𝚽𝒇 (3.6)
This equation is similar to Eq-3.2 (𝒚 = 𝚽𝒙) of MRI. However, for CT 𝚽 is constructed
from delta ridges by observing line integrals.
The parallel beam CT image can be recovered by using the Fourier slice theorem which
relates the Fourier transform of a projection to the Fourier transform of the object along
a radial line. It states that the one-dimensional Fourier transform of a parallel projection
of an image 𝑓(𝑥, 𝑦) taken at angle 𝜃 gives the value of the two-dimensional Fourier
transform 𝐹(𝜔𝑥, 𝜔𝑦) along one line subtending an angle 𝜃 with the 𝜔𝑥-axis. Therefore,
if the two-dimensional Fourier transform of the cross-sectional image 𝑓(𝑥, 𝑦) are
restricted to the radial lines (as shown in Fig-3.6), the original image can be estimated
by the inverse Fourier transform. It means that parallel beam CT can be modeled as a
Fourier encoded imaging modality where the measurement matrix 𝚽 is a radially
sampled Fourier matrix.
3.2 Microwave Imaging (MWI)
MWI not only finds applications in the medical imaging, but it has widely been used
for non-destructive testing as well. It is considered to be a preferred imaging technique
41
for early breast cancer detection as compared to X-rays and MRI because of its low
cost, safety and high contrast [66].
Figure-3.6: Radial lines used to sample the Fourier transform of an object for CT
imaging
MWI is transmission-reflection imaging modality that uses a scanning system with a
single antenna probe. In its simplest form, the scanner starts measurements on the
sample under test (SUT) and utilizes raster scanning with uniform stepsize. The antenna
collects data (reflection coefficients) as it moves from one position to another. Tissues
with anomaly have different electrical and magnetic properties that result in a different
reflection coefficient which is translated into a contrast during the final image
formation. For high resolution image, the stepsize is very small resulting in a longer
acquisition time. The theory of CS can, therefore, be used to randomly sample the SUT
and reduce the acquisition time.
Fig-3.7 shows the measurement arrangement for MWI. The transceiver (antenna probe)
is shown to be located at position (𝑥′, 𝑦′, 𝑧0) and a general point on SUT is selected at
location (𝑥, 𝑦, 0). Let 𝑓(𝑥, 𝑦) represent the reflectivity function of the SUT, which is
42
defined to be the ratio of the reflected to incident field. The backscattered microwave
reflection coefficients 𝑠(𝑥′, 𝑦′) is essentially the superposition of reflections from all
points on the illuminated area of SUT multiplied with the roundtrip phase [67,68]:
𝑠(𝑥′, 𝑦′) = ∬𝑓(𝑥, 𝑦)𝑒−𝑗2𝑘𝑅𝑑𝑥𝑑𝑦 (3.7)
Where 𝑅 = √(𝑥 − 𝑥′)2 + (𝑦 − 𝑦′)2 + 𝑧02 is the distance between transciever and
target point on SUT and 𝑘 = 𝜔𝑐⁄ is the wavenumber (𝑐 represents speed of light and
𝜔 is the angular frequency).
As in the case of CT and MRI, the data is acquired in discrete form. So if m discrete
measurements are acquired, then Eq-3.7 takes the form:
𝒔 = [
𝑠1
𝑠2
⋮𝑠𝑚
] = [
< 𝒇,𝝋𝟏 > < 𝒇,𝝋𝟐 >
⋮< 𝒇, 𝝋𝒎 >
] 𝑜𝑟 𝒔 = 𝚽𝒇 𝑤𝑖𝑡ℎ 𝝋𝒊 = 𝑒−𝑗2𝑘𝑅𝑖 (1 ≤ 𝑖 ≤ 𝑚) (3.8)
Figure-3.7: Measurement configuration for MWI
43
Fourier transform plays an important role in the MWI reconstruction. It has been shown
that Eq-3.8 can be solved for 𝑓(𝑥, 𝑦) using the relation [67,69]:
𝑓(𝑥, 𝑦) = 𝐹2𝐷−1{𝐹2𝐷{𝑠(𝑥, 𝑦)}𝑒−𝑗𝑘𝑧𝑧0} (3.9)
where 𝐹2𝐷 = 𝐹(𝜔𝑥, 𝜔𝑦) is the 2-D Fourier transform and 𝑘𝑧 = √4𝑘2 − 𝑘𝑥2 − 𝑘𝑦
2𝑧0
(where 𝑘𝑥 𝑎𝑛𝑑 𝑘𝑦 are the spatial wavenumbers).
Eq-3.9 shows that MWI shares one important property with MRI and CT. It can also be
modeled to obtain the image from the Fourier measurements.
3.3 POCS based Recovery of Fourier encoded images
Based on the idea of projection onto convex sets [15,70,71], a computationally low cost
algorithm can be obtained to recover Fourier encoded images from partial set of
measurements. The algorithm can be derived by considering solution to the scalar
version of the minimization problem presented in Eq-2.18. i.e.
𝑇𝛽(𝑧) ≔ argmin𝑥
( 1
2(𝑧 − 𝑥)2 + 𝛽|𝑥|)
1
2(𝑧 − 𝑥)2 + 𝛽|𝑥| =
{
1
2(𝑧 − 𝑥)2 − 𝛽𝑥, 𝑥 < 0
1
2(𝑧)2, 𝑥 = 0 (3.10)
1
2(𝑧 − 𝑥)2 + 𝛽𝑥, 𝑥 > 0
Where 𝑧, 𝑥 ∈ ℝ and 𝑇𝛽(𝑧) is the scalar-valued shrinkage function. Its value depends on
the minimizing variable 𝑥 and 𝑧. As |𝑥| is not differentiable at 𝑥 = 0, each term of
(3.10) is differentiated separately and solved for 𝑥. This yields the desired shrinkage
function that is given by [55,72]:
𝑇𝛽(𝑧) = max(|𝑧| − 𝛽, 0) 𝑠𝑖𝑔(𝑧)
= { 𝑧 + 𝛽, 𝑧 < −𝛽 𝑧 − 𝛽, 𝑧 > 𝛽 0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
(3.11)
44
The parameter 𝛽 is recognized as the thresholding parameter. The shrinkage function
𝑇𝛽(𝑧) maps the input value 𝑧 to a desired output value. It induces sparsity by setting
smaller values of 𝑧 (|𝑧| < 𝛽) to zero and shrinking the larger values (|𝑧| ≥ 𝛽) towards
zero.
With the assumption that the elements 𝑥𝑖 of vector 𝐱 are independent, each term of the
objective function of the form 𝑓(𝒙) =1
2‖𝒚 − 𝒙‖2
2 + 𝛽‖𝒙‖1 can be minimized
separately by solving argmin 1
2(𝑦𝑖 − 𝑥𝑖)
2 + 𝛽|𝑥𝑖| which has a closed form solution
𝑥�̂� = 𝑇𝛽(𝑦𝑖) [73].
Fig-3.8 shows how the computationally low-cost POCS based algorithm can be used to
recover the Fourier encoded image iteratively by solving (2.20).
Figure-3.8: POCS algorithm (with block diagram) for Fourier-encoded image
recovery
Initialization: 𝒙𝟎 = 𝟎, 𝒚𝟎 = 𝒚, 𝒊 = 𝟏
1) 𝒙𝒊 = 𝐅−𝟏(𝒚𝒊)
2) 𝒙𝒊 = 𝚿−𝟏{𝑻𝜷(𝚿𝒙𝒊 )}
3) 𝒚𝒊 = 𝐅(𝒙𝒊); 𝒚𝒊|𝒂𝒄𝒒 = 𝒚
4) Increment 𝒊 by 1 & repeat 1-3 until convergence
45
The algorithm moves back and forth between two main steps to estimate the missing
samples. It uses data consistency and soft-thresholding in the Fourier and sparsifying
domains respectively.
Fig-3.9 shows the experimental results of POCS based recovery when the original MR
image of Fig-2.5 is compressively undersampled in the k-space. Partial Fourier samples
are collected using variable density sampling pattern. The original image is recovered
using linear reconstruction and POCS based recovery technique. Linear recovery is
done by taking inverse FFT of the undersampled image by replacing the missing
sampling data with zeros. The resulting image is severely distorted. However, the POCS
based algorithm produces a reasonably accurate image. It uses soft-thresholding in the
wavelet domain with 𝛽=0.019. During each iteration, the missing samples are estimated
while the already acquired data remain unchanged.
The quality of the final reconstructed image vary with the selection of the thresholding
parameter 𝛽. Its actual value mainly depends on the undersampling pattern and the
sparsifying transform used. For improved reconstruction quality, proper selection of the
thresholding parameter is important. Fig-3.10 shows the decrease in mean-square-error
(MSE) between the original and the final reconstructed image for various values of 𝛽.
Similarly, Table-3.1 lists the final values of the MSE and correlations attained by the
final reconstructed MR image. These values are recorded by fixing different values of
the parameter 𝛽 and running the POCS algorithm for 10 iterations.
The algorithm of Fig-3.8 treats the CS recovery as a denoising problem. The effect of
undersampling in the Fourier domain is equivalent to adding (Gaussian-like) noise in
the image domain. So, the POCS based algorithm essentially estimates the original
image from its noisy (undersampled) version. The transformed domain coefficients of
46
the image having values less than the thresholding parameter 𝛽 are treated as noise and
are discarded during the shrinkage operation. However, values sparse coefficients
above the threshold are linearly adjusted to recover the original image.
Figure-3.9: Recover of compressively sampled MRI using POCS
To further demonstrate the effectiveness of the POCS based recovery scheme, the
algorithm is applied to reconstruct a microwave image as shown in Fig-3.11. This image
was obtained using random raster scanning. The undersampling pattern used in the
experiment is also shown in Fig-3.11. In this case, the recovery uses DCT as a
Original MR image k-space undersampling patterns
Linear reconstruction POCS
47
sparsifying transform and therefore the nonlinear shrinkage is applied to the DCT
coefficients. The algorithm runs for 10 iterations to generate the final image which is
shown in Fig-3.12. It can be seen that the simple POCS algorithm can produce
reasonably good image from the undersampled Fourier data.
Figure-3.10: Effect of thresholding parameter on the quality (MSE) of reconstructed
MR image.
Value of 𝛽 0.0100 0.0122 0.0144 0.0189 0.0233 0.0256 0.03
MSE 0.8079 0.7340 0.6903 0.6778 0.6970 0.7224 0.7522
Correlation 0.9966 0.9967 0.9968 0.9969 0.9964 0.9963 0.9959
Table-3.1 Values of MSE and correlations attained by the final MR image
1 2 3 4 5 6 7 8 9 100.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Number of iterations
Mea
n S
qu
are
Err
or
=0.01
=0.03
=0.019
48
Figure-3.11: Original MWI along with the undersampling pattern
Figure-3.12: POCS based recovery of compressively sampled MWI
Original image
inch
inch
0 1 2 3 4 5 6
0
0.5
1
1.5
Undersampling raster scaninch
inch
0 1 2 3 4 5 6
0
0.5
1
1.5
Under-sampled image
inch
inch
0 1 2 3 4 5 6
0
0.5
1
1.5
POCS based reconstruction
inch
inch
0 1 2 3 4 5 6
0
0.5
1
1.5
49
3.4 Summary
This chapter presented a common framework for the data acquisition of MRI, parallel-
beam CT and microwave imaging. It has been shown that the measurements recorded
by the scanners of these imaging modalities can be molded by a system of linear
equations. Fourier transform plays a vital role in the acquisition process. These
properties make them suitable for the application of compressed sensing. Finally, a
simple POCS based algorithm was used to recover the Fourier-encoded images from
less number of random measurements.
50
CHAPTER-4
COMPRESSIVELY SAMPLED FOURIER-ENCODED IMAGE
RECONSTRUCTION USING SEPERABLE SURROGATE FUNCTIONAL
In this chapter, a novel CS recovery technique is proposed that is based on the idea of
separable surrogate functional (SSF) method. Like POCS technique, the proposed
algorithm iterates between soft-thresholding in sparsifying domain and incorporates the
data-consistency constraint in the Fourier domain. However, the reconstruction quality
is improved by incorporating the linear estimate of the residual error. The performance
of the algorithm is validated using the real human head as well as phantom MR images
taken from the MRI scanner. The results of recovery are compared with the POCS and
Low-Resolution reconstruction methods based on the standard metrics like improved
signal-to-noise ratio, correlation and artifact power (AP). The method is also applied to
faithfully recover other Fourier-encoded biomedical images such as parallel-beam CT
and MWI.
4.1 Rapid imaging and compressed sensing
Compressed Sampling /Sensing technique facilitates simultaneous acquisition and
compression of compressible or sparse signals and has the potential to reduce the scan
time of biomedical images during the acquisition. Unlike the hardware-based
acceleration, compressed sensing is an algorithmic reduced acquisition method.
In the real world biomedical applications, higher temporal resolution and lower
radiation dose have been constantly pursued. The application of CS to Fourier encoded
images such as MRI is useful because of the fact that the data acquisition process in
51
MRI is inherently sequential and the scan time increases linearly with the number of
samples taken in the frequency domain (k-space) [74,75] .
One way to reduce the acquisition time in MRI is to decrease the repetition time (TR)
by applying stronger gradients for shorter time, that is rapid switching. However, high
gradient amplitudes and rapid switching can produce peripheral nerve stimulation
leaving little room for MR scanners to improve the imaging speed through the hardware
design implementation [76].
The MR imaging time can also be reduced by acquiring more k-space lines (phase
encoding steps) in one radio frequency (RF) excitation as in the echo-planner imaging
(EPI) [77,78]. However, this limits the amount of spatial information that can be
recorded in a single readout resulting in a lower signal to noise ratio (SNR). Another
way to reduce the MR data acquisition time is to under-sample the k-space by skipping
every other phase encoding line. This can be achieved at the cost of smaller field of
view (FOV) that contributes to the aliasing (folding over) of the original image [79,
80].
To increase SNR and improve the imaging speed, simultaneous data acquisition with
multiple receive coils in MR scanners have been used which is known as parallel
imaging (PI) [81-83]. However, the final image reconstruction needs coil sensitivities
information, which is sometimes difficult to measure with high accuracy [7, 84].
The imaging speed of MRI can also be accelerated by using non-Cartesian sampling
such as radial or spiral instead of the conventional Cartesian sampling. Although
sampling along the spiral trajectories well utilizes the gradient system hardware, the
reconstructions from non-Cartesian sampling are not generally robust to system
imperfections [6,85].
52
Accelerating MR measurements using CS exploits the sparsity of MR images during
the reconstruction from partial Fourier data. The requirement of incoherent sampling
can be achieved with the variable density k-space sampling method to reduce aliasing
artifacts during the MR image reconstruction. Variable density sampling scheme
sufficiently sample the center of the k-space that contains most of the energy of MR
images and significantly under sample the outer k-space region to reduce the scan time
[62,86].
Besides MRI, other potential applications of CS in biomedical imaging are CT and
MWI [87-91]. The acquisition time of CT is faster as compared to MRI but it involves
exposure to ionizing x-ray radiations. The commercially used CT scanners are mostly
based on the analytical reconstruction techniques such as filtered back projection
(FBP). The traditional FBP algorithm can reconstruct the final image accurately when
the projection data are densely sampled. However, if the projection data is sub-sampled
for the purpose of reducing the radiation dose, the analytic algorithms yield
reconstructed image with severe aliasing artifacts [92-93]. As CS reconstruction
techniques have a significant potential to recover the undersampled image, it can be
applied to reduce the radiation dose in CT.
In this chapter, we develop a simple CS-based iterative method that can be used to
reconstruct the Fourier encoded images (MRI, parallel-beam CT and MWI) from less
number of samples. The algorithm solves the 𝑙1-regularized least square problem to
recover the final image from compressively sampled measurements.
4.2 𝒍𝟏-regularized least square problem for Orthonormal basis
The general 𝑙1-regularized least square problem involves solution to the following
mixed 𝑙1 − 𝑙2 cost function:
53
𝑓(𝐱) =1
2‖𝐲 − 𝑨𝐱‖2
2 + 𝛽‖𝐱‖1 (4.1)
Here 𝑨 ∈ ℝ𝑚×𝑛, 𝐲 ∈ ℝ𝑚 and 𝐱 ∈ ℝ𝑛 are the sensing matrix, measurement vector and
estimated signal or image respectively. 𝛽 ∈ ℝ is the Lagrangian multiplier.
If the basis are orthonormal i.e. 𝑨 = 𝚿 (with 𝚿𝐻𝚿 = 𝐈), minimizing the objective
function of Eq-4.1 is quite simple. Utilizing the fact that unitary matrices preserves the
length after transformation. i.e. ‖𝚿𝐱‖22 = ‖𝚿𝐻𝐱‖2
2 = ‖𝐱‖22:
𝑓(𝐱) =1
2‖𝐲 − 𝚿𝐱‖2
2 + 𝛽‖𝐱‖1
= 1
2‖𝚿(𝚿𝐻𝐲 − 𝐱)‖2
2 + 𝛽‖𝐱‖1
= 1
2‖𝐱 − 𝚿𝐻𝐲‖2
2 + 𝛽‖𝐱‖1
= 1
2‖𝐱 − 𝐱𝟎‖2
2 + 𝛽‖𝐱‖1
= ∑[1
2(𝑥𝑖 − 𝑥0𝑖)
2 + 𝛽|𝑥𝑖|]
𝑛
𝑖=1
Here 𝐱𝟎 = 𝚿𝐻𝐲 is the projection of lower dimensional signal onto the original higher
dimension space. The last step shows that the overall problem reduces to 𝑛 independent
one-dimensional problems which can be solved using soft-thresholding of Eq-3.11. So
if matrix 𝑨 is unitary, the minimizer of Eq-4.1 can be obtained in two steps: (1) Find
back-projection using 𝐱𝟎 = 𝚿𝐻𝐲 and (2) applying thresholding 𝑇𝛽(∙) to individual
entries of 𝐱𝟎.
In the CS recovery problem, the sensing matrix 𝚽 is not unitary (and perhaps non-
square). However, the problem can be addressed in various ways to use the shrinkage
54
tools. One approach is to use proximal splitting methods, which is a natural extension
of POCS [55,94].
4.3 Proposed Recovery algorithm
The SSF algorithm is proximal algorithm. It works on the idea that instead of
minimizing the original cost function of Eq-4.1, a surrogate function 𝑓(𝐱) can be used
to get a closed form expression for its global minimizer. The new objective function is
obtained by adding a distance term 𝑑(𝐱, 𝐱𝟎) to the original function. Starting with an
initial vector 𝐱𝟎, and a suitable constant c, the solution to the following simpler
optimization problem (based on the proximal functions) can be easily computed:
argmin 𝐱
𝑓(𝐱) = argmin [𝐱
𝑓(𝐱) + 𝑑(𝐱, 𝐱𝟎)]
= argmin [𝐱
𝑓(𝐱) +𝑐
2‖𝐱 − 𝐱𝟎‖2
2 −1
2‖𝚽𝐱 − 𝚽𝐱𝟎‖2
2]
= argmin [𝐱
1
2‖𝐲 − 𝚽𝐱‖2
2 + 𝛽‖𝐱‖1+𝑐
2‖𝐱 − 𝐱𝟎‖2
2 −1
2‖𝚽𝐱 − 𝚽𝐱𝟎‖2
2]
= argmin𝐱
1
2‖𝐱 − 𝐳𝟎‖2
2 +𝛽
𝑐‖𝐱‖1
The last step can be obtained after a simple mathematical manipulation with 𝐳𝟎 =
1
𝑐𝚽𝑇(𝒚 − 𝚽𝐱) + 𝐱𝟎. The closed form solution for the minimizer of the surrogate
function 𝑓(𝐱) can be obtained by applying shrinkage on 𝐳𝟎 with thresholding parameter
equal to 𝛽
𝑐 i.e.
�̂� = 𝑇𝛽𝑐⁄(𝐳𝟎) = 𝑇𝛽
𝑐⁄(1
𝑐𝚽𝑇(𝒚 − 𝚽𝐱) + 𝐱𝟎) (4.2)
55
To use SSF for the recovery of compressively sampled Fourier-encoded biomedical
images, one needs to minimize the proximal function corresponding to the following
cost function (in Lagrangian form):
𝑓(𝒙) =1
2‖𝒚 − 𝐅𝒖𝒙‖𝟐
𝟐 + 𝛽‖𝚿𝒙‖1 (4.3)
The proposed SSF based method iteratively obtain the minimizer of Eq-4.3 by updating
the recovered image using the following update equation (derived from Eq-3.5):
�̂�𝒊+𝟏 = 𝚿−𝟏 {𝑇𝛽𝑐⁄(𝚿(
1
𝑐𝐅−𝟏(𝒚 − 𝐅𝐱𝒊) + 𝐱𝒊))} (4.4)
The algorithm is initialized with an initial guess that is computed by filling in the
uncollected Fourier data with zeros. This essentially corresponds to the least square
solution of the undersampled image and is severely degraded by noise artifacts due to
undersampling. Shrinkage in the Wavelet or DCT domain is applied to sparsify the
image. The proposed algorithm estimates the missing Fourier coefficients due to
undersampling, while the already acquired Fourier data remain unchanged. The detailed
steps involved in the proposed technique are shown in Fig-4.1. It also depicts the block
diagram of the SSF based recovery algorithm. The block “POCS” in Fig-4.1 refers to
the steps involved in the diagram of Fig-3.8. The SSF based recovery algorithm utilizes
POCS but it applies the thresholding step on the linear combination of the back-
projected error term and the previous estimate.
Various stopping criteria such as achieving the desired fitness value or attaining a fixed
number of iterations can be used to halt the execution of the algorithm. The fitness at
the 𝑖𝑡ℎ iteration can be computed through ‖𝐅𝐮𝒙𝑖 − 𝒚‖22.
56
Figure-4.1: The proposed SSF based recovery algorithm (with block diagram)
4.4 Recovery of MR images using the proposed algorithm
We apply the proposed SSF based algorithm to faithfully recover the original human
brain and phantom MR images from partial Fourier data. Both of these data sets are
acquired through 1.5 Tesla GE HDxt scanner with an eight-channel head coil and a
gradient echo sequence with the following parameters: TR/TE=55/10 𝑚𝑠𝑒𝑐, FOV
57
=20 cm, bandwidth=31.25 KHz, slice thickness= 3 mm flip angle=900, matrix
size=256 × 256 at St. Mary’s Hospital London. Fig-4.2 shows both the original brain
and phantom images used in the experiments.
Figure-4.2: Original phantom and brain images taken from the MRI scanner.
The images are undersampled 4-fold in the Fourier domain using variable density
sampling. These undersampled images are then reconstructed with the proposed
method using Wavelet (Daubechies 4) as the sparsifying transform. The proposed
algorithm removes the incoherent artifacts due to the undersampling and essentially
acts like a denoising algorithm.
To compare the final results, we also use zero-filling (ZF), low resolution (LR) and
POCS techniques to reconstruct the original image. ZF linearly reconstructs the
undersampled image by zero-filling the missing k-space data. For LR, the image data
is acquired with the same number of data points containing centric-ordered data around
the center of the k-space.
The proposed algorithm is initialized with ZF as an initial guess. Other parameters for
the reconstruction algorithms are set as follows: number of iterations=10 and SSF
Original phantom Image Original barin Image
58
constant 𝑐 = 100. Optimum values of 𝛽 are selected empirically for each case to have
a better performance comparison. Fig-4.3 shows the phantom image reconstruction
using LR, POCS and the proposed SSF based techniques along with the Fourier
sampling masks. To further emphasize the reconstruction accuracy of proposed
algorithm, the final reconstructed phantom image with the proposed algorithm is shown
in Fig-4.4. The difference of the recovered image with the original phantom image is
also given indicating the accuracy of the algorithm. The improvement in terms of the
SSIM at each iteration is depicted in Fig-4.5 for POCS and SSF based recovery. It is
clear from the figures that the proposed method is able to reconstruct the phantom
image faithfully as compared to LR and POCS.
Figure-4.3: Recovery (phantom MRI) using various techniques
Low resolution POCS Proposed method
Corresponding sampling patterns
59
Figure-4.4: Recovery of phantom image with proposed algorithm.
Figure-4.5: Comparison (phantom MR image) based on Structural similarity (SSIM)
The reconstruction accuracy of the proposed algorithm, based on the parameters like
ISNR, AP and correlation, has also been shown using the human brain image. Fig-4.6
shows the graphical comparison of POCS and SSF methods on the basis of ISNR, while
Fig-4.7 depicts the decrease in the reconstruction cost function at each iteration. It is
clear that the proposed SSF-based method yields better performance in the fixed
number of iterations. The final brain images reconstructed with ZF, LR, POC and SSF-
based methods are given in Fig-4.8. The reconstruction accuracy of the proposed
Sampling pattern Proposed Reconstruction Difference with original image
1 2 3 4 5 6 7 8 9 10
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
No. of iterations
Str
uctu
ral S
imilarity
Index
Proposed Mehtod
POCS
60
method can be seen from the outstanding difference of each recovered image with the
original MR image. It is clear that the image reconstructed with SSF based algorithm is
very close to the original image.
The parametric comparison of the proposed algorithm with other techniques in terms
of correlation, PSNR and artifact power, are as given in Table-4.1.
Figure-4.6: Comparison based on ISNR (brain image)
Figure-4.7: Comparison on the basis of fitness value (brain image)
1 2 3 4 5 6 7 8 9 1016
18
20
22
24
26
28
30
No. of iterations
ISN
R
Proposed Method
POCS
1 2 3 4 5 6 7 8 9 100
2000
4000
6000
8000
10000
12000
No. of iterations
Fit
ness v
alu
e:
||F
ux
- y
||2
Proposed Method
POCS
61
Figure-4.8: Difference of the recovered images with the original brain image.
PSNR Artifact power Correlation
LR 8.6419 0.0062 0.9796
POCS 16.3936 0.0047 0.9965
SSF 20.9532 0.0022 0.9982
Table-4.1: Comparison of MR brain image based on PSNR, AP and correlation
4.5 Recovery of parallel beam CT using the proposed method
The proposed SSF based algorithm can be used to recover the original CT image from
a reduced set of projections. It brings the projection data into the Fourier domain
ensuring data consistency during each iteration. To apply the algorithm, a 512 × 512
Shepp-Logan is generated, as shown in Fig-4.9(a). Each pixel value of the image
represents the attenuation coefficient. The projection data is generated by computing
62
the line integrals across the image at various angles. It is shown in Fig-4.9(b) and is
commonly known as ‘sinogram’.
Figure-4.9: CT phantom image (a) and corresponding sinogram (b)
The image is compressively sampled in the Fourier domain using radial lines. The star-
shaped sampling pattern is shown in Fig-4.10(a) that is used to acquire only 43028
Fourier measurements of the original image. Fig-4.10(b) shows the reconstructed image
that is computed by replacing the missing data with zero. This corresponds to the
minimum energy solution.
Figure-4.10: (a) undersampling pattern (b) aliased CT image
63
The proposed algorithm is compared with POCS based recovery and filtered back
projection (FBP) reconstruction. The necessary parameters of the proposed recovery
algorithms such as SSF constant 𝑐, number of iterations etc. are kept the same as those
used for MR experiments. Normally TV is used as sparsifying transform for CT
reconstruction. However, in our experiment we use Wavelet domain for thresholding.
The final reconstructed images with FBP, POCS and SSF based algorithms are shown
in Fig-4.11 having Peak-Signal-to-Noise ratios 63.38 𝑑𝑏, 74.71 𝑑𝑏 and 80.16 𝑑𝑏
respectively.
Figure-4.11: Final reconstructed CT image with various algorithms
4.6 Reconstruction of MWI using SSF based method
The CS based near field microwave imaging experiment was performed in our group
at the Electromagnetic and Acoustic Imaging and prognostics (LEAP) lab of the
University of Colorado Denver and Anschutz Medical Campus, using a customized
coaxial tip antenna with the following imaging parameters: excitation frequency =
10 𝐺𝐻𝑧, lift-off distance = 1𝑚𝑚 (much smaller than a couple of wavelengths) and step
64
size = 5 × 5 𝑠𝑡𝑒𝑝𝑠 where 1 inch=200 steps. A 6 × 1.5 inches microwave
backscattering image is acquired by using raster scanning for a sample under-test
(SUT), shown in Fig-4.12. It is worth noting that the spatial resolution using this high
resolution microwave imaging technique exceeds the Abbe’s limit (best spatial
resolution determined by half wavelength, which is 1.5 cm here) by measuring the
evanescent waves in the near field regime. Millimeter-to sub-mm scale resolution has
been obtained under current imaging setup. However, the point to point raster scanning
mode make the data acquisition (DAQ) tedious and time consuming. For CS
reconstruction the SUT is compressively undersampled using a sampling mask shown
in Fig-4.13 during DAQ phase. The probe stop at a random position while scanning the
SUT line by line. The undersampled image obtained is noisy and blurred.
Figure-4.12: fully sampled SAR image of SUT
Figure-4.13: Selective raster scanning for MWI acquisition
Original Scan
inch
inch
0 1 2 3 4 5 6
0
0.5
1
1.5
65
The proposed algorithm is applied to recover the original microwave image from the
under sampled image which is illustrated in Fig-4.14. Here DCT is used as a sparsifying
transform i.e. 𝚿 =DCT matrix. The rest of the parameters for the reconstruction
algorithm are the same as those used for MR and CT recovery problems.
Figure-4.14: Under-sampled (top) and recovered microwave image (bottom)
The aforementioned three experiments of Fourier encoded biomedical imaging
validates that the proposed SSF based recovery algorithm can be used to reconstruct the
original MR, parallel beam CT and MW images from compressively sampled data.
4.7 Summary
A novel CS reconstruction method for under-sampled Fourier-encoded images has been
presented. The recovery technique is based on the SSF algorithm with data consistency
ensured in the frequency domain. The proposed technique iteratively implements
66
shrinkage in the wavelet domain to minimize the mixed 𝑙1 − 𝑙2 reconstruction cost
function. During each iteration, the algorithm synthesizes the missing Fourier data
using back-projection and soft-thresholding, making the final image close to the
original. It has been demonstrated that the proposed technique can be used to faithfully
reconstruct the phantom as well as the original MR, parallel beam CT and microwave
images from compressively sampled data. The experimental results show that the SSF-
based recovery technique as applied to the partial Fourier data outperforms the LR,
POCS, FBP and linear recovery methods in term of PSNR, AP, correlation and ISNR.
67
CHAPTER-5
SPARSE SIGNAL RECONSTRUCTION USING HYBRID EVOLUTIONARY
ALGORITHMS
This chapter presents some novel ideas of recovering a k-sparse (1-D) signal from
compressed measurements using evolutionary techniques such as genetic algorithms
(GA) and particle swarm optimization (PSO) along with iterative shrinkage algorithms.
The proposed hybrid mechanisms with proper regularization constraints not only
accelerate the convergence of the evolutionary algorithms, but also estimate the original
sparse signal with an acceptable precision. Finally, a modified POCS algorithm for
Fourier-encoded images is presented that can recover the biomedical images from
compressively sampled incomplete Fourier data. The proposed algorithm is based on
the combined idea of POCS and evolutionary computing techniques, specifically
genetic algorithms.
5.1 Evolutionary algorithms
Unlike heuristic algorithms, deterministic algorithms are mathematically elegant, but
require a good starting point for convergence and are never user-friendly. GA and PSO
are examples of evolutionary algorithms which are simpler, but lack rigorous
mathematical foundations [95]. These algorithms are considered to be unconstrained
search techniques. So the application of GA and PSO for solving constrained
optimization is quite challenging [96-99]. Heuristic algorithms are considered suitable
for solving computationally intractable problems of the form:
68
�̂� = 𝐚𝐫𝐠𝐦𝐢𝐧𝒙
‖𝚽𝐱 − 𝐲‖𝟐𝟐 subject to ‖𝐱‖0 ≤ 𝑘 (5.1)
However to speed up the convergence, a deterministic algorithm is required for solving
the 𝑙0 minimization problem of (5.1) [100].
5.1.1 Particle Swarm Optimization (PSO)
PSO is a general-purpose heuristic optimization approach having simple structure that
uses a population (group of candidate solutions) of search agents called particles [101-
104]. The PSO based algorithm assigns randomized velocities to each particle to
explore the search space. The velocities of particles are iteratively updated, based on
their previous velocities and their distances from local and global bests. The velocity
update equation is given by [105]:
𝐯𝒊 = 𝑤 × 𝐯𝒊−𝟏 + 𝑐1𝑟1(𝐩𝑖 − 𝐱𝑖−1) + 𝑐2𝑟2(𝐩𝑔 − 𝐱𝑖−1) (5.2)
Where 𝑐1, 𝑐2 are problem dependent constants while 𝑟1 and 𝑟2 are two different
uniformly distributed random numbers in the interval (0,1). The scalar 𝑤 ∈ [0,1] is the
inertial weight. 𝐩𝑔 is the particle having best fitness in the entire population and is
referred to as global best. 𝐩𝑖 is the local best that represents the best previous position
of 𝑖𝑡ℎ particle as determined by the cost function. Varying the free parameters 𝑐1, 𝑐2 and
the inertial weight 𝑤 can greatly affect the performance of the algorithm [106].
In the conventional PSO, the position 𝐱𝑖 of the 𝑖𝑡ℎ particle is updated according to its
velocity:
𝐱𝑖 = 𝐱𝑖−1 + 𝐯𝑖 (5.3)
The algorithm generally starts with a random population of size ranging from 20 to 50
particles depending on problem. During each iteration, particles move based on their
velocity while the velocity itself is updated using the global and local best positions.
69
5.1.2 Genetic Algorithms (GA)
Genetic algorithm is another heuristic algorithm that is based on the principles of
genetics [107-110]. In GA, every individual in the population is referred to as a
chromosome that acts as a candidate solution. The chromosome comprises of elements
that are called genes. The effectiveness or cost of each chromosome is determined
through a fitness function (‖𝚽𝐱 − 𝐲‖𝟐𝟐). With the help of cross over, the genes of
different chromosomes (parents) can be combined in a variety of ways to produce the
offsprings having different fitness values. The new population is formed with the
natural selection by combining the best (in terms of fitness) parents and offsprings. In
this way the algorithm proceeds to search for the best candidate solution (chromosome).
The steps involved in using GA to solve an optimization problem are shown in Fig-5.1
[108,111].
Figure-5.1: Flow chart of generic GA
70
In spite of many achievements, one of the main problems of GA is the premature
convergence which is related to the loss of genetic diversity of the population [112-
113]. One way to avoid this problem is through mutation. However, for sparse signal
recovery, the ordinary mutation will not work, making the chromosome denser and
compromising the sparsity constraint. Constraints can be incorporated in the fitness
function (indirect constraint handling) as well as in the chromosomes (direct constraint
handling). However, indirect constraint handling does not work well for the sparse
problems [96,114].
5.2 The Proposed Hybrid Particle Swarm Optimization
The proposed technique uses a combination of stochastic (PSO) and deterministic (SSF)
algorithms to solve the sparse recovery problem for solving the constrained
optimization of (5.1). The desired sparsity level is guaranteed in all the initial particles
(population). However, the particles lose their sparsity after velocity updates, so hard
thresholding is followed by the next position update to make sure that the sparsity
constraint is properly maintained throughout.
When the fitness (mean square error) of the global best particle does not change in the
specified iterations, then SSF algorithm is accessed to update the position of the second
best particle in the population using:
𝐱𝒊 = 𝑇𝛽(𝐱𝒊−𝟏 + 𝑟𝑎𝑛𝑑 × 𝚽𝐓(𝐲 − 𝚽𝐱𝒊−𝟏)) (5.4)
where 𝑟𝑎𝑛𝑑 is a positive random number and 𝑇𝛽 is the shrinkage operator with
threshold 𝛽. The loss of sparsity is compensated by using the hard thresholding after
each SSF update.
71
Figure-5.2 describes the proposed hybrid PSO algorithm in detail for Matlab
implementation. The vector 𝐢𝐧𝐝𝐱 contains the indices of the array when sorted in the
descending order. The hard thresholding operator ⌊𝐳⌋𝑘 sets all except the k largest
elements of vector 𝐳 to zero.
Figure-5.2: The proposed hybrid PSO algorithm
Input: Dictionary 𝚽 ∈ 𝑅𝑚×𝑛 , compressed measurement 𝐲 ∈ 𝑅𝑚 , sparsity level k,
population size N, PSO parameters 𝑐1 , 𝑐2 𝑎𝑛𝑑 𝑤. SSF parameter 𝛽 for thresholding.
Output: A k-sparse vector 𝐱 ∈ 𝑅𝑛
1) Initialization: Generate N particles randomly with desired sparsity level
𝐗 = [𝐱1 , 𝐱2 , ………𝐱𝑁], 𝐱𝑖 ∈ 𝑅𝑛 and ‖𝐱𝑖‖𝟎 ≤ 𝑘 ∀ 𝑖 ≤ 1 ≤ 𝑁
𝐕 = 𝟎, Velocity matrix
2) Fitness Evaluation: Compute fitness of each particle
𝐟𝐱 = 𝑓𝑖𝑡(𝐱1 , 𝐱2 , ………𝐱𝑁)
= [𝑓1, 𝑓2, ………𝑓𝑁], 𝑓𝑖 = (𝚽𝐱𝒊 − 𝐲)𝑇(𝚽𝐱𝒊 − 𝐲)
[𝐟𝐱𝐬 𝐢𝐧𝐝𝐱 ] = 𝑠𝑜𝑟𝑡(𝐟𝐱, 𝑑𝑒𝑠𝑐𝑒𝑛𝑑)
𝐟𝐱𝐬 = [𝑓𝑥1, 𝑓𝑥2, ………𝑓𝑥𝑁 ] With 𝑓𝑥1 < 𝑓𝑥2 ……… < 𝑓𝑥𝑁
3) Local and global best (Initial): Matrix P contains local best particles
𝐏 = 𝐗(𝐢𝐧𝐝𝐱)
= [𝐩1 , 𝐩𝟐, ………𝐩𝑁] where 𝐩𝑖 ℎ𝑎𝑠 𝑓𝑖𝑡𝑛𝑒𝑠𝑓𝑥𝑖 𝐩𝑔 = 𝐩1 , initial global best
4) Velocity & Position update: Velocity and position update of each particle
according to Eq-5.2 and Eq-5.3 respectively.
V=velocity (𝐕, 𝐏, 𝐗, 𝐩𝑔 , 𝑐1 , 𝑐2 , 𝑤)
=[𝐯1, 𝐯2 , ………𝐯𝑁] 𝐗 = 𝑝𝑜𝑠𝑖𝑡𝑖𝑜𝑛(𝐗, 𝐕)
=[⌊𝐱𝟏⌋𝐾 , ⌊𝐱𝟐⌋𝐾 , ………⌊𝐱⌋𝐾], where 𝐱𝑖 = 𝐱𝑖−1 + 𝐯𝑖
5) SSF Algorithm: If 𝑓𝑥1remains the same in the specified consecutive
iterations then replace the second particle with SSF update:
𝐱𝟐 = 𝑇𝛽 (𝐱1 + 𝑟𝑎𝑛𝑑 × 𝚽𝐓(𝐲 − 𝚽𝐱1)) , ‖𝐱𝑖‖𝟎 ≤ 𝑘
6) Update local and global best: Based on fitness, local and global best
particles are updated
𝐟𝐱𝟐 = 𝑓𝑖𝑡(𝐗)
[𝐟𝐱𝟐𝐬 𝐢𝐧𝐝𝐱 ] = 𝑠𝑜𝑟𝑡(𝐟𝟐𝐱, 𝑑𝑒𝑠𝑐𝑒𝑛𝑑)
𝐟𝐱𝟐𝐬 = [𝑓𝑥21 , 𝑓𝑥22, ………𝑓𝑥2𝑁] With 𝑓𝑥21 < 𝑓𝑥22 ……… < 𝑓𝑥2𝑁
𝐗𝟐 = 𝐗(𝐢𝐧𝐝𝐱)
= [𝐱𝟐1 , 𝐱𝟐2 , ………𝐱𝟐𝑁] where 𝐱𝟐𝑖 ℎ𝑎𝑠 𝑓𝑖𝑡𝑛𝑒𝑠𝑓𝑥2𝑖
If (𝑓𝑥21 < 𝑓𝑥1) then 𝐩𝐠 = 𝐱𝟐1 and 𝑓𝑥1=𝑓𝑥21 ,( new global best)
If (𝑓𝑥2𝑖 < 𝑓𝑥𝑖 ) then 𝐩𝑖 = 𝐱𝟐𝑖 and 𝑓𝑥𝑖=𝑓𝑥2𝑖, ∀ 𝑖 ≤ 1 ≤ 𝑁 (Local bests)
Loop to step (4) until the stopping criteria (a sufficiently good fitness or
maximum number of iterations) is meet.
7) Output: Global best 𝐱 = 𝐩𝐠
72
5.3 Proposed Hybrid Genetic Algorithm
Solving (5.1) with the conventional GA is not possible as offsprings may not follow the
sparsity constraint even if it is fulfilled by the parents. In the proposed algorithm, the
direct constraint handling has been used to ensure the desired sparsity level before and
after cross-over through hard thresholding. Like hybrid PSO, SSF is used to update a
chromosome when the fitness of the best chromosome does not change in a predefined
consecutive iterations, thereby preventing the convergence issue. These modifications
allow the hybrid GA to recover the sparse signal with an acceptable level of accuracy.
Figure-5.2 lists the detailed description of the proposed hybrid genetic algorithm.
Figure-5.3: The proposed hybrid genetic algorithm for sparse signal recovery
Input: Sensing matrix 𝚽𝜖 𝑅𝑚×𝑛 , measurement vector 𝐲 ∈ 𝑅𝑚 , sparsity level k, population size N,
thresholding parameter 𝛽 for SSF.
Output: An k-sparse vector 𝐱 ∈ 𝑅𝑛
1) Population Generation: Randomly generate N chromosomes
𝐆 = [𝐠𝟏, 𝐠𝟐, ………𝐠𝐍], 𝐠𝒊 ∈ 𝑅𝑛 and ‖𝐠𝒊‖𝟎 ≤ 𝑘 ∀ 𝑖 ≤ 1 ≤ 𝑁
2) Fitness Evaluation of parents & Sorting: Evaluate the fitness of each chromosome and sort
them in the descending order (the lower the fitness, the better the chromosome)
𝐟𝐩 = 𝑓𝑖𝑡(𝐠𝟏, 𝐠𝟐, ………𝐠𝐍)
= [𝑓𝑝1, 𝑓𝑝2, ………𝑓𝑝𝑁 ], 𝑓𝑝𝑖 = (𝚽𝐠𝒊 − 𝐲)𝑇(𝚽𝐠𝒊 − 𝐲)
[𝐟𝐩𝐬 𝐢𝐧𝐝𝐱 ] = 𝑠𝑜𝑟𝑡(𝐟𝐩, 𝑑𝑒𝑠𝑐𝑒𝑛𝑑)
𝐟𝐩𝐬 = [𝑓1, 𝑓2, ………𝑓𝑁] With 𝑓1 < 𝑓2 ……… < 𝑓𝑁
𝐆𝒔 = 𝐆(𝐢𝐧𝐝𝐱)
= [𝐠𝐬𝟏, 𝐠𝐬𝟐 , ………𝐠𝐬𝐍] where 𝐠𝐬𝐢 ℎ𝑎𝑠 𝑓𝑖𝑡𝑛𝑒𝑠𝑓𝑖
3) SSF Algorithm: If 𝑓1 remains the same in the specified consecutive iterations then replace
the second particle with SSF update:
𝐠𝐬𝟐 = 𝑇𝛽 (𝐠𝐬𝟏 + 𝑟𝑎𝑛𝑑 × 𝚽𝐓(𝐲 − 𝚽𝐠𝐬𝟏)) , ‖𝐠𝒊‖𝟎 ≤ 𝑘
4) Cross over: offsprings of size half of the population are generated in random fashion:
𝐂 = 𝑥𝑜𝑣𝑒𝑟(𝐆𝒔)
= [𝐜𝟏, 𝐜𝟐 , ……… 𝐜𝑵𝟐
]
𝐜𝒋 = 𝐠𝒔𝒋 + 𝛾(𝐠𝒔𝒋 − 𝐠𝒔𝒓𝒏𝒅𝒊) 𝑘 1 ≤ 𝑗 ≤
𝑁
2 𝑎𝑛𝑑 1 ≤ 𝑖 ≤ 𝑁
5) Fitness Evaluation of children & Sorting: Same as step-2 but executed for offspring.
𝐟𝐜 = 𝑓𝑖𝑡 (𝐜𝟏, 𝐜𝟐, ………𝐜𝐍𝟐)
[𝐟𝐜𝐬 𝐢𝐧𝐝𝐱 ] = 𝑠𝑜𝑟𝑡(𝐟𝐜, 𝑑𝑒𝑠𝑐𝑒𝑛𝑑)
𝐂𝒔 = 𝐂(𝐢𝐧𝐝𝐱)
= 𝐜𝐬𝟏, 𝐜𝐬𝟐, ………𝐜𝐬𝐍
𝟐
6) New population: Generate new population using half of the best parents and all children.
𝐆 = 𝐠𝐬𝟏, 𝐠𝐬𝟐 , …𝐠𝐬𝐍
𝟐
𝐜𝐬𝟏, 𝐜𝐬𝟐, … 𝐜𝐬𝐍
𝟐
Repeat (2) - (6) until the stopping criteria meet.
7) Output: The chromosome with best fitness is the candidate solution 𝐱 = 𝐠𝐬𝟏
73
5.4 Results and Discussions
The results produced are based on the random Gaussian measurement matrix 𝚽 ∈
𝑅256×512. The rows of sensing matrix represent the measurements i.e. 𝑚 = 256 and its
columns denote the size of the sparse signal i.e. 𝑛 = 512. This matrix is generated by
taking the first 𝑚 rows of an orthonormal matrix built through Gram-Schmidt procedure
using an 𝑛 × 𝑛 matrix consisting of ±1 random entries. A one-dimensional test signal
𝐱𝟎 ∈ 𝑅512 having sparsity k=85 with random support and magnitude is used for sparse
signal reconstruction. This signal is compressively sampled to produce the
measurement vector 𝐲 = 𝚽 𝐱𝟎 ∈ 𝑅256. The population size comprising of N=20
random particles (chromosomes for GA), having proper sparsity is used in the
experiments. The thresholding parameter (𝛽) required for SSF is taken as 0.001
For PSO, the required parameters are set as follows: 𝑐1 = 𝑐2 = 2 and inertia
weight 𝑤 = 0.9. Fig-5.4 shows the effect of decrease in the cost function by keeping
all other simulation parameters same but varying the inertial weight only. It is clear that
𝑤 = 0.9 produces optimum results for the current experiment.
Figure-5.4: Performance of PSO with varying inertial weights
74
Figure-5.5 shows how the proposed algorithm accelerates the convergence of the
conventional PSO. In the initial iterations, the SSF is not used, so the decrease in the
cost function is similar for both PSO and hybrid algorithm. However, when the
algorithm starts using SSF based on the access criteria, then the convergence of the
proposed method becomes faster.
Figure-5.5 showing fast convergence achieved with the proposed method
Fig-5.6 shows the signal amplitude values of recovered signal using conventional and
the proposed hybrid PSO only. It is clear that the sparse signal reconstruction with the
proposed algorithm can recover both the support as well as the signal amplitude values
to an acceptable limit.
Similarly, integrating SSF with GA also produces similar results. Fig-5.7 shows the
comparison of hybrid GA with parallel coordinate descend (PCD) and SSF based on
the normalized MSE.
75
Table-5.1 lists the performance comparison based on other parameters like fitness value
and correlation of the final reconstructed signals through various algorithms showing
that the proposed hybrid algorithms can recover the original test signal precisely.
Figure-5.6: Signal reconstruction through conventional and hybrid PSO
Figure-5.7: Comparison of hybrid GA based on mean-square-error
76
Algorithm Used Correlation MSE Fitness
SSF 0.9087 0.2017 2.7383
PCD 0.7195 0.4836 6.8735
PSO 0.8563 0.2668 5.1068
GA 0.6102 0.6276 11.9948
Hybrid GA 1.0000 8.0656e-06 1.5115e-04
Hybrid PSO 1.0000 3.6982e-06 8.5285e-05
Table-5.1: Values of various parameters such as correlation, MSE etc. achieved by
different recovery algorithms
The direct application of hybrid GA and PSO to recover biomedical images is
computationally inefficient because of the large dimensions of the problem. An initial
population of 20 particles or chromosomes mean that these algorithms will start with
20 different biomedical images requiring lot of memory and computation power. All
these images will need to undergo velocity and position updates in case of PSO.
Similarly applying GA means that mutation and crossover will be applied to generate
offsprings. However, some of the ideas of heuristic algorithms can be incorporated in
iterative shrinkage algorithms to reconstruct biomedical images.
5.4 Proposed modified POCS algorithm for biomedical images
This section presents a novel CS reconstruction method for Fourier encoded images. It
is based on POCS algorithm but it takes two images and then randomly combines them
at each iteration to estimate the original MR image. Like POCS algorithm, the proposed
technique iterates between soft-thresholding in the sparsifying domain with data
consistency constraint in the frequency domain. Similarly, like GA it uses a random
combination of the previous (two) estimates to reduce the mean square error of the
reconstructed (offspring) image. During each iteration, the candidate solution is
77
updated based on the fitness (i.e. MSE) values. The detail description of the proposed
algorithm is shown in Fig-5.8. The algorithm can be used to recover compressively
sampled Fourier encoded images.
Figure-5.8: Modified POCS based algorithm for CS recovery
For the purpose of demonstration, the proposed modified POCS algorithm is applied to
recover the original MR image from variable density undersampling scheme. The
algorithm is initialized with two images (𝒙𝑖1 and 𝒙𝑖2) that are directly reconstructed
from the undersampled data using zero filling with and without density compensation.
During the first iteration, the two images are combined together to compute a new
estimate. In the remaining iterations, the algorithm is used to consider two best (on the
Input: 𝒚 = 𝐅𝒖𝒙 (𝑝𝑎𝑟𝑡𝑖𝑎𝑙 𝐹𝑜𝑢𝑟𝑖𝑒𝑟 𝑚𝑒𝑎𝑠𝑢𝑟𝑒𝑚𝑒𝑛𝑡𝑠 ) 𝛽 = 𝑇ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑𝑖𝑛𝑔 𝑝𝑎𝑟𝑎𝑚𝑡𝑒𝑟 𝚿 = 𝑆𝑝𝑎𝑟𝑠𝑖𝑓𝑦𝑖𝑛𝑔 𝑡𝑟𝑎𝑛𝑠𝑓𝑜𝑟𝑚 Output: 𝒙 = 𝑟𝑒𝑐𝑜𝑛𝑠𝑡𝑟𝑢𝑐𝑡𝑒𝑑 𝑖𝑚𝑎𝑔𝑒 Algorithm:
I. Initialization: 𝒙𝑖1 , 𝒙𝑖2 (Two initial solutions)
II. Fitness Evaluation & sorting:
[𝒙𝟏 , 𝒙𝟐,, 𝑓1,𝑓2] = 𝑓𝑖𝑡𝑛𝑒𝑠𝑠(𝒙𝑖1, 𝒙𝑖2 , 𝒚)
Where 𝑓𝑗 = (𝐅𝒖𝒙𝑖𝑗 − 𝒚)𝐻(𝐅𝒖𝒙𝑖𝑗 − 𝒚), 𝑗 = 1,2
With 𝒙𝟏 = {𝒙𝒊𝟏, 𝑖𝑓 𝑓1 ≤ 𝑓2
𝒙𝒊𝟐, 𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
III. Iterations: 1. Back-projection of error
𝒆𝒃 = 𝐅−𝟏(𝒚 − 𝐅𝒙𝟏) + 𝒙𝟏 2. Shrinkage & random combination
𝒆𝒔 = 𝚿−𝟏{𝑇𝑐𝛽 (𝚿{𝑐 × 𝒆𝒃 + 𝒙𝟏})}
Where c is a random number. 𝒙𝒏 = 𝑟𝑎𝑛𝑑 × (𝒆𝒔 − 𝒙𝒓) + 𝒙𝟐 Where 𝒙𝒓 is a randomly selected vector from 𝒙𝟏 𝑎𝑛𝑑 𝒙𝟐
3. Data consistency (frequency domain) 𝒚𝟏 = 𝐅(𝒙𝒏)
𝒚𝟏[𝑖] = {𝒚𝟏[𝑖], 𝑖𝑓 𝒚[𝑖] = 0
𝒚[𝑖], 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
4. Re-assignment based on fitness 𝒙𝒕 = 𝐅(𝒙𝒏), 𝑓𝑡 = (𝐅𝒖𝒙𝑡 − 𝒚)𝐻(𝐅𝒖𝒙𝑡 − 𝒚) 𝑖𝑓 𝑓𝑡 ≤ 𝑓1 , 𝒙𝟏 = 𝒙𝒕 ; 𝒙𝟐 = 𝒙𝟏; But 𝑖𝑓 𝑓𝑡 ≤ 𝑓2, 𝒙𝟐 = 𝒙𝒕;
Repeat (1-4) until stopping criteria is met.
IV. Output: 𝒙 = 𝒙𝟏
78
basis of fitness value) out of the three images to refine the estimated image. The final
reconstructed image is shown in Fig-5.9. The Daubechies D4 Wavelet is used as
sparsifying transform (𝚿) with scaling coefficients:
ℎ = {0.4830, 0.8365, 0.2241,−0.1294}.
Fig-5.10 shows the comparison of POCS and SSF based reconstruction methods with
the proposed algorithm. For the same number of iterations, the proposed algorithm
achieves better value of fitness value (i.e. MSE) indicating that it outperforms SSF and
POCS based recovery techniques.
Figure-5.9: Final image recovery with modified POCS algorithm
79
Figure-5.10: Comparison of proposed algorithm with POCS and SSF
Table-5.2 shows the summary of the various performance metrics achieved by applying
different recovery techniques to the same undersampled MR image. The algorithm can
be applied to recover other Fourier encoded images including parallel beam CT and
MWI.
PSNR Artifact power Correlation
LR 8.6419 0.0062 0.9796
POCS 16.3936 0.0047 0.9965
SSF 20.9532 0.0022 0.9982
𝒙𝒊𝟏 8.9166 0.0325 0.9818
𝒙𝒊𝟐 6.7314 0.0538 0.9753
Proposed 21.422 0.0021 0.9988
Table-5.2: Comparison of algorithms for MR image reconstruction
1 2 3 4 5 6 7 8 9 100
2000
4000
6000
8000
10000
12000
No. of iterations
Fit
ness
valu
e:
||F
ux
- y
||2
SSF
POCS
Proposed
80
5.6 Summary
Solving the NP-hard 𝑙0-minimization problem with the conventional PSO and genetic
algorithms has the issue of slow convergence. Using the sparsity-constrained version
of PSO and GA along with SSF not only accelerates the convergence but also precisely
recover the support of the original sparse signal. The proposed hybrid combination is
able to reconstruct the target (1-D) signal faithfully from less number of non-adaptive
Gaussian projections. Based on the idea of GA, a modified POCS based algorithm is
presented for the recovery of Fourier encoded imaging. MRI related experiments show
that the novel algorithm performs better than the POCS and SSF based recovery
techniques.
81
CHAPTER-6
CS RECOVERY BASED ON SMOOTH 𝒍𝟏-NORM APPROXIMATION
Sparse signal reconstruction methods are used in a wide range of applications such as
compressive sensing, denoising, signal separation and general linear inverse problems.
The numerical algorithms used for the sparse signal recovery frequently involve finding
solution to the least squares optimization problem with 𝑙1-norm regularization. As the
𝑙1-norm penalty is not differentiable, so it rules out the possibility of using the efficient
optimization techniques that call for the derivative of the objective function. This
chapter presents a hyperbolic tangent based surrogate function to closely approximate
the 𝑙1-norm regularization. Simultaneously, an iterative algorithm is developed for
sparse signal reconstruction that utilizes the gradient of the proposed smooth function.
The algorithm can be used to recover the compressively sampled (1-D) signals as well
as images from a reduced set of measurements. Various numerical and imaging
experiments are used to illustrate the performance of the promising recovery method.
It has been shown that the algorithm can be applied to reconstruct the compressively
sampled Fourier encoded images from less number of acquired data, which makes fast
imaging possible without compromising high spatial resolutions.
6.1 Problem Statement
It is a well-known fact that most of the efficient unconstrained optimization techniques
such as method of steepest descent, Gauss-Newton method and least-mean-square
(LMS) algorithm need the gradient of the cost function to obtain the optimum solution
[115-117]. However, we cannot directly use these methods to minimize the objective
82
function of problem (4.1) which involves expression of the form ∑ |𝑥𝑖|𝑛𝑖=1 . The function
𝑓(𝑥) = |𝑥| is continuous but not smooth everywhere, since it has kink at 𝑥 = 0 and is
not differentiable [118,119]. Differentiability is also an essential requirement for the
non-linear activation function used to model a neuron in a neural network [120,121]. It
is, therefore, necessary to use a smooth surrogate function to approximate |𝑥|.
6.2 Proposed hyperbolic Tangent based surrogate function
We propose a hyperbolic tangent based surrogate function that can be used to closely
approximate |𝑥|.
The hyperbolic tangent is an odd, non-convex, smooth and strictly increasing analytical
bounded function. For the general case of 𝑦 = 𝑧(𝑥) = 𝑐 𝑡𝑎𝑛ℎ (𝛾𝑥), the slope of the
function at the origin can be adjusted to any desired value with proper selection of the
parameters 𝑐 and 𝛾 i.e.:
𝑦′ =𝛾
𝑐(𝑐 − 𝑦)(𝑐 + 𝑦), with 𝑦′(0) = 𝛾𝑐 (6.1)
For the aforementioned general case, the inverse output-input relation can be expressed
as:
𝑥 = 𝑧−1(𝑦) = −1
2𝛾(𝑙𝑜 𝑔𝑒(𝑐 + 𝑥) − 𝑙𝑜 𝑔𝑒(𝑐 − 𝑥)) (6.2)
The role of hyperbolic tangent to approximate various functions is not uncommon in
the field of signal processing and neural networks [121,122]. It can be used to
approximate the signum (discontinuous) function, which is widely used for hard
thresholding:
𝑠𝑖𝑔(𝑥) = {−1, 𝑖𝑓 𝑥 < 0+1, 𝑖𝑓 𝑥 > 0
≅ tanh(𝛾𝑥) , 𝛾 ≫ 1 (6.3)
83
In neural networks, hyperbolic tangent function is used for non-linear mapping because
of its “S-shaped” curve. It performs better than other sigmoid activation functions with
respect to computability, training times etc. and therefore plays a vital role in the
backpropagation and Hopfield networks. The hyperbolic tangent is essentially
equivalent to the logistic sigmoid function in that, one can be expressed in terms of the
other, by scaling and translation transformation [121,123,124].
1
1+𝑒−𝑥 −1
2=
1
2tanh(𝑥 2⁄ ) (6.4)
Recently, the hyperbolic tangent function has been used in the application of sparse
signal processing to approximate the 𝑙0 norm [125, 126]. The proposed smooth
approximation is shown to perform better than the Gaussian and inverse tangent
function based approximations.
The facts that the hyperbolic tangent function has adjustable slope at the origin and is
bounded by the line 𝑦 = ±1, make it suitable surrogate function for the 𝑙1 norm. We
use it to approximate the non-differentiable function |𝑥| that is used to compute the 𝑙1
norm i.e.:
|𝑥| ≈ 𝑥𝑧(𝑥) = 𝑐𝑥 tanh(𝛾𝑥) [6.5]
Fig-6.1 shows the comparison of approximations for 𝑐 = 1 𝑎𝑛𝑑 𝛾 = 1,2,4. It is clear
that larger 𝛾 provides a close estimate of the function.
From Eq-6.3 and Eq-6.5, it is clear that for 𝑐 = 1 𝑎𝑛𝑑 𝛾 ≫ 1, the approximation error
is negligible for larger values of 𝑥. Fig-6.2 shows the plot of mean-square-error of the
proposed approximation, with respect to various selections of 𝛾, using three different
bounds of 𝑥. These results show that, for better approximation, the appropriate value of
the parameter 𝛾 depends on range (magnitude values) the signal 𝑥. A signal with
smaller values requires large value of 𝛾 for better approximation near the origin. It is
84
also worth mention that the function 𝑓(𝑥) = 𝑥 tanh(𝑥) is convex in the interval 𝑥 ∈
[1, −1].
Figure-6.1 Comparison of approximation for different values of 𝛾
Figure-6.2 Mean-square error of the approximation for different bounds of signal
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
x
f(x
)
y=abs(x)
y=x.tanh(x)
y=x.tanh(2x)
y=x.tanh(4x)
0 2 4 6 8 10 12 14 16 18 200
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Value of parameter
MS
E o
f ap
pro
xim
ati
on
x [-0.9,0.9]
x [-0.4,0.4]
x [-0.2,0.2]
85
6.3 Proposed Recovery Algorithm
For the scalar case of problem (4.1), the proposed approximation leads to the following
minimization problem:
𝑇𝛽(𝑦) ≔ argmin𝑥
𝑓(𝑥) = argmin {𝑥
1
2(𝑦 − 𝑥)2 + 𝛽(𝑥 tanh (𝛾𝑥))} (6.6)
where 𝑥, 𝑦, 𝑇𝛽(𝑦) ∈ ℝ. The solution 𝑇𝛽(𝑦) of problem (6.6) depends on the value of 𝑦
and the Lagrangian multiplier 𝛽.
Differentiating 𝑓(𝑥), equating to zero and solving for 𝑦 yields:
𝑦 = 𝑥 + 𝛽[𝛾𝑥 (1 − 𝑡𝑎𝑛ℎ2(𝛾𝑥)) + 𝑡𝑎𝑛ℎ(𝛾𝑥)]
≅ 𝑥 + 𝛽 𝑡𝑎𝑛 ℎ(𝛾𝑥), |𝛾𝑥| ≫ 1 [6.7]
A closed form mathematical equation for 𝑇𝛽(𝑦) cannot be obtained by solving the non-
linear Eq-6.7. However, it can be solved either graphically or using the approximation
of Eq-6.3. Solving Eq-6.7 graphically is shown in Fig-6.3, where the solution 𝑇𝛽(𝑦) in
Fig-6.3b is obtained by exchanging the axes of Fig-6.3a. Using the approximation of
Eq-6.3 leads to the following closed form solution:
𝑇𝛽(𝑦) ≅ { 𝑦 − 𝛽, 𝑦 > 𝛽𝑦 + 𝛽, 𝑦 < −𝛽0, 𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
(6.8)
The update equation for the iterative form of the algorithm is derived by using the
classical steepest descent method:
𝐱𝒌+𝟏 = 𝐱𝒌 − 𝜂𝛁𝑓(𝐱𝒌) (6.9)
where 𝜂 is a positive constant called the stepsize or learning rate. ∇𝑓(x𝑘) represents the
gradient operator of the cost function at the k-th iteration. Assuming 𝑧(𝑥𝑖) =
𝑐 𝑥𝑖 𝑡𝑎𝑛ℎ (𝛾𝑥𝑖), the cost function is
𝒇(𝐱) =𝟏
𝟐‖𝚽𝐱 − 𝐲‖𝟐
𝟐 + 𝛽 ∑ 𝑧(𝑥𝑖)𝑛𝑗=1 (6.10)
86
Figure-6.3: Graphical solution to Eq-6.7 corresponding to 𝛾 = 50 .
The gradient of the cost function is easy to compute:
𝛁𝒇(𝐱) = 𝚽𝑻(𝚽𝐱 − 𝐲) + 𝛽 ∑ 𝑧′(𝑥𝑖) (6.11)𝑛𝑗=1
where for 𝛼𝑖 = 𝑐 𝑡𝑎𝑛ℎ (𝛾𝑥𝑖),
𝑧′(𝑥𝑖) = 𝛾
𝑐𝑥𝑖 (𝑐 − 𝛼𝑖)(𝑐 + 𝛼𝑖) + 𝛼𝑖 (6.12)
The complete description of the proposed iterative recovery algorithm is shown in Fig-
6.4. The algorithm is initialized by using the minimum 𝑙2 norm solution i.e. 𝐱𝟎 =
𝚽𝑻(𝚽𝚽𝑻)𝐲.
The stepsize can be set empirically or can be adaptively computed using the well-known
Barzilai and Borwein method [127]. As the proposed algorithm is based on the standard
gradient-descent method using smooth regularization penalty, its convergence can be
followed directly in the optimization literature.
87
Figure-6.4: Proposed hyperbolic tangent based iterative algorithm for (1-D)
sparse recovery
The algorithm works well even if the sparsity of the signal is not known. However, the
prior knowledge about the sparsity can greatly reduce the number of iterations by using
the optional (step-4) of the algorithm. In order to use the proposed algorithm for the
recovery of compressively sampled biomedical images such as MRI and microwave
imaging, we also need to take the data consistency constraint into the consideration.
The algorithm can be halted either after a fixed number of iterations or when the MSE
of the estimated signal ‖𝚽�̂� − 𝐲‖𝟐𝟐 reaches an acceptable limit.
6.4 Results of Simulation with Discussions
In order to validate and evaluate the performance of the proposed algorithm, it is applied
to reconstruct 1-D sparse signal followed by CS recovery of biomedical images.
Input: Sensing matrix 𝚽 𝜖 𝑅𝑚×𝑛, measurement vector 𝐲 ∈ 𝑅𝑚,
parameters 𝛾, 𝛽, 𝜂 and sparsity level 𝑘 = ‖𝐱‖0 (optional),
Output: A k-sparse vector �̂� ∈ 𝑅𝑛
Initialization: Select 𝐱0 according to Eq.[12], index i=0
Step-1 (Gradient Computation): Find ∇𝑓(𝐱𝒊) using Eqs-6.11 & 6.12
Step-2 (Solution Update): Compute 𝐱𝑖+1 using Eq-6.9
Step-3 (Shrinkage): Estimate Solution using (6.8) i.e. �̂�𝒊 = 𝑇𝛽(𝐱𝑖+1)
Step-4 (Optional): Incorporate Sparsity, �̂�𝒊 = ⌊�̂�𝒊⌋𝑘
where the operator ⌊𝒙⌋𝑘 sets all except the k largest elements of
vector 𝐱 to zero
Step-5 (Repeat): If stopping criterion is not met, i=i+1 and go to step-1
Output: �̂� = �̂�𝒊
88
6.4.1 Sparse Signal Recovery (1-D)
In the first experiment, we generate a one dimensional sparse signal x ∈ ℝ512 having
sparsity level 𝑘 = ‖x‖0 = 85. The non-zero elements of the signal are randomly
distributed at various locations. The signal is sampled using a random Gaussian matrix
Φ ∈ ℝ256×512 to obtain linear measurements. The values of several necessary
parameters for the recovery algorithm are set as follows: 𝜂 = 0.9, 𝜆 = 0.01, 𝛾 ≥ 10.
The proposed algorithm takes the measurement vector y = Φx ∈ ℝ256 and estimates
the locations and values of the non-zero entries in the original signal.
Figure-6.5: Compressively sampled sparse signal
Fig-6.5 shows the random measurements obtained after sub-sampling the original
signal. We take only 50% random projections as compared to the dimensionality of the
original signal. Fig-6.6 shows the recovery of the signal using the minimum norm
solution and the proposed algorithm. It is clear that the proposed recovery method is
89
able to reconstruct the original signal almost perfectly from the sub-sampled signal of
Fig-6.5. The hyperbolic tangent based 𝑙1-norm approximation can also be used to
recover the signal even if the level of sparsity is not known in advance. However, the
algorithm takes more iterations to converge. Fig-6.7 shows the comparison of signal
recovery using the proposed algorithm with and without the knowledge of sparsity.
Figure-6.6: Recovery with the proposed reconstruction algorithm
0 100 200 300 400 500 600-0.5
0
0.5
Indices of nonzero entries
sig
nal
am
pli
tud
e
Values of recoverd signal through Least Square Approximation
Original
LS approx.
0 100 200 300 400 500 600-0.5
0
0.5
Indices of nonzero entries
sig
nal
am
pli
tud
e
Values of recoverd signal through proposed algorithm
Original
Recovered
90
Figure-6.7: Signal recovery with and without knowledge of sparsity
6.4.2 CS recovery of biomedical images
As biomedical images are generally considered to be sparse in the transformed domain
(such as DCT and Wavelet), so the non-linear shrinkage function of Fig-6.3 is applied
to the entries of the estimated images in the sparsifying domain. Additionally, the
algorithm incorporates the data-consistency in the Fourier domain and only estimates
the missing Fourier samples. The already acquired Fourier samples remain unaltered
during each iteration. The detailed description of the proposed algorithm for the CS
reconstruction of biomedical images is shown in Fig-6.8.
0 50 100 150 200 250 300 350 400 4500
0.1
0.2
0.3
0.4
0.5
0.6
0.7
No. of iterations
||x
-y||2
Unknown sparsity
Known sparsity
91
Figure-6.8: Proposed hyperbolic tangent based algorithm for CS recovery of Fourier
encoded imaging
6.4.1 Experimental results with 2-D imaging
In order to further validate the performance, we apply the proposed algorithm to recover
the parallel-beam CT, MR and microwave images from partial Fourier data. All the
images are compressively sampled in the frequency domain (k-space for MRI) by
skipping 20% of their frequency data points. Different sampling patterns are used to
check the robustness of the proposed recovery algorithm. The algorithm is then run for
15 iterations to recover the images. The various necessary parameters needed for the
algorithm are set as: 𝛾 = 50, 𝜆 = 0.02 and 𝜂 = 0.9.
92
Fig-6.9 shows a 512 × 512 Shepp-Logan phantom that has been undersampled using
radial sampling. The proposed algorithm is applied to reconstruct the original image.
The recovered parallel-beam CT image along with the undersampled version are shown
in Fig-6.10.
Figure-6.9: Phantom image and the under-sampling pattern used
Figure-6.10: Recovery of phantom image (parallel-beam CT) with proposed
algorithm
93
For the recovery of MR images, the original human brain MR image obtained from 1.5
Tesla GE HDxt scanner is used for the recovery. The image is undersampled using a
variable density pattern. The original MR image and the corresponding variable density
sampling pattern are shown in Fig-6.11, while Fig-6.12 depicts the undersampled and
the reconstructed images. It is clear that the proposed algorithm can accurately
reconstruct the compressively sampled MR images from the reduced data set of k-space
samples.
Standard metrics such as structural similarity index (SSIM), correlation, artifact power
(AP), improved signal-to-noise ratio etc. are used to evaluate the performance of the
proposed algorithm for MR images [22,23]. Fig-6.13 shows the improvement in SSIM
during the recovery process for both the original and phantom MR images.
Figure:-6.11: Original human head MR image and the corresponding sampling
pattern
94
Figure-6.12: Recovery of original MR image using proposed algorithm
Figure-6.13: SSIM improvement of the reconstruction images in each iteration
The CS based near field microwave imaging experiment was performed in our group
at the Electromagnetic and Acoustic Imaging and prognostics (LEAP) lab of the
University of Colorado Denver and Anschutz Medical Campus by acquiring a 6 ×
1.5 𝑖𝑛𝑐ℎ𝑒𝑠 microwave backscattering image using raster scanning for a sample under-
test (SUT), shown in Fig-6.14. The SUT is compressively undersampled using a
95
random sampling mask shown in Fig-13 during DAQ phase. Here DCT is used as a
sparsifying transform. The rest of the parameters for the reconstruction algorithm are
the same as those used for MR image recovery problem. The proposed algorithm is
applied to recover the original microwave image from the under sampled image which
is illustrated in Fig-6.15.
Finally, table-6.1 provides the numerical values of the various performance metrics
such as AP, SSIM and ISNR etc. for the final images reconstructed using the proposed
algorithm.
Figure-6.14: SAR image and corresponding under-sampling pattern used
96
Figure-6.15: Under-sampled and recovered microwave image
CT (Phantom Image) Original MRI Microwave Image
Under-sampled Recovered Under-sampled Recovered Under-sampled Recovered
Correlation 0.9630 0.9973 0.9818 0.9987 0.9572 0.9978
ISNR (db) 14.0975 26.4193 16.9548 28.5396 16.8199 30.7623
SSIM 0.4193 0.9575 0.5099 0.9640 0.8131 0.9601
AP 0.0697 0.0041 0.0325 0.0021 0.0380 0.0015
Table-6.1: Comparison based on various performance metrics
6.5 Summary
In this chapter, a hyperbolic tangent based approximation for the 𝑙1-norm is presented.
Based on the differentiable surrogate function of the 𝑙1 penalty, a novel sparse recovery
algorithm is developed. The algorithm can reconstruct a 1-D sparse signal with and
without the knowledge of the sparsity of the original signal. It has been demonstrated
that by incorporating the data-consistency constraint, the proposed technique can be
97
used to recover the compressively sampled MR and high resolution microwave images
from less number of samples. The performance of the algorithm is verified by using
different types of under-sampling patterns and sparsifying transforms. The proposed
method has the potential to speed up the imaging acquisition while retaining the
superior spatial resolutions.
98
CHAPTER-7
A FLEXIBLE SOFT THRESHOLDING FOR ITERATIVE SHRINKAGE
ALGORITHMS
There is an equivalence between undersampling and additive Gaussian noise. In fact,
CS undersampling introduces noise in the linear reconstructed images that can be
modeled by the Gaussian like probability distribution function (pdf). The denoising
algorithms generally require a thresholding step to set small coefficients to zero and
shrink the larger coefficients towards zero. This chapter introduces a novel soft-
thresholding method based on the hyperbolic tangent function. The proposed nonlinear
function has adjustable parameters and can lead to various nonlinear shrinkage curves.
It can be used with any iterative algorithm for denoising or equivalently CS recovery.
Using the proposed thresholding function in the sparsifying domain and a data
consistency step in the frequency domain, the iterative-shrinkage algorithms can be
used to effectively recover the under-sampled Fourier encoded images.
7. 1 CS Recovery and denoising
The elementary denoising algorithms aim to estimate a signal or image from its
perturbed observations. i.e.
𝒚 = 𝒙 + 𝒗 (7.1)
Here 𝒙 ∈ ℝ𝑛 is the original image vector that is measured in the presence of an additive
zero mean Gaussian noise 𝒗 having probability distribution function (pdf) given by:
𝑝𝒗(𝜃) =1
√2𝜋𝜎𝑣2𝑒
−‖𝜽‖2
2
2𝜎𝑣2
99
If the image has a sparse representation in a transform domain (𝒙 = 𝑨𝜶), then the
denoising algorithm seeks to find solution to the following optimization problem:
min𝜶
‖𝜶‖0 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 ‖𝒚 − 𝐀𝜶‖𝟐𝟐 ≤ 𝜗
The threshold 𝜗 is closely related to the noise power [128-131].
Assuming 𝐀 to be an orthonormal basis e.g. DCT or wavelet, the solution of (7.1) can
be easily obtained by simplifying the constrained term:
‖𝒚 − 𝐀𝜶‖22 = ‖𝐀(𝐀𝐇𝒚 − 𝜶)‖2
2
= ‖𝐀𝐇𝒚 − 𝜶‖22 = ‖𝒛 − 𝜶‖2
2
here 𝒛 = 𝐀𝐇𝒚 are the transformed domain coefficients. Incorporating the sparsity part
(by applying thresholding), the following simple relationship can be used to estimate
the denoised image [128,132]:
�̂� = 𝐀𝑆𝛽(𝒛) = 𝐀𝑆𝛽(𝐀𝐇𝒚) = 𝐀�̂� (7.2)
Where the scalar 𝛽 is thresholding parameter which depends on the noise power and
sparsifying transform used. 𝑆𝛽 is the scalar-valued hard-thresholding operator defined
by:
𝑆𝛽(𝑧) = {𝑧, |𝑧| ≥ 𝛽0, 𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
(7.3)
The step by step description of computing the denoised version of the image using
wavelet transform (𝐀𝐇 = 𝚿) is shown in Fig-7.1
1. Compute transform coefficients: 𝒛 = 𝚿𝒚
2. Use element-wise hard-thresholding: �̂� = 𝑆𝛽(𝒛) = ⌊𝒛⌋𝑘
3. Denoise image: �̂� = 𝚿−𝟏�̂�
Figure-7.1: Transformed based image denoising
100
To improve the results under various assumptions, different mathematical thresholding
operators have been proposed in literature [133-136]. The idea is to map the values near
the origin to zero and those away from the origin are shrunk towards zero.
The random and irregular sampling at the CS encoder results in an incoherent (noise
like) artifacts in the sparsifying domain. For the case of MRI or other Fourier encoded
biomedical imaging (𝒚 = 𝑭𝒖𝒙), the linear reconstruction (by simply replacing the
missing Fourier data with zeros and taking inverse FFT) results in artifacts that are
much like additive Gaussian noise. The actual resulting noise due to subsampling
depends on the undersampling pattern used [52].
For the variable density and radial undersampling patterns, used in our experiments, the
histogram of the noise (in image domain) is shown in Fig-7.2. The error is obtained by
the relation:
𝒆 = 𝒙 − �̂�
= 𝒙 − 𝐅𝐻𝒚 (7.4)
In order to recover the original image, the CS decoder has to estimate the noise first.
This essentially makes the CS recovery as a denoising problem. The CS reconstruction
algorithm iteratively estimates the target signal from the noisy measurements.
Fig-7.3 shows the histogram of the reconstructed error after 10 iterations of the SSF
algorithm when applied to the brain MR image. It can be seen that the width of the
Gaussian like error has decreased significantly. The variance (width) of the distribution
is the mean-square-error of the estimated vector.
101
Figure:-7.2: Histogram of error (between original Fourier encoded image and linear
reconstructed image) for (a) parallel beam CT (b) MRI
Figure-7.3: Reduction in the variance of error during CS recovery
-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.60
0.5
1
1.5
2x 10
4
Occu
ran
ce
Coefficient Values
-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.60
1000
2000
3000
Coefficient Values
Occu
ran
ce
(b)
(a)
-0.600 -0.400 -0.200 0 0.200 0.400 0.6000 0.8000
500
1000
1500
2000
2500
3000
3500
Coefficient values
Nu
mb
er
of
Occu
rren
ce
Error after 10 iterations of SSF algorithm
Error befor applying the algorighm
102
7.2 MAP estimator for denoising and proposed thresholding
Any linear transform of the zero-mean Gaussian noise results in a zero-mean Gaussian
noise in the transformed domain. So applying wavelet transform to Eq-7.1 results in:
𝒛 = 𝒘 + 𝒗 (7.5)
Where the random vectors 𝒛 = 𝚿𝐲 and 𝒘 = 𝚿𝐱 are the wavelet coefficients of the
noisy observations and noise-free image respectively. The maximum a posteriori
(MAP) estimator for the random vector 𝒘 is given by:
�̂� = argmax𝒘∈ℝ𝒏
𝑝(𝒘|𝒛)
Using Baye’s rule and ignoring 𝑃(𝒛) as using the fact that 𝑝(𝒛) does not depend on 𝒘,
the MAP estimator takes the form:
�̂� = argmax𝒘∈ℝ𝒏
𝑝(𝒛|𝒘)𝑝𝒘(𝒘) (7.6)
The problem of (7.6) can be simplified by using some simple mathematics starting
with 𝑝(𝒛|𝒘) = 𝑝𝒗(𝒛 − 𝒘):
�̂� = argmax𝒘
[𝑝𝒗(𝒛 − 𝒘)]𝑝𝒘(𝒘)
= argmax𝒘
[𝑙𝑛([𝑝𝒗(𝒛 − 𝒘) + 𝑙𝑛 𝑝𝒘(𝒘)]
= argmax𝒘
[𝑙𝑛(1
√2𝜋𝜎𝑣2𝑒
−‖𝒛−𝒘‖2
2
2𝜎𝑣2 ) + 𝑙𝑛 𝑝𝒘(𝒘)]
= argmax𝒘
[𝑙𝑛 {(1
√2𝜋𝜎𝑣2)
𝑛
𝑒−
‖𝒛−𝒘‖22
2𝜎𝑣2 } + 𝑙𝑛 𝑝𝒘(𝒘)]
= argmax𝒘
[−‖𝒛 − 𝒘‖2
2
2𝜎𝑣2
+ 𝑓(𝒘)] (7.7)
Where 𝑓(𝒘) = 𝑙𝑛 𝑝𝒘(𝒘). The MAP estimator for the wavelet coefficients of the
original image can be obtained by differentiating the argument of (7.7) w.r.t. 𝒘 and
equating the result to zero i.e.
103
(𝑧𝑖 − �̂�𝑖)
𝜎𝑣2
+ 𝑓′(�̂�𝑖) = 0, 1 ≤ 𝑖 ≤ 𝑛 (7.8)
For natural and biomedical images, the pdf of wavelet domain coefficients is generally
more peaked at the center than Gaussian. Assuming the distribution of transformed
domain coefficients be Laplacian i.e.
𝑝𝑤(𝑤𝑖) =1
√2𝜎𝑒
√2𝜎
|𝑤𝑖|
gives 𝑓′(�̂�𝑖) = −√2
𝜎 𝑠𝑖𝑔(�̂�𝑖). Solving Eq-7.8 results in
𝑧𝑖 = �̂�𝑖 +√2𝜎𝑣
2
𝜎. 𝑠𝑖𝑔(�̂�𝑖) (7.9)
Representing √2𝜎𝑣
2
𝜎= 𝛽 and solving Eq-7.9 for �̂�𝑖 to get the nonlinear shrinkage:
�̂�𝑖(𝑧) = 𝑇𝛽(𝑧) = max{|𝑧| − 𝛽, 0} . 𝑠𝑖𝑔(𝑧) (7.10)
Eq-7.10 indicates that the Laplacian distributed wavelet coefficients of the original
image can be estimated by applying the element-wise shrinkage of Eq-7.10 to wavelet
coefficients of the noisy image (or compressively undersampled image in case of CS).
It means that the hard-thresholding (step-2) of Fig-7.1 will be replaced by Eq-7.10.
In practice, the noise distribution may not be exactly Laplacian. So, the shrinkage
function may slightly vary. We present a soft-thresholding nonlinear function based on
hyperbolic tangent function that can closely approximate the operator of Eq-710.
However, the proposed thresholding scheme is versatile and provides various
adjustable parameters. The general form of the novel shrinkage operator is given by:
𝑇𝛽(𝑧) = { 0 𝑖𝑓 |𝑧| < 𝛽
𝑐𝑧{𝑡𝑎𝑛ℎ(𝛾(|𝑧| − 𝛽))} 𝑖𝑓 |𝑧| ≥ 𝛽 (7.11)
As shown in Fig-7.4, the parameter 𝑐 and 𝛾 can be tuned to get various magnitude levels
and adjustable rise of the thresholding function respectively. Fig-7.4 demonstrates
104
another fact that Eq-7.11 can also be used to approximate Eq-7.10 very closely if
𝑘 𝜖 [−1,+1] and 𝛽 ≤ 0.2 by taking 𝑐 = (1 − 𝛽) and fixing 𝛾 = (1/𝛽 − 1). This is
helpful because an image can always be normalized so that its values fall in the interval
[−1,1] either in pixel domain or in a transformed domain such as Wavelet and Fourier.
Figure-7.4: Versatility of the hyperbolic-tangent based soft-thresholding
Figure-7.5: Approximation of Eq-7.10 using the proposed soft-thresholding (Eq-7.11)
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
Original values of signal
Th
resh
old
ed
valu
es
of
sig
nal
Proposed thresholding (c=0.8)
Soft Thresholding (Eq-7.10)
Tan-Hybperbolic (=4)
Tan-Hybperbolic (=2.4)
105
The proposed thresholding function can be used with any iterative shrinkage algorithm.
To show its performance, a 1-D sparse signal 𝑥 ∈ ℝ512(k=85) is recovered by applying
the algorithm of Fig-6.4 but replacing the original shrinkage step (Step-3) by the
proposed one (Eq-7.11). Similarly the same signal is recovered with SSF using the soft-
thresholding of Eq-7.10. The shrinkage is performed on the estimated signal directly as
it is assumed to be sparse in the time-domain. The promising results of the proposed
algorithm are clear from Fig-6.4.
Figure-7.6: performance of the proposed thresholding function
The proposed nonlinear thresholding can be used with any iterative shrinkage algorithm
to recover the compressively sampled Fourier encoded images.
0 10 20 30 40 50 60 70 80 90 1000
5
10
15
Number of iterations
Mean
Sq
uare
Err
or
Algorithm of Fig-6.4 with proposed thresholding
Algorithm of Fig-6.4 with soft thresholding
SSF with soft thresholding
106
7.3 CS recovery of biomedical imaging (Denoising approach)
In general, the iterative shrinkage algorithms estimate the denoised image by
thresholding the wavelet coefficients i.e.
�̂�𝑑𝑒𝑛𝑜𝑖𝑠𝑒 = 𝚿−𝟏{𝑇𝛽(𝚿�̂�)} (7.11)
The error between the denoised version and the original image is estimated and
incorporated in the update equation. Assuming the acquisition transform to be
orthonormal, the linear estimate of the error can be computed as:
�̂� = 𝒙 − �̂�𝑑𝑒𝑛𝑜𝑖𝑠𝑒
= 𝐀𝐇(𝒚 − 𝐀 �̂�𝑑𝑒𝑛𝑜𝑖𝑠𝑒) (7.12)
where 𝒚 = 𝐀𝒙 are the encoded measurements of the original image vector. The solution
can be refined by using the update equation that considers the denoised image and the
error estimate together. In general, the update equation takes the form:
�̂� = �̂�𝑑𝑒𝑛𝑜𝑖𝑠𝑒 + 𝐖�̂� (7.13)
where 𝐖 is the diagonal weight matrix. For SSF algorithm, 𝐖 = 𝐈. For PCD, the weight
matrix is computed offline using 𝐖 = (𝐀𝐻𝐀)−𝟏 [56,57].
Eq-7.13 represents one iteration of the iterative shrinkage algorithms which comprises
of a denoising step combined with the estimate of error.
For the Fourier encoded biomedical images, 𝐀 = 𝐅. So at the ith iteration, Eq-7.13 takes
the form:
𝒙𝒊 = 𝚿−𝟏{𝑇𝛽 , (𝚿{𝐖( 𝐅−𝟏(𝒚𝒊−𝟏 − 𝐅𝒙𝒊−𝟏) + 𝒙𝒊−𝟏)})} (7.14)
The thresholding parameter is usually scaled with different values that depends on the
algorithm. For Fourier encoded biomedical imaging, the data consistency condition is
also incorporated in the frequency domain at each iteration which means that the
107
originally acquired Fourier samples remain intact. The general description of CS
recovery is shown in Fig-7.7
Figure-7.7: A general denoising based CS recovery algorithm
7.4 Experimental results
For the purpose of demonstration, the algorithm of Fig-7.7 is applied to recover the
original MR image using variable density undersampling scheme. The weight matrix is
directly computed from the undersampled image by taking the inverse of the matrix
108
corresponding to zero-filled reconstruction and then normalizing its value to the range
[0, 1]. For the shrinkage, the proposed function of Eq-7.11 is applied. Fig-7.8 shows
the distribution (histogram) of the original MR image in the pixel (image) domain while
Fig-7.9 displays the histogram of linear reconstructed image. The algorithm runs for 10
iterations to reconstruct the original image. Each iteration comprises of linear estimate
of the error, denoising and data consistency. The histogram of the recovered image is
shown in figure in Fig-7.10 which is very close to that of Fig-7.8.
Figure-7.8: Distribution of pixel values of the original MR image
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.5
1
1.5
2
2.5
3x 10
4
normalized coefficients (pixel values)
Fre
qu
en
cy o
f o
ccu
rra
nce
109
Figure-7.9: Distribution of pixel values of the original undersampled image
Figure-7.10: Distribution of pixel values of the reconstructed image
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.90
2000
4000
6000
8000
10000
12000
normalized pixel values
Fre
qu
en
cy o
f o
ccu
rra
nce
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.5
1
1.5
2
2.5
3x 10
4
normalized coefficients (pixel values)
Fre
qu
en
cy o
f o
ccu
rra
nce
110
In the second experiment, the MR phantom image taken at St. Marry hospital London
is compressively sampled and recovered through the algorithm of Fig-6.4 using (a) soft-
thresholding of Eq-7.10 and (b) proposed hyperbolic soft-thresholding of Eq-7.11. The
sampling pattern, number of iterations and other common parameters are kept the same
to have better performance comparison. The correlation based results are shown in Fig-
7.11. The original as well as recovered images are shown in Fig-7.12.
The versatility of the proposed hyperbolic tangent based soft thresholding has been
validated using the SSF algorithm with both non-linear thresholding functions. Fig-7.13
depicts the improvement in SSIM achieved using the proposed nonlinear function of
Eq-7.11. As shown in Fig-7.14, the image reconstructed with the proposed nonlinear
shrinkage has reasonably good quality as compared to the one recovered using soft-
thresholding of Eq-7.10.
Figure-7.11: Improvement in correlation using soft (Eq-7.10) and proposed (Eq-7.11)
thresholding function for MR recovery
1 2 3 4 5 6 7 8 9 100.992
0.993
0.994
0.995
0.996
0.997
0.998
0.999
1
Number of iterations
Co
rrela
tio
n
Soft-thresholding
Proposed thresholding
111
Figure-7.12: Reconstruction of MR phantom image using thresholding of Eq-7.11
Figure-7.13: Comparison based on SSIM using two different thresholding
Original image Reconstruction using proposed thresholding
1 2 3 4 5 6 7 8 9 100.993
0.994
0.995
0.996
0.997
0.998
0.999
1
Number of iterations
SS
IM
Proposed Thresholding
Soft Thresholding
112
Figure-7.14: SSF based Original MR Image reconstruction using soft-thresholding
(Eq-7.10) and the proposed (Eq-7.11)
7.5 Summary
This chapter explores the connection between CS recovery and denoising.
Undersampling in CS essentially leads to the addition of Gaussian noise in the original
image. CS recovery using iterative thresholding algorithms effectively comprise of
denoising step (computing the linear estimate of the error and applying thresholding)
followed by data consistency in frequency domain. Nonlinear shrinkage curves plays a
vital role in the reconstruction process. A novel shrinkage function (based on hyperbolic
tangent) with adjustable parameters is also presented that performs well in the CS
recovery as compared to the soft-thresholding.
Recovery with Soft thresholding Recovery with proposed thresholding
113
CHAPTER-8
CONCLUSION
8.1 Summary of thesis
Compressed sensing is a novel acquisition protocol that operates at low sampling rates
and has the ability to reduce the acquisition time; a feature that is highly desirable in
biomedical applications such as magnetic resonance imaging (MRI), microwave
imaging (MWI) and computed tomography (CT) where conventional acquisition
techniques are time consuming or require longer exposure to hazardous radiations. The
application of CS to biomedical imaging modalities has shorten the acquisition time
and the amount of raw data, unfortunately the computation time of the image recovery
has increased. In the initial work of CS, convex optimization based methods were
applied to solve the non-linear CS recovery problem. However, these general purpose
reconstruction algorithms are often slow and inefficient requiring too many
computations especially for high-dimensional biomedical images. The main focus of
this dissertation is to propose a novel suite of algorithms that can efficiently reconstruct
Fourier-encoded biomedical images from sub-sample measurements. The proposed
algorithms are used to estimate the missing Fourier samples by using data-consistency
in the Fourier domain and shrinkage in the sparsity domain. The techniques presented
are mainly derived from the iterative-shrinkage algorithms that are widely used for
image denoising.
It has been shown that a simple projection-onto-convex set algorithm can be
applied to MR and microwave imaging modalities to reconstruct the final image
114
from a reduced set of Fourier measurements. POCS estimates the missing
samples by simply applying soft-thresholding in the sparsifying domain.
A novel CS recovery algorithm based on separable surrogate functional (SSF)
method is presented next. The update equation of SSF incorporates the linear
estimate of residual error before applying non-linear shrinkage. Experimental
results validate hat the SSF based reconstruction yields a better image quality
as compared to POCS.
Evolutionary techniques such as genetic algorithms (GA) and particle swarm
optimization (PSO) are used in combination with SSF algorithm for the
recovery of 1-D sparse signals. This (heuristic-deterministic) hybrid mechanism
greatly improves the convergence of GA and PSO. Based on the idea of GA, a
modified POCS algorithm is also proposed for the reconstruction of Fourier
encoded images. The novel recovery algorithm uses multiple initialization and
randomly combines them during each iteration to estimate the original image.
Introducing randomness in the recovery process provides improved results as
compared to SSF based recovery.
CS recovery algorithms generally use to reconstruct a high-quality image from
the sub-sampled incoherent measurements by finding solution to the least
squares optimization problem with 𝑙1-norm regularization. However, the 𝑙1-
norm is not differential at the origin as the function 𝑓(𝑥) = |𝑥| has a kink at 𝑥 =
0. To overcome this problem, a hyperbolic tangent based approximation has
been proposed. Simultaneously, a gradient based algorithm is developed for the
recovery of Fourier-encoded images. The proposed technique also proves its
ability to recover a sparse signal with and without the knowledge of sparsity.
115
Shrinkage is an appealing and well-known denoising technique. Undersampling
in CS results in an image corrupted by Gaussian-like noise. The MAP estimator
of a Laplace random variable in Gaussian noise leads to the soft-thresholding.
However, in practice, the assumption of Gaussian noise and Laplacian
distribution of transformed coefficients may not meet exactly. Therefore, a
novel hyperbolic-tangent based shrinkage is proposed that can closely
approximate the non-linear soft-thresholding function. The proposed non-linear
shrinkage has adjustable parameters and is shown to perform well in
reconstructing biomedical images from partial Fourier measurements.
8.2 Directions for future work
There are several directions and extensions for the future research work based on the
ideas presented in this dissertation. For example:
The work can be extended to dynamic MR imaging which requires rapid data
acquisition to monitor fast signal-intensity changes. Because of the
computational simplicity, the proposed algorithms can be also be used with
parallel imaging to reconstruct rapid dynamic volumetric MR images with high
temporal resolution, spatial resolution and motion robustness.
In the current work, analytical sparsifying transforms (dictionary) were used.
The recovery process may improve if an adaptive dictionary learning
mechanism is incorporated in the reconstruction. The currently used dictionary
learning algorithms such as K-SVD etc. are computationally intensive. These
116
learning techniques usually use pursuit algorithms for sparse coding which can
be easily replaced by the proposed computationally low cost algorithms.
The proposed CS reconstruction algorithms can be used for the recovery of
videos from compressed measurements. However, it may need to incorporate
an intelligent sensing mechanism to exploit the high correlation in a video
sequence.
Instead of recovering the entire biomedical image at once, the iterative
shrinkage based CS recovery can be extended to patch-based CS reconstruction.
Additionally, the non-linear shrinkage functions/curves for each patch may be
learned adaptively. This shift from global to local and adaptive modeling is
expected to provide better reconstruction quality.
117
REFERENCES
[1] E. J. Candès, J. Romberg, and T. Tao, "Robust uncertainty principles: Exact
signal reconstruction from highly incomplete frequency information,"
Information Theory, IEEE Transactions on, vol. 52, pp. 489-509, 2006.
[2] D. L. Donoho, "Compressed sensing," IEEE Transactions on Information
Theory, vol. 52, pp. 1289-1306, Apr 2006.
[3] R. G. Baraniuk, E. Candes, R. Nowak, and M. Vetterli, "Compressive
sampling," IEEE Signal Processing Magazine, vol. 25, pp. 12-13, Mar 2008.
[4] E. J. Candes and M. B. Wakin, "An introduction to compressive sampling,"
IEEE Signal Processing Magazine, vol. 25, pp. 21-30, Mar 2008.
[5] J. Romberg, "Imaging via compressive sampling [introduction to compressive
sampling and recovery via convex programming]," IEEE Signal Processing
Magazine, vol. 25, pp. 14-20, 2008.
[6] M. Lustig, D. Donoho, and J. M. Pauly, "Sparse MRI: The application of
compressed sensing for rapid MR imaging," Magnetic Resonance in Medicine,
vol. 58, pp. 1182-1195, Dec 2007.
[7] M. Lustig, D. L. Donoho, J. M. Santos, and J. M. Pauly, "Compressed sensing
MRI," IEEE Signal Processing Magazine, vol. 25, pp. 72-82, Mar 2008.
[8] J. P. Haldar, D. Hernando, and Z.-P. Liang, "Compressed-sensing MRI with
random encoding," Medical Imaging, IEEE Transactions on, vol. 30, pp. 893-
903, 2011.
118
[9] M. Elad and M. Aharon, "Image denoising via sparse and redundant
representations over learned dictionaries," IEEE Trans Image Process, vol. 15,
pp. 3736-45, Dec 2006.
[10] S. Ravishankar and Y. Bresler, "MR image reconstruction from highly
undersampled k-space data by dictionary learning," Medical Imaging, IEEE
Transactions on, vol. 30, pp. 1028-1041, 2011.
[11] V. M. Patel, R. Maleh, A. C. Gilbert, and R. Chellappa, "Gradient-Based Image
Recovery Methods From Incomplete Fourier Measurements," IEEE
Transactions on Image Processing, vol. 21, pp. 94-105, Jan 2012.
[12] S. S. Chen, D. L. Donoho, and M. A. Saunders, "Atomic decomposition by basis
pursuit," SIAM journal on scientific computing, vol. 20, pp. 33-61, 1998.
[13] B. Ayazifar, "Rethinking Fourier's legacy in signals and systems education," in
Circuits and Systems (ISCAS), 2011 IEEE International Symposium on, 2011,
pp. 599-602.
[14] P. J. Olver and C. Shakiban, "Applied linear algebra," Upper Saddle River,
2006.
[15] E. Candes and J. Romberg, "Signal recovery from random projections," in Proc.
SPIE, 2005, pp. 76-86.
[16] S. Mallat, A wavelet tour of signal processing: the sparse way: Academic press,
2008.
[17] G. Strang, "The discrete cosine transform," SIAM review, vol. 41, pp. 135-147,
1999.
119
[18] V. Strela, P. N. Heller, G. Strang, P. Topiwala, and C. Heil, "The application of
multiwavelet filterbanks to image processing," Image Processing, IEEE
Transactions on, vol. 8, pp. 548-563, 1999.
[19] M. Rabbani and D. Santa-Cruz, "The JPEG2000 still-image compression
standard," in Course given at the 2001 International Conference in Image
Processing (ICIP), Thessaloniki, Greece, 2001.
[20] I. Bayram and M. E. Kamasak, "Directional total variation," Signal Processing
Letters, IEEE, vol. 19, pp. 781-784, 2012.
[21] K. Choi, J. Wang, L. Zhu, T.-S. Suh, S. Boyd, and L. Xing, "Compressed
sensing based cone-beam computed tomography reconstruction with a first-
order methoda)," Medical physics, vol. 37, pp. 5113-5125, 2010.
[22] R. Baraniuk, M. Davenport, R. DeVore, and M. Wakin, "A simple proof of the
restricted isometry property for random matrices," Constructive
Approximation, vol. 28, pp. 253-263, 2008.
[23] E. J. Candes, "The restricted isometry property and its implications for
compressed sensing," Comptes Rendus Mathematique, vol. 346, pp. 589-592,
2008.
[24] H. Kajbaf, R. Z. YAHONG, and R. Zoughi, "Improving Efficiency of
Microwave Wideband Imaging using Compressed Sensing Techniques,"
Materials evaluation, vol. 70, pp. 1420-1432, 2012.
[25] I. F. Gorodnitsky, J. S. George, and B. D. Rao, "Neuromagnetic source imaging
with FOCUSS: a recursive weighted minimum norm algorithm,"
120
Electroencephalography and clinical Neurophysiology, vol. 95, pp. 231-251,
1995.
[26] J. C. Ye, Y. Bresler, and P. Moulin, "A self-referencing level-set method for
image reconstruction from sparse Fourier samples," International Journal of
Computer Vision, vol. 50, pp. 253-270, 2002.
[27] M. Akçakaya, S. Nam, P. Hu, M. H. Moghari, L. H. Ngo, V. Tarokh, et al.,
"Compressed sensing with wavelet domain dependencies for coronary MRI: a
retrospective study," Medical Imaging, IEEE Transactions on, vol. 30, pp. 1090-
1099, 2011.
[28] M. Lustig, J. M. Santos, D. L. Donoho, and J. M. Pauly, "kt SPARSE: High
frame rate dynamic MRI exploiting spatio-temporal sparsity," in Proceedings
of the 13th Annual Meeting of ISMRM, Seattle, 2006.
[29] L. Feng, R. Grimm, K. T. Block, H. Chandarana, S. Kim, J. Xu, et al., "Golden‐
angle radial sparse parallel MRI: Combination of compressed sensing, parallel
imaging, and golden‐angle radial sampling for fast and flexible dynamic
volumetric MRI," Magnetic Resonance in Medicine, 2013.
[30] S. Vasanawala, M. Murphy, M. T. Alley, P. Lai, K. Keutzer, J. M. Pauly, et al.,
"Practical parallel imaging compressed sensing MRI: Summary of two years of
experience in accelerating body MRI of pediatric patients," in Biomedical
Imaging: From Nano to Macro, 2011 IEEE International Symposium on, 2011,
pp. 1039-1043.
[31] M. Murphy, M. Alley, J. Demmel, K. Keutzer, S. Vasanawala, and M. Lustig,
"Fast-SPIRiT compressed sensing parallel imaging MRI: scalable parallel
121
implementation and clinically feasible runtime," Medical Imaging, IEEE
Transactions on, vol. 31, pp. 1250-1262, 2012.
[32] S. Bélanger, X. Intes, C. Casanova, F. Lesage, and M. Abran, "Real-time diffuse
optical tomography based on structured illumination," Journal of biomedical
optics, vol. 15, pp. 016006-016006-7, 2010.
[33] J. Provost and F. Lesage, "The application of compressed sensing for photo-
acoustic tomography," Medical Imaging, IEEE Transactions on, vol. 28, pp.
585-594, 2009.
[34] G.-H. Chen, J. Tang, and S. Leng, "Prior image constrained compressed sensing
(PICCS)," in Proc Soc Photo Opt Instrum Eng, 2008, p. 685618.
[35] H. Yu and G. Wang, "A soft-threshold filtering approach for reconstruction
from a limited number of projections," Physics in medicine and biology, vol.
55, p. 3905, 2010.
[36] Z. T. Harmany, R. F. Marcia, and R. M. Willett, "Sparsity-regularized photon-
limited imaging," in Biomedical Imaging: From Nano to Macro, 2010 IEEE
International Symposium on, 2010, pp. 772-775.
[37] W. Ge, Y. Bresler, and V. Ntziachristos, "Guest Editorial Compressive Sensing
for Biomedical Imaging," Medical Imaging, IEEE Transactions on, vol. 30, pp.
1013-1016, 2011.
[38] C. Quinsac, A. Basarab, and D. Kouamé, "Frequency domain compressive
sampling for ultrasound imaging," Advances in Acoustics and Vibration, vol.
2012, 2012.
122
[39] M. Usman, D. Atkinson, F. Odille, C. Kolbitsch, G. Vaillant, T. Schaeffter, et
al., "Motion corrected compressed sensing for free‐breathing dynamic cardiac
MRI," Magnetic Resonance in Medicine, vol. 70, pp. 504-516, 2013.
[40] X. Li, "Compressed sensing and matrix completion with constant proportion of
corruptions," Constructive Approximation, vol. 37, pp. 73-99, 2013.
[41] J. Liu, P. Musialski, P. Wonka, and J. Ye, "Tensor completion for estimating
missing values in visual data," Pattern Analysis and Machine Intelligence, IEEE
Transactions on, vol. 35, pp. 208-220, 2013.
[42] Q. Liu, M. Zhang, and J. Zhao, "Adaptive dictionary learning in sparse gradient
domain for CT reconstruction," in Biomedical Imaging (ISBI), 2014 IEEE 11th
International Symposium on, 2014, pp. 169-172.
[43] J. A. Tropp and S. J. Wright, "Computational methods for sparse solution of
linear inverse problems," Proceedings of the IEEE, vol. 98, pp. 948-958, 2010.
[44] Y. C. Pati, R. Rezaiifar, and P. Krishnaprasad, "Orthogonal matching pursuit:
Recursive function approximation with applications to wavelet decomposition,"
in Signals, Systems and Computers, 1993. 1993 Conference Record of The
Twenty-Seventh Asilomar Conference on, 1993, pp. 40-44.
[45] D. L. Donoho, Y. Tsaig, I. Drori, and J.-L. Starck, "Sparse solution of
underdetermined systems of linear equations by stagewise orthogonal matching
pursuit," Information Theory, IEEE Transactions on, vol. 58, pp. 1094-1121,
2012.
123
[46] D. Needell and J. A. Tropp, "CoSaMP: Iterative signal recovery from
incomplete and inaccurate samples," Applied and Computational Harmonic
Analysis, vol. 26, pp. 301-321, 2009.
[47] J. Z. Duan, Y. Liu, and L. Y. Zhang, "Bregman Iteration Based Efficient
Algorithm for MR Image Reconstruction From Undersampled K-Space Data,"
IEEE Signal Processing Letters, vol. 20, pp. 831-834, Aug 2013.
[48] M. A. Figueiredo, R. D. Nowak, and S. J. Wright, "Gradient projection for
sparse reconstruction: Application to compressed sensing and other inverse
problems," Selected Topics in Signal Processing, IEEE Journal of, vol. 1, pp.
586-597, 2007.
[49] S. J. Wright, R. D. Nowak, and M. A. Figueiredo, "Sparse reconstruction by
separable approximation," Signal Processing, IEEE Transactions on, vol. 57,
pp. 2479-2493, 2009.
[50] Y. Le Montagner, E. Angelini, and J.-C. Olivo-Marin, "Comparison of
reconstruction algorithms in compressed sensing applied to biological
imaging," in Biomedical Imaging: From Nano to Macro, 2011 IEEE
International Symposium on, 2011, pp. 105-108.
[51] C. A. Metzler, A. Maleki, and R. G. Baraniuk, "From denoising to compressed
sensing," arXiv preprint arXiv:1406.4175, 2014.
[52] J. Jin, B. Yang, K. Liang, and X. Wang, "General image denoising framework
based on compressive sensing theory," Computers & Graphics, vol. 38, pp. 382-
391, 2014.
124
[53] M. Elad, "Why simple shrinkage is still relevant for redundant
representations?," Information Theory, IEEE Transactions on, vol. 52, pp.
5559-5569, 2006.
[54] T. Blumensath, "Accelerated iterative hard thresholding," Signal Processing,
vol. 92, pp. 752-756, 2012.
[55] A. Beck and M. Teboulle, "A Fast Iterative Shrinkage-Thresholding Algorithm
for Linear Inverse Problems," Siam Journal on Imaging Sciences, vol. 2, pp.
183-202, 2009.
[56] M. Elad, B. Matalon, and M. Zibulevsky, "Coordinate and subspace
optimization methods for linear least squares with non-quadratic
regularization," Applied and Computational Harmonic Analysis, vol. 23, pp.
346-367, 2007.
[57] M. Zibulevsky and M. Elad, "L1-L2 Optimization in Signal and Image
Processing," IEEE Signal Processing Magazine, vol. 27, pp. 76-88, May 2010.
[58] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality
assessment: from error visibility to structural similarity," Image Processing,
IEEE Transactions on, vol. 13, pp. 600-612, 2004.
[59] D. W. McRobbie, E. A. Moore, M. J. Graves, and M. R. Prince, MRI from
Picture to Proton: Cambridge University Press, 2006.
[60] T. Schaeffter, M. S. Hansen, and T. S. Sørensen, "Fast Implementation of
Iterative Image Reconstruction," in International Society for Magnetic
Resonance in Medicine.
125
[61] H. Omer and R. Dickinson, "Regularization in Parallel MR Image
Reconstruction," Concepts in Magnetic Resonance Part A, vol. 38A, pp. 52-60,
Mar 2011.
[62] C. M. Tsai and D. G. Nishimura, "Reduced aliasing artifacts using variable‐
density k‐space sampling trajectories," Magnetic resonance in medicine, vol.
43, pp. 452-458, 2000.
[63] R. Cierniak and M. Butynski, X-ray computed tomography in biomedical
engineering: Springer, 2011.
[64] O. Barkan, A. Averbuch, S. Dekel, and Y. Tenzer, "A Mathematical Model for
Extremely Low Dose Adaptive Computed Tomography Acquisition," in
Mathematical Methods for Curves and Surfaces, ed: Springer, 2014, pp. 13-33.
[65] R. Maleh, "Efficient sparse approximation methods for medical imaging," The
University of Michigan, 2009.
[66] Z. Wang, E. G. Lim, Y. Tang, and M. Leach, "Medical Applications of
Microwave Imaging," The Scientific World Journal, vol. 2014, 2014.
[67] D. M. Sheen, D. L. McMakin, and T. E. Hall, "Three-dimensional millimeter-
wave imaging for concealed weapon detection," Microwave Theory and
Techniques, IEEE Transactions on, vol. 49, pp. 1581-1592, 2001.
[68] B. Zhang, W. Hong, and Y. Wu, "Sparse microwave imaging: Principles and
applications," Science China Information Sciences, vol. 55, pp. 1722-1754,
2012.
126
[69] L. C. Potter, E. Ertin, J. T. Parker, and M. Cetin, "Sparsity and compressed
sensing in radar imaging," Proceedings of the IEEE, vol. 98, pp. 1006-1020,
2010.
[70] L. Bregman, "The method of successive projection for finding a common point
of convex sets (Theorems for determining common point of convex sets by
method of successive projection)," Soviet Mathematics, vol. 6, pp. 688-692,
1965.
[71] D. C. Youla and H. Webb, "Image Restoration by the Method of Convex
Projections: Part 1ߞTheory," Medical Imaging, IEEE Transactions on, vol. 1,
pp. 81-94, 1982.
[72] M. Elad, B. Matalon, J. Shtok, and M. Zibulevsky, "A wide-angle view at
iterated shrinkage algorithms," in Optical Engineering+ Applications, 2007, pp.
670102-670102-19.
[73] M. Elad, Sparse and redundant representations: from theory to applications in
signal and image processing: Springer, 2010.
[74] M. Lustig, D. Donoho, and J. Pauly, "Rapid MR imaging with compressed
sensing and randomly under-sampled 3DFT trajectories," in Proc. 14th Ann.
Meeting ISMRM, 2006.
[75] U. Gamper, P. Boesiger, and S. Kozerke, "Compressed sensing in dynamic
MRI," Magnetic Resonance in Medicine, vol. 59, pp. 365-373, 2008.
[76] G. A. Wright, "Magnetic resonance imaging," IEEE Signal Processing
Magazine, vol. 14, pp. 56-66, Jan 1997.
127
[77] M. K. Stehling, R. Turner, and P. Mansfield, "Echo-planar imaging: magnetic
resonance imaging in a fraction of a second," Science, vol. 254, pp. 43-50, 1991.
[78] F. Schmitt, M. K. Stehling, and R. Turner, "Echo-planar imaging," 1998.
[79] M. Lustig and J. M. Pauly, "SPIRiT: Iterative self‐consistent parallel imaging
reconstruction from arbitrary k‐space," Magnetic Resonance in Medicine, vol.
64, pp. 457-471, 2010.
[80] K. Scheffler and J. Hennig, "Reduced circular field‐of‐view imaging," Magnetic
resonance in medicine, vol. 40, pp. 474-480, 1998.
[81] A. Deshmane, V. Gulani, M. A. Griswold, and N. Seiberlich, "Parallel MR
imaging," Journal of Magnetic Resonance Imaging, vol. 36, pp. 55-72, 2012.
[82] K. P. Pruessmann, "Encoding and reconstruction in parallel MRI," NMR in
Biomedicine, vol. 19, pp. 288-299, 2006.
[83] L. Grady and J. R. Polimeni, "Method for reconstructing images of an imaged
subject from a parallel MRI acquisition," ed: Google Patents, 2014.
[84] M. Uecker, P. Lai, M. J. Murphy, P. Virtue, M. Elad, J. M. Pauly, et al.,
"ESPIRiT—an eigenvalue approach to autocalibrating parallel MRI: where
SENSE meets GRAPPA," Magnetic Resonance in Medicine, vol. 71, pp. 990-
1001, 2014.
[85] K. L. Wright, J. I. Hamilton, M. A. Griswold, V. Gulani, and N. Seiberlich,
"Non‐Cartesian parallel imaging reconstruction," Journal of Magnetic
Resonance Imaging, vol. 40, pp. 1022-1040, 2014.
128
[86] D. h. Kim, E. Adalsteinsson, and D. M. Spielman, "Simple analytic variable
density spiral design," Magnetic resonance in medicine, vol. 50, pp. 214-219,
2003.
[87] X. Li, S. Luo, A. Talbert, R. Eisner, and F. DiBianca, "A compressed sensing-
based iterative algorithm for CT reconstruction and its possible application to
phase contrast imaging," Biomed Eng Online, vol. 10, pp. 1-14, 2011.
[88] H. Kudo, "Introduction to advanced image reconstruction methods and
compressed sensing in medical computed tomography," Microscopy, vol. 63,
pp. i15-i15, 2014.
[89] R. Chartrand, E. Y. Sidky, and X. Pan, "Nonconvex compressive sensing for X-
ray CT: an algorithm comparison," in Signals, Systems and Computers, 2013
Asilomar Conference on, 2013, pp. 665-669.
[90] S. Uḡur and O. Arıkan, "SAR image reconstruction and autofocus by
compressed sensing," Digital Signal Processing, vol. 22, pp. 923-932, 2012.
[91] G. Oliveri, L. Poli, and A. Massa, "On the exploitation of the a-priori
information through the Bayesian compressive sensing for microwave
imaging," in Antennas and Propagation (EUCAP), 2012 6th European
Conference on, 2012, pp. 1502-1503.
[92] A. Korn, M. Fenchel, B. Bender, S. Danz, T.-K. Hauser, D. Ketelsen, et al.,
"Iterative reconstruction in head CT: image quality of routine and low-dose
protocols in comparison with standard filtered back-projection," American
journal of neuroradiology, vol. 33, pp. 218-224, 2012.
129
[93] A. Moscariello, R. A. Takx, U. J. Schoepf, M. Renker, P. L. Zwerner, T. X.
O’Brien, et al., "Coronary CT angiography: image quality, diagnostic accuracy,
and potential for radiation dose reduction using a novel iterative image
reconstruction technique—comparison with traditional filtered back projection,"
European radiology, vol. 21, pp. 2130-2138, 2011.
[94] P. L. Combettes and J.-C. Pesquet, "Proximal splitting methods in signal
processing," in Fixed-point algorithms for inverse problems in science and
engineering, ed: Springer, 2011, pp. 185-212.
[95] E. Gopi, Algorithm collections for digital signal processing applications using
Matlab: Springer Science & Business Media, 2007.
[96] B. Craenen, A. Eiben, and E. Marchiori, "How to handle constraints with
evolutionary algorithms," Practical Handbook Of Genetic Algorithms:
Applications, pp. 341-361, 2001.
[97] G. T. Pulido and C. A. C. Coello, "A constraint-handling mechanism for particle
swarm optimization," in Evolutionary Computation, 2004. CEC2004. Congress
on, 2004, pp. 1396-1403.
[98] Y. Wang, Z. Cai, Y. Zhou, and Z. Fan, "Constrained optimization based on
hybrid evolutionary algorithm and adaptive constraint-handling technique,"
Structural and Multidisciplinary Optimization, vol. 37, pp. 395-413, 2009.
[99] G. Coath and S. Halgamuge, "A comparison of constraint-handling methods for
the application of particle swarm optimization to constrained nonlinear
optimization problems," in Evolutionary Computation, 2003. CEC'03. The 2003
Congress on, 2003, pp. 2419-2425.
130
[100] J. A. Shah, I. Qureshi, A. A. Khaliq, and H. Omer, "Sparse Signal Recovery
based on Hybrid Genetic Algorithm," Research Journal of Recent Sciences, vol.
2277, p. 2502, 2013.
[101] J. Kennedy, "Particle swarm optimization," in Encyclopedia of Machine
Learning, ed: Springer, 2010, pp. 760-766.
[102] G. Venter and J. Sobieszczanski-Sobieski, "Particle swarm optimization,"
AIAA journal, vol. 41, pp. 1583-1589, 2003.
[103] Y. Shi, "Particle swarm optimization," IEEE Connections, vol. 2, pp. 8-13,
2004.
[104] R. Poli, J. Kennedy, and T. Blackwell, "Particle swarm optimization," Swarm
intelligence, vol. 1, pp. 33-57, 2007.
[105] M. Clerc, Particle swarm optimization vol. 93: John Wiley & Sons, 2010.
[106] R. C. Eberhart and Y. Shi, "Particle swarm optimization: developments,
applications and resources," in Evolutionary Computation, 2001. Proceedings
of the 2001 Congress on, 2001, pp. 81-86.
[107] K. De Jong, "Learning with genetic algorithms: An overview," Machine
learning, vol. 3, pp. 121-138, 1988.
[108] M. Mitchell, "Genetic algorithms: An overview," Complexity, vol. 1, pp. 31-
39, 1995.
[109] M. Mitchell, An introduction to genetic algorithms: MIT press, 1998.
[110] R. L. Haupt and S. E. Haupt, Practical genetic algorithms: John Wiley & Sons,
2004.
131
[111] C. R. Houck, J. Joines, and M. G. Kay, "A genetic algorithm for function
optimization: a Matlab implementation," NCSU-IE TR, vol. 95, 1995.
[112] L. J. Eshelman and J. D. Schaffer, "Preventing Premature Convergence in
Genetic Algorithms by Preventing Incest," in ICGA, 1991, pp. 115-122.
[113] M. Rocha and J. Neves, "Preventing premature convergence to local optima in
genetic algorithms via random offspring generation," in Multiple Approaches
to Intelligent Systems, ed: Springer, 1999, pp. 127-136.
[114] M. Schoenauer and S. Xanthakis, "Constrained GA Optimization," in ICGA,
1993, pp. 573-580.
[115] B. Widrow, J. M. McCool, M. Larimore, and C. R. Johnson, "Stationary and
nonstationary learning characteristics of the LMS adaptive filter," Proceedings
of the IEEE, vol. 64, pp. 1151-1162, 1976.
[116] S. S. Haykin, Adaptive filter theory: Pearson Education India, 2008.
[117] J. R. Treichler, C. R. Johnson, and M. G. Larimore, Theory and design of
adaptive filters: Wiley, 1987.
[118] H. R. Miller, Optimization: foundations and applications: John Wiley & Sons,
2011.
[119] S. Boyd and L. Vandenberghe, Convex optimization: Cambridge university
press, 2004.
[120] S. Grossberg, "Nonlinear neural networks: Principles, mechanisms, and
architectures," Neural networks, vol. 1, pp. 17-61, 1988.
132
[121] S. S. Haykin, S. S. Haykin, S. S. Haykin, and S. S. Haykin, Neural networks
and learning machines vol. 3: Pearson Education Upper Saddle River, 2009.
[122] B. I. Yun, "A non-iterative method for solving non-linear equations," Applied
Mathematics and Computation, vol. 198, pp. 691-699, 2008.
[123] A. Menon, K. Mehrotra, C. K. Mohan, and S. Ranka, "Characterization of a
class of sigmoid functions with applications to neural networks," Neural
Networks, vol. 9, pp. 819-835, 1996.
[124] B. L. Kalman and S. C. Kwasny, "Why Tanh: choosing a sigmoidal function,"
in Neural Networks, 1992. IJCNN., International Joint Conference on, 1992, pp.
578-581.
[125] X.-F. Fang, J.-S. Zhang, and Y.-Q. Li, "Sparse Signal Reconstruction Based on
Multiparameter Approximation Function with Smoothed Norm," Mathematical
Problems in Engineering, vol. 2014, p. 9, 2014.
[126] W. Wang and Q. Wang, "Approximated Function Based Spectral Gradient
Algorithm for Sparse Signal Recovery," Statistics, Optimization & Information
Computing, vol. 2, pp. 10-20, 2014.
[127] J. Barzilai and J. M. Borwein, "Two-point step size gradient methods," IMA
Journal of Numerical Analysis, vol. 8, pp. 141-148, 1988.
[128] D. L. Donoho, "De-noising by soft-thresholding," Information Theory, IEEE
Transactions on, vol. 41, pp. 613-627, 1995.
[129] F. Luisier, T. Blu, and M. Unser, "A new SURE approach to image denoising:
Interscale orthonormal wavelet thresholding," Image Processing, IEEE
Transactions on, vol. 16, pp. 593-606, 2007.
133
[130] S. G. Chang, B. Yu, and M. Vetterli, "Adaptive wavelet thresholding for image
denoising and compression," Image Processing, IEEE Transactions on, vol. 9,
pp. 1532-1546, 2000.
[131] I. K. Fodor and C. Kamath, "Denoising through wavelet shrinkage: an empirical
study," Journal of Electronic Imaging, vol. 12, pp. 151-160, 2003.
[132] R. Rangarajan, R. Venkataramanan, and S. Shah, "Image denoising using
wavelets," Wavelet and Time Frequencies, 2002.
[133] L. Breiman, "Better subset regression using the nonnegative garrote,"
Technometrics, vol. 37, pp. 373-384, 1995.
[134] L. Sendur and I. W. Selesnick, "Bivariate shrinkage functions for wavelet-based
denoising exploiting interscale dependency," Signal Processing, IEEE
Transactions on, vol. 50, pp. 2744-2756, 2002.
[135] M. Nasri and H. Nezamabadi-pour, "Image denoising in the wavelet domain
using a new adaptive thresholding function," Neurocomputing, vol. 72, pp.
1012-1025, 2009.
[136] J. Prinosil, Z. Smekal, and K. Bartusek, "Wavelet thresholding techniques in
MRI domain," in Biosciences (BIOSCIENCESWORLD), 2010 International
Conference on, 2010, pp. 58-63.
