Post on 18-Nov-2021
1
HighlyAcceleratedMultishotEPIthrough
SynergisticMachineLearningandJointReconstruction
BerkinBilgic1,2,3,ItthiChatnuntawech4,MaryKateManhard1,2,QiyuanTian1,2,CongyuLiao1,2,StephenF.
Cauley1,2,SusieY.Huang1,2,3,JonathanR.Polimeni1,2,3,LawrenceL.Wald1,2,3,KawinSetsompop1,2,3
1AthinoulaA.MartinosCenterforBiomedicalImaging,Charlestown,MA,USA
2DepartmentofRadiology,HarvardMedicalSchool,Boston,MA,USA
3Harvard-MITHealthSciencesandTechnology,MIT,Cambridge,MA,USA
4NationalNanotechnologyCenter,NationalScienceandTechnologyDevelopmentAgency,PathumThani,
Thailand
Correspondingauthor:
ItthiChatnuntawech
itthi.cha@nanotec.or.th
Bodywordcount:~6400
Keywords:
MultishotEPI,parallelimaging,machinelearning,deeplearning,convolutionalneuralnetwork,joint
reconstruction
2
ABSTRACT
Purpose: To introduce a combined machine learning (ML) and physics-based image reconstruction
framework that enables navigator-free, highly acceleratedmultishot echo planar imaging (msEPI), and
demonstrateitsapplicationinhigh-resolutionstructuralanddiffusionimaging.
Methods:SingleshotEPIisanefficientencodingtechnique,butdoesnotlenditselfwelltohigh-resolution
imaging due to severe distortion artifacts and blurring.WhilemsEPI canmitigate these artifacts, high-
quality msEPI has been elusive because of phase mismatch arising from shot-to-shot variations which
preclude the combination of the multiple-shot data into a single image.We employ deep learning to
obtainaninterimimagewithminimalartifacts,whichpermitsestimationofimagephasevariationsdueto
shot-to-shotchanges.Thesevariationsarethen includedinaJointVirtualCoilSensitivityEncoding(JVC-
SENSE)reconstructiontoutilizedatafromallshotsandimproveupontheMLsolution.
Results:Our combinedML+physicsapproachenabledRinplane xMultiBand (MB)=8x2-foldacceleration
using2EPI-shotsformulti-echoimaging,sothatwhole-brainT2andT2*parametermapscouldbederived
from an 8.3 sec acquisition at 1x1x3mm3 resolution. This has also allowed high-resolution diffusion
imagingwithhighgeometricfidelityusing5-shotsatRinplanexMB=9x2-foldacceleration.Tomakethese
possible,weextendedthestate-of-the-artMUSSELSreconstructiontechniquetoSimultaneousMultiSlice
(SMS)encodinganduseditasaninputtoourMLnetwork.
Conclusion:CombinationofMLandJVC-SENSEenablednavigator-freemsEPIathigheraccelerationsthan
previouslypossiblewhileusing fewershots,withreducedvulnerability topoorgeneralizabilityandpoor
acceptanceofend-to-endMLapproaches.
3
INTRODUCTION
Slow imageencodinghasconstrainedclinicalMRI scans touse2-dimensionalencodingand thick slices,
often with slice gaps, so that whole-brain exams can be completed within acceptable time frames. In
addition to the information loss, such inefficient acquisition poses a barrier to MRI evaluation of
hospitalizedpatientswhoare critically ill andcanneitherhold still nor tolerate long scans.Lowpatient
throughputduetoinefficientimagingalsoincreasesthetimefromsymptomonsettodiagnosis,thereby
delayingtreatment.
To overcome the slow image encoding barrier, recent screening protocols havemoved to singleshot
Echo Planar Imaging (ssEPI) to provide multi-contrast information (1,2). Unfortunately, the reduced
geometric fidelityof theseprotocolsmayconfound/obscure localizationof salient imaging findings.The
problemarisesfromseveredistortionandblurringartifactsinssEPIathighin-planeresolutions,wherea
largeareaofk-spacehastobecoveredwithinasinglereadoutinthepresenceofB0inhomogeneityand
T2* signal decay. These effects are only partiallymitigated at relatively high in-plane acceleration (e.g.
Rinplane=3).
Whilemultishot EPI (msEPI) canmitigateblurring anddistortion, high-qualitymsEPI has beenelusive
becausecombiningthemultiple-shotdataintoasingleimageisprohibitivelydifficult,especiallyathighin-
plane acceleration. Image phase mismatches between the shots caused by physiological variations
(respiration, cardiac pulsation) or motion under the influence of diffusion encoding gradients lead tosevereghostingartifacts.Todate,theapplicationofmsEPIhasbeenrestrictedtodiffusionimaging,where
two types of solutions have been proposed to combine the shots: (i) navigator-based approaches that
requireadditionaldataacquisitiontocaptureshot-to-shotphasevariations (3–7),and(ii)navigator-free
techniques that estimate these variations from the data itself (8–11). In (ii), multiplexed sensitivity
encoding (MUSE) (9) and its extensions (12,13), relyonparallel imaging to reconstruct an intermediate
imageforeachshotindependentlytoestimatethephysiologicalvariationsbeforejointlyreconstructingall
multishotdatatogether.This limitstheachievabledistortionandblurringreductionto4to6-fold,since
parallel imagingwithmodernRFreceivecoilarraysbreaksdownbeyondsuchaccelerationinthephase-
encoding direction.MUSSELS, on the other hand, does not explicitly estimate the phase of each shot
image,butemploys sensitivityencodingandsimilaritiesacrossmultishotdata in the formof structured
low-rankmatrixcompletion(11,14).ThishasallowedMUSSELStoundersamplethek-spaceofeachshot
byRinplane=8-foldtoreducedistortionandblurringartifactsaswellastheechotime(TE).Imagescouldbe
successfullyreconstructedusing4-shotsofdata,sothatthenetaccelerationfactorbecameRnet=8/4=2-
fold.Itisimportanttonoteanotherclassofnavigator-freemultishotdiffusionimagingtechniques,which
utilize non-Cartesian trajectories that allow for estimation of low-resolution image phase information
from thedensely sampledportionofeach shot (15,16). Such self-navigationpropertymaycomeat the
costofblurring/distortionintheresultingimages.
4
In this contribution, we introduce a new reconstruction framework that utilizes a synergistic
combination of machine learning (ML) and physics (or forward-model) based reconstruction, and
demonstrate itsapplicationinstructuralanddiffusionmsEPIwithhighgeometricfidelity.Wetermour
combinedML + physics approachNetwork EstimatedArtifacts for Tempered Reconstruction (NEATR),
and incorporate Simultaneous MultiSlice (SMS) for extra efficiency. To this end, we have extended
MUSSELStoSMSencoding,andutilizedthereadoutextendedFOVconcept(17)toseamlesslyintegrate
sliceaccelerationintothisframework.WestartfromSMS-MUSSELSreconstructionofhighlyaccelerated
msEPIusingasmallernumberofacquisitionshots,andpasstheintermediatesolutionthroughourdeep
neural network tomitigate the reconstruction artifacts from SMS-MUSSELS. Using this interim image
with minimal artifacts allows us to solve for the image phase of each shot using phase-regularized
parallel imaging (18). Given the phase of each shot, we then perform a Joint Virtual Coil Sensitivity
Encoding(JVC-SENSE)reconstructionwhereweutilizethek-spacedatafromallshotsaswellasvirtual
coilconcept(19–21)tosolveforthecombinedmagnitudeimage.
We demonstrate the application of SMS-NEATR in spin-and-gradient-echo (SAGE (22)) msEPI
acquisition at Rinplane xMultiBand (MB) = 8x2-fold acceleration using 2-shots. Compared to the newly
developedSMS-MUSSELSreconstruction,whichisalsousedasaninputtoournetwork,wedemonstrate
~30%improvementinroot-mean-squarederror(RMSE)inhigh-resolutionstructuralimages.Weobserve
larger gains in ghosting/aliasing artifactmitigation in the harder problem of diffusion imaging, where
SMS-NEATRallowsforRinplanexMB=9x2-foldaccelerationusing5-shots.
These are made possible by the deep learning step that enables phase estimations at such high
accelerationfactors.Importantly,thefinaluseofarigorousphysics-basedforward-modelreconstruction
limits theroleofML in the final reconstruction.Thus,SMS-NEATRallowsus to tap intothepotentialof
convolutionalneuralnetworks (CNN)tosolve for importantnuisancemodulationsandunknowns in the
forward model without treating the reconstruction as an end-to-end process. The result is a better
harnessedsensitivityencodingwithfullutilizationofthescannerhardware.Ourstrategypavesthewayto
reaping the benefits of ML while constraining potential damage from utilizing it on data beyond its
training experience, andwithout being exposed to the vulnerabilities of not knowing exactly what the
reconstruction is doing. Our approach of using ML to estimate nuisance parameters that are hard to
determine could allow physics-based reconstructions to work well in other applications, such as
retrospectivemotion-correctionwithoutnavigationoradditionalhardware.
We provide Matlab source code and data to reproduce our diffusion msEPI results here:
https://bit.ly/2QgBg9U
SupportingInformationfigurescanbeaccessedhere:https://bit.ly/2unY2iJ
5
METHODS
ReconstructionOverview
TheSMS-NEATRflowchartispresentedinFig.1.WebeginbyperforminganSMS-MUSSELSreconstruction
on the highly-accelerated (e.g. Rinplane xMB = 8x2)msEPI data to obtain an initial image estimatewith
mitigatedartifacts from thenuisancephasebetween shots. The image is further improvedusingU-Net
processing(23)whichestimatesarefinedimagewithminimalartifacts.Startingfromthisreconstruction,
weestimate thephase imagecorresponding toeachshotusingphase-regularizedparallel imaging (24).
Giventheestimatedshot-to-shotphasevariations,wethenperformaphysics-basedjointreconstruction
(JVC-SENSE)toarriveatthefinalsolution.JVC-SENSEincorporatessliceaccelerationandusesk-spacedata
fromallshotsandtheirconjugatesymmetriccounterpartstosolveforacommonmagnitudeimage.We
detailtheindividualstepsnext.
SMS-MUSSELSFormalism
ThefirststepofSMS-NEATRisbasedonaMUSSELSreconstruction,wheretheinputistheacquiredmulti-
shot k-space data, and the output is an estimate of shot imageswhich are further refined in the later
steps.Tobeginwith,wewill ignoreSMSencodingandconsideronly in-planeacceleration. In this case,
MUSSELSentailsthesolutionofthefollowingoptimizationproblem:
𝑚𝑖𝑛𝒙 𝐹&𝐶𝑥& − 𝑑& ++
,-
&./+ 𝜆 ℋ(𝒙) ∗
Eq1
where𝐹& represents theundersampleddiscreteFourier transform(DFT)corresponding toshot𝑡,𝐶 arethecoilsensitivities,𝑥& istheunknowncomplex-valuedimageinshot𝑡withsize𝑁/×𝑁+,and𝑑&aretheacquiredk-spacedatainthisshot.Theterm 𝐹&𝐶𝑥& − 𝑑& +
+thusrepresentsourdataconsistencythrough
sensitivityencoding(25).Theoperatorℋ(∙)firstappliestheDFT,andthenextracts𝑟×𝑟×𝑁;patchesink-spacetogenerateadatamatrixℋ(𝒙)withblock-wiseHankelstructure(11,26–28).Thisoperatoractsona 3-dimensional data structure𝒙 of size𝑁/×𝑁+×𝑁;, which is formed by concatenating the images𝑥&fromall𝑁;shotstogether.Thenuclearnormconstraint ℋ(𝒙) ∗thusenforcesa low-rankprioronthe
Figure1 SMS-NEATR is a combinedmachine learningandphysics-based reconstruction technique for highly-acceleratedmsEPIacquisition.We developed SMS-MUSSELS algorithm to provide an initial solution, whichmay suffer from artifacts due to highacceleration(RinplanexMB=8x2with2-shots).Startingfromthis,residuallearningwithU-Netarchitectureprovidesaninterimimagewithminimalartifacts.Giventhissolution,phasecyclingalgorithm isusedforestimatingshot-phases,whicharethenutilizedassensitivityvariationsinafinaljointvirtualcoil(JVC)SENSEreconstruction.
6
block-Hankelrepresentationofthemultishotdataink-space.ThispriorissimilartotheSAKEformulation
(28), albeit with two differences: the coil axis is now replaced by the shot dimension, and sensitivity
encodingisexplicitlyexploited.Assuch,wefollowtheSAKEapproachandpursueasimple,POCS-SENSE
likealgorithm(29)tosolveEq1asdetailedintheAppendix.WewillshowthattheadvancedFISTAupdate
rules(30) improveconvergenceandimagequality.Wealsonotethatthecostfunctionbeingminimized
differsfromtheconvexoptimizationproblemsetbytheoriginalMUSSELSapproach,andismoresimilar
toLORAKS-typeapproaches(31)astheysolveanon-convexproblem.
ExtensiontoSMS:WedevelopedanewapproachtoallowMUSSELStoworkwithSMSencodingusingthe
readout-extendedFOVconcept(17).ThisrepresentsSMSasundersamplinginthekxaxisbyconcatenating
thetwoslicesalongthereadout(Fig.1a).In-planeandsliceaccelerationcouldthusbecapturedusingthe
Fourieroperator𝐹&inEq1,nowwithsimultaneouskx-kyundersampling.
InthenextstepoftheSMS-NEATRreconstruction,weusetheestimatedshotimages 𝑥& &./,- asinput
to a residual CNN (32,33) with U-Net architecture (23). The network aims to learn and mitigate the
reconstructionerrorsinSMS-MUSSELSandprovideoutputshotimages, 𝑢& &./,- ,withminimalartifacts.
NetworkArchitecture
Weusedapatch-basedU-Nettolearnthemappingbetweentheinitialreconstructionanditsdifference
tothegroundtruth image inaslice-by-slicemanner.Thenetworkconsistedof5 levels (Fig.2),andthe
numberofconvolutionalfilterswas64atthehighestlevel.Asthesizeofinputwasreduced2-foldbymax
poolinginthenext level,thenumberoffilterswasincreased2-foldtoretainthetotalnumberofkernel
weightsineachlevel.Thekernelshadsize3x3,andeachdropoutlayersetarandomlyselected5%ofits
input units to zero to help avoid overfitting (34). Leaky ReLUwas selected as the nonlinear activation
function (35). Batch normalization (BN) was utilized to help accelerate training and avoid saturating
nonlinearities(36).
Figure2U-Netarchitectureisusedtolearnthemappingbetweenpatchesofshot-imagesreconstructedwithSMS-MUSSELS,andtheir difference to reference data. Both the input and output have been decomposed into real and imaginary components toenablecomplex-valuedprocessing forSAGEreconstruction.64×64patchesfromall theshotsarepresentedasinputtoa5-levelnetwork,wherethefirstleveluses64convolutionalfilters.Tohelpprovidescaleinvariance,maxpoolingoperatorsdownsamplethepatchesaftereachlayer.Atthesametime,thenumberoffiltersaredoubledtoretainthetotalnumberofkernelweightsateachlevel.
7
ForSAGEreconstruction,wetrainedacomplex-valuednetworkbyseparatingeachofthe2-shotimages
into its real and imaginary components. Thiswayweused4 channels, ℜ 𝑥/ , ℑ 𝑥/ , ℜ 𝑥+ , ℑ(𝑥+) , in
this network configuration (Fig. 2). For DWI, we explored using both complex-valued and magnitude-
basednetworks.Thecomplexnetworkmadeuseof10channels(realandimaginarycomponentsfrom5-
shots)whiletheabsolutevalueofeachofthe5-shots, 𝑥/ , … , 𝑥A wasusedasinputchannelsforthe
magnitude-basedmodel.Theremainderofthemanuscriptfocusesonthemagnitude-basedDWInetwork,
andthecomplex-valuedU-NetresultsarereportedinSupportingInformationFigS9andS10.
NetworkImplementation
Kerasprogramming interface (37)withTensorflow (38)backendwerechosen toperform the training.
ADAMoptimizer(39)wasusedwithlearningrate=0.001anddecay=0.001.Whilelearningrateactsas
agradientdescentstepsize,decayparameterdampensthisstepsizetotakeprogressivelysmallersteps
ineachepoch.Anℓ+lossfunctionwasminimizedusing200epochsandabatchsizeof128.AnNVIDIA
TitanXPgraphicscardwith12GBmemorywasusedfortraining,whichtook~18hoursforSAGEand~19
hoursforDWI.
ForSAGEprocessing,slicesandechoesweretreatedasdifferenttraininginstances.Attheteststage,
thepatch-basednetworkwasappliedinasliding-windowmannerwithastepsizeof10voxels,andthe
estimated residuals from overlapping patches were averaged together. This process took 4.5
seconds/slice.SimilarlyforDWI,slicesanddiffusiondirectionsweretreatedasdifferenttrainingsamples
andtheinferencetook9seconds/slice.
Phasecycling
WeformamagnitudeestimatefromtheU-Netshotimagesbyaveraging,𝑚CDEF =/,-
𝑢&,-&./ .Wekeep
this improved magnitude fixed, and solve only for the image phase of each EPI-shot𝜙& using phase-regularizedparallelimaging,orphasecycling(24):
𝑚𝑖𝑛IJ 𝐹&𝐶𝑚CDEF ∙ 𝑒LIJ − 𝑑& +++ 𝛼 𝑊𝜙& / Eq2
Here,only thehighlightedphase information𝜙& isunknown,𝑊 is awaveletoperator that imposes
sparsity prior on the shot phase via ℓ/ penalty, and 𝛼 is a parameter that controls the degree of
regularization. The solution of this problem is made easier by the fact that we are using sensitivity
encoding to solve only for the real-valued individual shot phases rather than the complex-valued
individual shot images. The shot phases from the complex-valued SAGE network,∢𝑢&, were used toinitializethisnon-convexproblem.ForDWIreconstructionwithmagnitude-basedU-Netprocessing,shot
phases from SMS-MUSSELS reconstruction, ∢𝑥&, served as initial guess. The complex-valued DWI
network was still able to provide shot phase information,∢𝑢&, to initialize phase cycling (SupportingInformation Figs S9 and S10). SMS acceleration is again embedded in Eq2 via the 2-dimensional
undersampling in𝐹&, and the coil sensitivities of the slices concatenated in the readout direction as
8
representedby𝐶.
JVC-SENSE
Givenestimatesofshot-to-shotphasevariations𝜙&,wecannowjointlysolveforthecommonmagnitude
image𝑚 using the data from all shots, through harnessing sensitivity encoding for slice and in-plane
acceleration(25)andthevirtualcoil(VC)concept(19,20)inJVC-SENSE.Todothis,wesolveasimpleleast
squaresproblem:
𝑚𝑖𝑛P𝐹&𝐶𝑒LIJ
𝐹Q&𝐶∗𝑒QLIJ𝑚 −
𝑑&𝑑Q&∗ +
+,-
&./+ 𝛽 ∙ ℛ(𝑚)
Eq3
Here,theonlyunknownisthehighlightedmagnitudeimage𝑚,andthecoilsensitivitiesaremodified
toincludethephasevariationineachshottoyieldthecombinedsensitivities𝐶𝑒LIJ .TheVCconceptisenforced by augmenting the optimization with the conjugate symmetric k-space data 𝑑Q&∗ and the
conjugatesensitivities𝐶∗𝑒QLIJ .Conjugatesymmetrick-spaceisderivedfromtheacquiredk-spacedata
bycomplex-conjugationandflippingtheaxesinthekx-kyplane.Wehaveusedtheshorthandnotation– 𝑡toexpressthismirroringoperationink-space.Jointreconstructionacrossallshotsisperformedviathe
summationoperator (∙),-&./ .ForstructuralimagingwithSAGE,wehaveusedtotalvariation(TV)penalty
as the regularizerℛ(∙),with the corresponding regularization parameter𝛽. For diffusion imaging,we
haveexploredusingTVregularizationaswellasasimpleTikhonovpenalty.
DuringthereviewofthispaperandafterpublicdisseminationofSMS-NEATRpreprint(40),abstract(41)andcode(https://bit.ly/2QgBg9U)whichintroducedSMS-MUSSELSanditsimprovementusingdeeplearning,twopreprintshavealsoappearedthatdescribeSMS-MUSSELSandanalternativedeeplearningenhancedmsEPIreconstruction(42,43).
TrainingData
Spin-and-gradient-echo(SAGE)
In compliancewith Institutional ReviewBoard (IRB) requirements, three volunteerswere scannedon a
SiemensPrisma3TsystemwithSAGE(22)msEPIsequencetobuildatrainingdataset.Multishotdatawere
collected, where each shot was acquired at Rinplane=8-fold acceleration, and a total of 8-shots were
collected with a ∆ky sampling shift between the shots. When combined, this corresponded to a fully-
encoded acquisition at Rnet=1. Relevant parameters were: field of view (FOV) = 220x220x120 mm3,
resolution = 1x1x3mm3, echo times (TEs) = 26/61/95/130/165ms, repetition time (TR) = 8.3 sec, and
effectiveechospacing=0.148ms.Eachshotsampledonly27phaseencodinglinesduetoRinplane=8-fold
acceleration. First twoechoeswere sampledbefore the180˚pulse, and the latter threewereacquired
aftertherefocusingpulse.Thefifthechowastimedsothatitwasaspinechoimage.
9
Coil sensitivities used in reconstructions were estimated using ESPIRiT (44,45) based on a FLEET
acquisition (46). FLEETautocalibration signal (ACS) acquisition collectsmultishot gradient echoEPIdata
withlowflipangles,wherealltheshotsforaspecificsliceareacquiredfirst.Thenallshotsforthesecond
slice are sampled, and this is repeateduntil every slice in the FOVprescription are accounted for. This
way,encodingofeachsliceiscompletedwithinatimeframeontheorderof100msec,andshot-to-shot
variations are minimized. Unlike the FLEET calibration scan, the “standard mode” for ACS acquisition
wouldsamplethe1stshotforalltheslicesfirst,thenacquirethe2ndshotagainforallslicepositions.The
time frame for samplingall the shots is thuson theorder seconds,which increases thevulnerability to
shot-to-shotmotionandhasdetrimentalimpactontheACSdataquality(46).UsingFLEETacquisitionhas
thusallowedus to improve the robustnessofourcoil sensitivityestimation.Allacquisitionsweremade
withaSiemens32-channelheadcoil.
To obtain clean reference data, MUSSELS reconstruction was performed using all of the 8-shots at
Rinplane=8 acceleration,which yielded “fully-sampled” ground-truth images. To enable higher acquisition
efficiency,only2-shotsoutofthe8-shotdatawereselectedforsubsampledreconstructions.The2-shots
were acquiredwith a k-space shift of ∆ky=4 samples to provide complementary coverage. Thesewere
furthercollapsedintheslicedirectiontosimulateMB=2-foldacceleration,sothatthetotalacceleration
factorpershotbecameRinplanexMB=8x2.ThishighlyundersampledmsEPIdatawerethenreconstructed
using SMS-MUSSELS. Due to the very high acceleration rates, SMS-MUSSELS algorithm incurred
reconstructionartifacts.Theseerrorswithrespecttothecleanreferenceimagewereusedasthetraining
targetinourresiduallearningapproach(Fig.2).
Weextracted57600overlappingpatchesofsize64×64withastepsizeof16voxelsfromthetraining
data.The2-shotsdecomposedintorealandimaginarycomponentsintheSAGEacquisitionweretreated
as input channels, andwere concatenated to create64×64×4patches thatwere fed to thenetwork to
enable joint reconstruction across shots. The training dataset was enriched by 16-fold using
augmentations including scaling (0.5×, 1×, 2×), flipping the axes (left-right, anterior-posterior and echo
dimension)androtations(±135,±90,±45degrees).
DiffusionWeightedImaging(DWI)
Threevolunteerswerescannedona3TPrismasystemtobuildupaDWItrainingdataset,consistingof9-
shotdataacquiredatRinplane=9-foldacceleration.Theparameterswere:fieldofview(FOV)=224x224x120
mm3, resolution = 1x1x3mm3, TE/TR = 54/5100 ms, and effective echo spacing = 0.13 ms. Each shot
sampledonly24phaseencoding linesdue toRinplane=9-foldacceleration. Inaddition toab=0 image, six
diffusiondirectionsatb=1000s/mm2werecollected.FLEETcalibrationdatawereusedtoestimateESPIRiT
coilsensitivitymaps.
Reference “fully-sampled” images were obtained using all 9-shots in MUSSELS reconstruction.
Subsampledacquisitionswereobtainedbyselecting5-shotsoutofthis9-shotdataset.The5-shotswere
shifted by ∆ky={0,2,4,6,8} samples to provide complementary information. These were further
10
undersampled by collapsing two slices that are 60mm apart to simulateMB=2 slice acceleration. The
highly subsampled diffusion msEPI data (RinplanexMB=9x2) were reconstructed using SMS-MUSSELS.
Reconstruction errors with respect to the “fully-sampled” reference data were learned using a deep
network.Similarpatchextractionandaugmentationstepswereperformed.
ReconstructionExperiments
SMS-MUSSELSparameteroptimization:Weexploredthedependenceofthereconstructionperformance
ofSMS-MUSSELSonthek-spacewindowsize𝑟,aswellastherankconstraintenforcedbythenumberof
singularvalues,𝑘.FortheSAGEdataset,weevaluatedtheRMSEmetriconaslicegroupfromatraining
subject,andconsideredarangeofwindowsizes𝑟 ∈ {2,3,4,5,6,7}.Tocontroltherankofthedatamatrix
ℋ(𝒙) which has𝑁;×𝑟×𝑟 columns in an intuitive manner, we varied the “effective number of shots
(𝑁_``)” between 𝑁_`` ∈ {0.75, 1, 1.25, 1.5}. For instance, using a window size 𝑟 = 6 and enforcing𝑁_`` = 1.25 would imply that the number of singular values 𝑘 = 𝑁_``×𝑟×𝑟 = 45 is used during thereconstruction.Thisway,𝑁_``givesusahandleontherankconstraintintermsoftheeffectivenumber
of shotswe allow themsEPI data to have. The optimal parameter setting turned out to be 𝑟 = 5 and𝑁_`` = 1forthe2-shotSAGEreconstructionatRinplanexMB=8x2.Terminationcriterionwaslessthan0.1%
update between image estimates from successive iterations. This analysis is presented in Supporting
InformationFigureS1.
UsingDWIdatasetfromatrainingsubject,asimilaranalysisrevealedthattheoptimalparametersetting
is 𝑟 = 7 and𝑁_`` = 1.25 for a 5-shot reconstruction at RinplanexMB=9x2. Best RMSE was obtained by
terminatingtheSMS-MUSSELSiterationswhentheupdateintheimageestimatebetweeniterationswas
lessthan0.3%.
POCS versus FISTA updates: To improve the convergence rate and image quality of our POCS-like
optimization algorithm for SMS-MUSSELS,wehaveexplored FISTAupdate rule,whichmakesuseof a
combinationof the current andprevious iterates to form thenext image estimate (as detailed in the
Appendix).FISTAwaspreviouslyusedinthecontextofdiffusionmsEPIwithlocallowrankconstraintin
image-space (47). A comparison on one of the SAGE training datasets indicated that FISTA provided
substantial reduction of aliasing/ghosting artifacts as well as RMSE improvement, when other
parameterswereheld constant (𝑟 = 5 and𝑁_`` = 1).As such,wehaveusedFISTA iterations for theremaining SMS-MUSSELS reconstructions reported herein. Convergence analysis is provided in
SupportingInformationFigS11.
11
SAGEreconstruction@RinplanexMB=8x2with2-shots:msEPISAGEdatawereacquiredona4thsubject(not
seen during the training of the network). This acquisition was then reconstructed with SMS-MUSSELS
usingFISTA iterationsandtheoptimizedparametersetting inMatlab,runningonaworkstationwith64
CPUprocessorsand256GBmemory.
SMS-MUSSELS shot images were then processed with the trained U-Net, which allowed improved
estimation of shot-phases using phase cycling. We used 500 iterations and “db4” wavelets in phase
cycling,andsettheregularizationparameterto𝛼=10-5foroptimalRMSE.Havingestimatedthephaseof
each shot, JVC-SENSEwith total variation penalty (𝛽=3·10-4)was used to compute the finalmagnitude
image. For all the remaining experiments, we used these reported parameter values without further
optimization.
SAGEreconstructionwithoutML:ToassessthecontributionoftheMLsteptoSMS-NEATR,weperformed
anadditionalreconstructionwithoutU-Netprocessing.Weusedthesameundersamplingsetupfromthe
first experiment, namely RinplanexMB=8x2 acceleration with 2-shots. Starting from the SMS-MUSSELS
magnitudeestimate𝑚dCeeEfe,weemployedphasecyclingtosolve:
𝑚𝑖𝑛IJ 𝐹&𝐶𝑚dCeeEfe ∙ 𝑒LIJ − 𝑑& +++ 𝛼 𝑊𝜙& / Eq4
Havingobtainedrefinedshot-to-shotphaseestimates𝜙&,wewentontouseJVC-SENSEandarriveatarefinedmagnitude solution. Thisway,we followed the flowchart outlined in Fig. 1, except thatwe by-
passedtheU-Netprocessingstep.
Figure 3RinplanexMB=8x2-fold accelerated SAGE msEPI acquisition with 2-shotsfromatrainingdataset.OneSMSslicegroupandtwoechoesoutofatotaloffivearedepicted.UsingaPOCS-likesolverforSMS-MUSSELSoptimizationledtoresidual aliasing/ghosting artifacts (arrows). FISTA update rules improvedconvergence and image quality of SMS-MUSSELS, and mitigated thesestructurederrors
12
SAGE reconstructionusingBM3D instead ofU-Net:Wehave also explored replacing the deepnetwork
withaconventionaldenoiser,BM3D(48),tohelpimprovetheSMS-MUSSELSoutput.Afterdecomposing
theshotimagesintorealandimaginarycomponents,weprocessedeachoftheseimagesseparately,and
normalizedtheirintensitytobewithin[0,1].WeoptimizedtheBM3Dfilterwidth𝜎forthebestRMSE.The
resulting shot imageswereused to initializephase-cyclingand JVC-SENSE reconstructions, i.e.weagain
followedtheflowchartinFig.1,butreplacedU-NetwithBM3D.
T2andT2*parameterfittingusingSAGEdata:ThefiveechoimagesproducedbySMS-MUSSELSandSMS-
NEATRalgorithmswereusedinaBloch-equationbasedmodelfit(22)toestimateT2andT2*parameter
maps. As supplementary information, we have also explored parameter fitting to the reconstructions
obtainedfromU-NetandBM3Ddenoisers.
DWI reconstruction@ RinplanexMB=9x2with 5-shots: DWI data at six directions were acquired on a 4th
subject (not seenduring the training). Twoof thesedirectionswere reconstructedwith SMS-MUSSELS.
Theseimageswererefinedusingthedeepdiffusionnetwork,andwereprocessedwithphase-cyclingand
JVC-SENSE to compute SMS-NEATR results. In this case, 50 phase-cycling iterations with 𝛼=10-3, andTikhonov regularized JVC-SENSE with 𝛽=10-2 yielded optimal RMSEs. We have also explored BM3D
denoising, again with complex-valued processing for each shot separately, and optimized for the filter
width𝜎.
DWI analysis: Six direction diffusion data reconstructed by the algorithms under comparison were
registeredusingMCFLIRT (49).Diffusiontensor fittingwasperformedusing theDTIFIT function inFSL,
whichalsoproducedfractionalanisotropy(FA)andmeandiffusivity(MD)maps.
RESULTS
SAGE reconstruction @ RinplanexMB=8x2 with 2-shots: The left column of Fig. 4 shows SMS-MUSSELS
reconstructions,whereonlythefirstandlastechoesandroot-sum-of-squareserrormapcalculatedover
the entire 5 echoes are displayed. U-Net processing mitigated some of the noise amplification and
improvedtheRMSEfrom10.8%to8.3%.Startingfromthis,SMS-NEATRwasabletoprovideasmallerror
reduction(8.1%),withsimilarlyhighimagequality.
13
SAGEreconstructionwithoutML:SupportingInformationFigS2demonstratestheeffectofnotusingU-
Netdenoising in theSMS-NEATRpipeline. In thiscase, therewasstill somegain fromrefining theshot-
phaseestimatesusingphase-cyclingandjointparallelimagingreconstruction(RMSEwentfrom10.8%to
9.2%),buttheimprovementoverSMS-MUSSELSwasyethigherwhenMLwasincluded(8.1%).
SAGEreconstructionusingBM3DinsteadofU-Net:UsingaconventionalBM3Ddenoisercouldstillprovide
RMSEreduction (9.3%)but the learnedU-Netmodelwasmoresuccessful in refining theSMS-MUSSELS
output (8.3%). Supporting Information Fig S3 also explores using BM3D to replace U-Net in the SMS-
NEATRflowchart,whichappearedtobeslightlylesseffective(8.4%withBM3Dinitializationversus8.1%
withU-Netjumpstart).
T2 and T2* parameter fitting using SAGE data: Parametermaps from a slice group reconstructed using
SMS-MUSSELSandSMS-NEATRaredepictedinFig.4,correspondingtoan8.3secacquisitionwithwhole-
braincoverageat1x1x3mm3resolution.SMS-NEATRwasabletomitigatenoiseamplificationand image
artifactsmainlyaffectingthemiddleoftheFOVintheSMS-MUSSELSmaps.T2andT2*fitsafterBM3Dand
Figure4SAGEtestdatasetatRinplanexMB=8x2-foldaccelerationusing2-shots.ThefirstandlastechoesareshownforasingleSMSslice group. SMS-MUSSELS with FISTA (left) was successful in reconstructing images despite the high acceleration with 10.8%error.Thebottomrowshowsroot-sum-of-squarescombinationoferrorimagesacrossthefiveechoes.U-NetdenoisingofSMS-MUSSELS reconstruction provided improvement (8.3%,middle), andwas used for initializing SMS-NEATR for additional qualitygain(8.1%,right).
14
U-NetdenoisingarecomparedinSupportingInformationFigS4.U-Netestimatesagainappearedtohave
higherqualitythantheBM3Dresults,andweresimilartotheSMS-NEATRmaps.
Supporting Information Figs S12 and S13 show parameter maps from fully-sampled MUSSELS
reconstructionaswellaserrormapsfromtheacceleratedreconstructions.
DWI reconstruction@RinplanexMB=9x2with 5-shots: Fig. 6 showsDWI slice groups fromonediffusion
direction.TheselowersliceswithpoorB0uniformitywereselectedtodemonstratetheeffectofhighin-
planeaccelerationinavoidingdistortionandvoxelpile-upartifacts.Regardless,SMS-MUSSELSdidsuffer
fromresidualghostingandnoiseamplification in thesedifficult reconstructiontasks.BM3DandU-Net
processinghelpeddenoisethedata,butfailedtoeliminatethestructuredartifacts(indicatedbywhite
arrows).BM3Dresultsappearedover-smooth,whereasU-Netprovidedabettertrade-offbetweenover-
smoothing and denoising. SMS-NEATR did not suffer from over-smoothing, and could mitigate noise
amplification and structured artifacts.We anticipate that RMSE values have contributions from both
noiseandreconstructionerrorsincethegroundtruthdataarealsonoisy.Assuch,RMSEis likelytobe
partiallyindicativeofreconstructionperformance(U-Netconsistentlyhadthebestperformance).
Figure5T2andT2*parametermapsobtainedbyBlochequation fittingto thefive-echo SAGE reconstruction. This 2-shot acquisition at RinplanexMB=8x2-foldacceleration provides whole-brain coverage in 8.3 sec with low geometricdistortion. While SMS-MUSSELS parameter maps appeared noisy (left), theseartifactsweremitigatedintheSMS-NEATRestimates(right).
15
Using complex-valuedormagnitude-basedU-Netprocessing led to similar SMS-NEATR results inDWI
(Supporting Information Fig S10). TV-regularizer could provide further RMSE reduction than Tikhonov
penalty,butthiscameatthecostofsomeover-smoothing(SupportingInformationFigS8).Reducingthe
TV regularization parameter led to comparable RMSEs and image sharpness as ℓ+-penalty using bothcomplex-andmagnitude-valuedU-Netinitialization(SupportingInformationFigS8andS9).
Fig.7showsSMS-NEATRresults fromthesixdirectionacquisition,aswellas theaverageDWI image,
colorFAandMDmaps,andtheroot-sum-of-squarecombinationoferror imagesacrossalldirections.A
similaranalysisispresentedforthefully-sampled,SMS-MUSSELSandU-NetreconstructionsinSupporting
InformationFigsS14–S16.
Figure6AnSMSslicegroupofaseconddiffusiondirectionfromthetestmsEPIacquisitionisshown.SMS-MUSSELSsufferedfromnoiseamplificationandsomestructuredartifacts.BM3DandU-NetcoulddenoisetheSMS-MUSSELSresultatthe potential cost of over-smoothing, while some artifacts persisted (arrows).SMS-NEATR could provide better SNR and image quality without thevulnerabilitytoover-smoothing.
16
DISCUSSION
WepresentedSMS-NEATR,asynergisticMLandphysics-basedreconstructionapproach,thatallowedup
toRnet=8-foldacceleratedmsEPIwithhighimagequality.Thiswasmadepossiblebytakingadvantageof
phase-cycling algorithm, the newly developed SMS-MUSSELS, deep learning, and joint parallel imaging
reconstruction.Our residual CNN learned topredict andmitigate the errors in highly accelerated SMS-
MUSSELS reconstruction,which thenpermittedphase-cycling toestimateshot-to-shotphasevariations.
Including this information as additional sensitivity variations then allowed JVC-SENSE to solve for a
commonmagnitudeimageusingtheentiremultishotk-spacedataandVCconcept.
Partial Fourier sampling was not performed during any of the acquisitions. While this would have
helped achieve shorter TE and higher SNR in DWI, it would not affect the geometric fidelity of the
acquisition.Ourmotivation inemployinghigh in-planeaccelerationrateswas to reducedistortionand
T2*-related blurring, as well as enabling shorter TE. Particularly with the SAGE scan, our aim was to
replace the currently inefficient spin-warp imagingwith themuch faster,msEPI-basedacquisitions for
rapidclinicalimaging,whileminimizingghosting,blurringanddistortionartifactsthatplagueEPI.
CNNs can represent very complicated and non-linear input/output relations.While thismakes them
verypowerful,suchacomplexmappingbetweeninputandoutputcausesthenetworktobedifficultto
characterize.Sinceitsdirectapplicationmayleadtounpredictableerrors,end-to-endCNNreconstruction
inclinical settings is likely to raiseacceptance issues.AML reconstructionapproach thatcanovercome
thisissuewasproposedintheVariationalNetwork(VN)(50)formulation.Thisallowsatransparentdeep
learningreconstructionofacceleratedacquisitionwhereboththekernelweightsandnonlinearactivation
Figure7SMS-NEATRreconstructionforsixdirectiondiffusiondata,aswellasaverageDWI,colorFAandMDmapsandroot-sum-of-squareserroracrossthedirectionsarepresented.
17
functionsarelearntandcanbevisualizedatanylayer.VNalsoutilizessensitivityencodingandenforces
consistencytotheacquiredk-spacedata.Similarly,modelbaseddeeplearning(MoDL) ispowerful in its
ability to combine data consistency and convolutional layers (51). These ideas treat the iterations in
gradient-descent type reconstructions as unrolled networks to retain fidelity to acquired data via a
forwardmodel,whilelearningmodelparametersthatmapthereconstructiontoareferenceimage(52).
Importantly, such combination of a forward-model and learned filtering provided further improvement
thanamodel-basedreconstructionfollowedbyU-Netdenoising(53).
SMS-NEATRalsotaps intothepotentialofCNNwithout treating itasanend-to-endtool,while fully
harnessingtheencodingprovidedbythescannerhardware.WeachievedthisbyusingCNNtoobtainan
interim imagewithminimalartifacts,whileutilizinga rigorousphysics-basedapproach tovalidateand
improveuponthissolutioninthefinalstepofreconstruction.OurgoalinSMS-NEATRistocaptureshot-
to-shotphasevariationsaccurately,sincewhentheyareknown,aJVC-SENSEreconstructionthatsolves
forthemagnitudeimageiscapableofoutperformingalternativeapproaches.Synergisticcombinationof
MLandphysics-basedreconstructionprovedtobepowerful,leadingto~30%RMSEreductionoverour
SMS-MUSSELS implementation (Figs. 4 and 6). In the absence of deep learning initialization, the
subsequentphase cyclingand JVC-SENSE stepsprovideda smaller, <20% improvementover the SMS-
MUSSELSreconstruction(SupportingInformationFig.S2).AconventionalBM3Ddenoiseralsoprovedto
beeffective in jumpstartingSMS-NEATR,but theperformancewas consistentlybetterusinga learned
denoiser tailored for the specific application (Figs. 6, Supporting Information Figs. S3 and S4). We
anticipatefurthergainsfromadvancedmodelsthatcouldsimultaneouslyenforcedataconsistencyand
perform learned filtering (54–56). Thiswouldalso streamline theSMS-NEATRpipelineand reduce the
numberofsteps.
ApplicationofmsEPI instructural imagingismadedifficultbyghostingartifactsfromhard-to-estimate
physiologicsignalchangesbetweenshots.ThisisparticularlytrueforgradientechoimagingatlateTEs.To
illustrate,we performed a “slidingwindow” combination of 8-shots of SAGE data acquired at Rinplane=8
acceleration toobtain “fully-sampled”data (Supporting Information Fig. S5). The ghosting artifacts that
stemfromphysiologicalnoiseisespeciallystronginthe2ndand3rdechoesduetoincreasedphaseaccrual
atlongTEs(thelastechoisinfactaspinecho,whichrefocusesmostofthephaseevolutionandresultsin
the cleanest image). Using a standard forward-model based reconstruction for structuralmsEPI would
thusnecessitatethesimultaneousestimationoftheimagecontentandthephasevariationsineachshot.
Since both the clean image and the phase information used in the forward-model are unknown, this
wouldentail the solutionof a computationallyprohibitive,non-convexoptimizationproblem that could
get stuck at localminima. As such, existingmsEPI techniques circumvent this difficulty by dividing the
reconstruction into two separate parts: shot-phase estimation, and combination ofmultishot given the
estimatedphaseinformation.Navigator-basedapproachesderivethisphaseinformationfromadditional
calibration acquisitions made for each shot (3–7). Diffusion imaging with MUSE and its extensions
(9,12,13)operatewithoutanavigator, andperform thephaseestimation stepusingparallel imaging to
18
reconstructacompleximageforeachshot.Smoothingthephaseofeachintermediateimagethenyields
anestimateofshot-to-shotvariations,whichallowsjointreconstructionofallmultishotdatatogether.
msEPI reconstruction is indeed harder in diffusion imaging, since the phase variations amplified by
diffusion gradients can be much stronger than the physiologic noise in structural imaging, as
demonstrated in the “sliding window” combination in Supporting Information Fig. S6. The final
Supporting InformationFig.S7showsasimilarslidingwindowdatacombination,butthistimewithout
any diffusion gradients (b=0). Even in this case where one would not expect any artifacts, there are
minor ghosts that may be stemming from patient motion. Given that the TR was 5.1sec, msEPI
acquisitionsare indeedsusceptible tomotionartifacts since it took~46sec tosample these9-shots.A
sidebenefitofhighlyacceleratedmsEPIcouldbeanimprovementinmotionrobustness.Wehaveseen
similar gains in the final SMS-NEATR diffusion reconstructions using either magnitude- or complex-
valueddeeplearning.Weexpectthiswasbecausethemagnitudenetworkcouldprovidehigherquality
magnitude priors which helped phase-cycling to better solve for the shot-phase data, whereas the
complexnetworkprovidedanoverallgaininbothmagnitudeandphaseestimates–butthemagnitude
outputwasimprovedtoalesserextent(ascanbeseenintheRMSEvaluesinSupportingInformationFig
S10).HavingobtainedsimilarSMS-NEATRresultsmayindicatethatthereisflexibilityintheblocksinthe
pipeline,aslongastheshot-phaseestimatesareimprovedbeyondthoseofSMS-MUSSELS.
MUSSELS exploits similarities between the shot-images using a low-rank prior on the block-Hankel
representationoftheirk-space(11,14),sothatitcanperformmsEPIreconstructionwithoutexplicitshot-
phase estimation. MUSSELS has allowed Rinplane=8-fold acceleration per each shot in msEPI diffusion
imagingusingasfewas4-shots(Rnet=2).Unlikeearliernavigator-based(5–7)ornavigator-freeapproaches
(8–10) where the number of acquired shots was equal to the in-plane acceleration factor (𝑁;=Rinplane),
MUSSELS could thus perform in the (𝑁;<Rinplane) regime to improve acquisition efficiency. With SMS-
NEATR,wepushedtheefficiencygainevenfurthertoenableRnet=8-foldacceleration(RinplanexMB=8x2with
2-shots) in structural imaging, and Rnet=3.6-fold (RinplanexMB=9x2 with 5-shots) in diffusion imaging.
AlthoughSMS-MUSSELShadsomeresidualartifactsatsuchhighaccelerations, itprovidedagood initial
guess for our residual network to further clean up the shot-images. Starting from these estimates,we
couldthensolveforthephasevariationsusingphasecycling,whichconstitutedaneasierproblemsince
theunknowninformationwasareal-valuedphaseimage.Thisprovideda2-foldreductioninthenumber
ofunknownscomparedtoacomplex-valuedSENSEsolution.
Thebest RMSEperformance for structural imagingwasobtainedwith𝑟×𝑟 = 5×5windows andwith𝑁_``=1,whereas the optimal parameterswere𝑟×𝑟 = 7×7windows andwith𝑁_``=1.25 for DWI. The
increasedwindowsizeandrankconstraintshouldhelpcapturegreatershot-to-shotvariations,whichare
morelikelytobeobservedindiffusionimagingthantheSAGEscan.Furtherrelaxingtherankconstraint
and using larger windows could help representmore spatially varying phases differences between the
shots, but relaxing thesepriorsbeyond their optimal valueswould comeat thepotential costofRMSE
19
performance. Indeed, using𝑁_`` = 𝑁; would be a non-informative prior and the outcome would be
identicaltoashot-by-shotSENSEreconstruction.
Limitations and theirmitigation: SMS-NEATRusesML to provide an initial estimate to a difficult image
reconstruction problem, thereby avoiding the vulnerability of poor generalization of “direct” ML
reconstruction.Inadditiontothis,wehaveaugmentedourtrainingdatasetsizeby16-foldtosubjectthe
networktogreatervariation.Usingapatch-wiserepresentationwithoverlappingpatcheshelpedfurther
increasetheavailablenumberof trainingsamples.Finally,wehaveprovidedthenetworkwithdifferent
contrasts (echoes or diffusion directions) as training samples to help improve generalization. Despite
theseprecautions, thenetworkwouldbenefit fromre-training if largechanges in sequenceparameters
are desired to be made, or if they are dictated by hardware limitations of other scanners. We also
anticipate that having the subsequent physics-based reconstruction will mitigate some of the
generalizationconcerns,astheMLoutputisusedforinitializingthismodelbasedstep.Exploringunrolled
networks with data consistency layers (50–53,55) or using conventional denoisers could provide
additional robustness.Using smoothness priors embedded in theMUSSELS reconstructionwith the SR-
MUSSELS formalism rather than relyingon learnedor conventional denoiserswouldalsobeanelegant
solution.
Anotherconsiderationistheselectionofthereconstructiontechniquethatprovidestheinitialsolution
toU-Net.WehavedevelopedaFISTA-basedsolverforSMS-MUSSELS,butotheradvancedreconstruction
strategiessuchasMUSE(9)orPOCS-MUSE(10)couldalsobeutilizedtoprovidethisinitialestimate.
Qualitatively, SMS-NEATR provided greater gains in the more challenging multishot DWI
reconstruction than the SAGE application. It has bettermitigated ghosting/aliasing artifacts and noise
amplification than SMS-MUSSELS, but the RMSE metrics remained above 20%. We think that this is
becausethegroundtruthdiffusiondataisalsocorruptedbynoise,whichmakesitdifficulttodisentangle
reconstruction artifacts from the noise contribution. As such, othermeasures of fidelity to reference
datacouldbettergaugetheimprovementinDWIreconstruction.
For Nyquist ghost correction, we have used a simple 1-dimensional navigator in a slice-specificmanner. Especially in oblique acquisitions, more involved ghost correction techniques such as DualPolarityGrappa(57)andLORAKS(31)shouldallowforimprovedsuppressionoftheseerrors.Toensurethat the residual ghosts seen in presented results stem only from reconstruction errors due toacceleration,wehave includedfully-sampledMUSSELSreconstructionsandFLEETcalibrationdatathatdonotexhibitghostingintheSupportingInformationFigsS17andS18.
Extensions:WehavedemonstratedtheapplicationsofSMS-NEATRinmsEPISAGEandDWIacquisitions.
EnablingRinplane=8or9-foldaccelerationinotherpulsesequencescouldhelpcreateamulti-contrastmsEPI
clinicalprotocolwithhighgeometricfidelity.Thiswouldminimizethedistortionandblurringartifactsthat
hamperimagequalityandachievableresolutionintherecentlydevelopedsingleshotEPIprotocols(1,2).
Employing msEPI readout in multi-inversion T1 mapping (58,59) and FLAIR (60) acquisitions with SMS-
NEATRreconstructioncouldenablearapidMRexamwithsimilartabletimeasaCTscan.Otheradvanced
20
encoding strategies such as wave-EPI (61) could provide additional efficiency gain and/or in-plane
accelerationcapability.
WebelievethatthestrategyofutilizingMLtoestimateunknownnuisanceparametersinphysics-based
forwardmodelreconstructionscanbeimpactfulinsolvingotherprohibitivelydifficultproblems.Wehave
recentlydemonstratedthisconceptinprospectivemotioncorrection(62),whereweusedresidualdeep
learning to provide an interim image with largely reduced motion artifacts. This interim CNN
reconstructionprovides an initial image andmotionparameter estimate thus jumpstarting thephysics-
based TAMER algorithm (63), which uses the extra degrees of freedom in multi-coil data to jointly
estimatemotionparametersandtheclean image.Havingaccesstoagoodinitialguesshelpedthenon-
convexTAMERoptimizationconverge30× faster to the final solution. Othervenues thatmightbenefit
from this synergistic approach couldbe innavigator-freeNyquist (N/2) ghost correction, calibrationless
parallelimagingandreference-freek-spacetrajectoryestimation.
CONCLUSION
WedemonstratedtheabilityofSMS-NEATR,acombinedMLandphysics-basedreconstructionalgorithm,
inprovidinghighqualityreconstructionsfromupto8-foldacceleratedmsEPIacquisitionsusing2–5shots
ofdata.Theabilitytoacquirehighin-planeresolutionimageswithminimaldistortionandblurringcould
enableanmsEPI-basedMRIexamwithmultiplecontrasts,whilematchingthetabletimeofaCTscan.
ACKNOWLEDGMENT
We acknowledge a GPU grant fromNVIDIA, and support fromNIH NINDS (K23 NS096056) NIMH (R24
MH106096andR01MH116173)andNIBIB(U01EB025162,R01EB020613,R01EB019437,R01EB017337
andP41EB015896).AdditionalsupportwasprovidedbytheMGH/HSTAthinoulaA.MartinosCenterfor
Biomedical Imaging. This research was made possible by the resources provided by NIH Shared
InstrumentationGrantsS10-RR023401andS10-RR023043.
APPENDIX
WepursueaPOCS-likesolutiontotheoptimizationproblemposedintheMUSSELSformalism(Eq1),and
followthestepsbelow:
21
𝒚𝟏 = 𝒙𝟎%initialguessfrome.g.SMS-SENSEreconstruction
𝜏/ = 1for𝑖 = 1: 𝑁L&_m
%Low-rankconstraint:
𝐴 = ℋ(𝒚𝒊)𝑈Σ𝑉s = 𝑠𝑣𝑑(𝐴)𝐴 = 𝑈Σv𝑉s %Σvisobtainedviahardthresholdingbykeepingthe𝑘largestsingularvalues𝒙𝒊 = ℋ∗ 𝐴 %ℋ∗isatransposedmappingthatinsertsHankelmatrixelementsintomulti-shotk-space
for𝑡 = 1: 𝑁;𝑥& = 𝒙𝒊(: , : , 𝑡)%Generatecoilimagesbymultiplicationwithsensitivities:
𝑥w = 𝐶𝑥&%Resubstituteacquiredk-space:
𝑥w = 𝑥w + 𝐹s(𝑑& − 𝐹&𝑥w)%Coilcombination:
𝑥& = 𝐶s𝐶 Q/𝐶s𝑥w 𝒙𝒊 : , : , 𝑡 = 𝑥&
end
ifuse_fista
𝜏Lx/ =1 + 1 + 4𝜏L+
2
𝒚𝒊x𝟏 = 𝒙𝒊 +𝜏L − 1𝜏Lx/
𝒙𝒊 − 𝒙𝒊Q𝟏
else
𝒚𝒊x𝟏 = 𝒙𝒊end
end
The flag “use_fista” togglesbetweenconventionalPOCS-likeupdate ruleandFISTA iteration,whichhas
earlieriteratestoformthenextimageestimate.𝑁L&_m denotesthemaximumnumberofiterations,which
wehavetakentobe200.𝒙𝟎 isaninitialguessforthemsEPIimages,andwereestimatedusinganSMS-
SENSEreconstructionforeachoftheshotsindependently.
REFERENCES
1.SkareS,SprengerT,NorbeckO,RydénH,BlombergL,AvventiE,EngströmM.A1-minutefullbrainMRexamusingamulticontrastEPIsequence.Magn.Reson.Med.2017;00:1–10.doi:10.1002/mrm.26974.
22
2.NaelK,KhanR,ChoudharyG,MeshksarA,VillablancaP,TayJ,DrakeK,CoullBM,KidwellCS.Six-minutemagnetic resonance imaging protocol for evaluation of acute ischemic stroke: Pushing the boundaries.Stroke2014;45:1985–1991.doi:10.1161/STROKEAHA.114.005305.
3.PorterDA,HeidemannRM.Highresolutiondiffusion-weightedimagingusingreadout-segmentedecho-planarimaging,parallelimagingandatwo-dimensionalnavigator-basedreacquisition.Magn.Reson.Med.2009;62:468–475.doi:10.1002/mrm.22024.
4.JeongH-K,GoreJC,AndersonAW.High-resolutionhumandiffusiontensorimagingusing2-DnavigatedmultishotSENSEEPIat7T.Magn.Reson.Med.2013;69:793–802.doi:10.1002/mrm.24320.
5. LiuW, ZhaoX,Ma Y, Tang X,Gao J-H.DWIusingnavigated interleavedmultishot EPIwith realignedGRAPPAreconstruction.Magn.Reson.Med.2016;75:280–286.
6.Ma X, Zhang Z, Dai E, Guo H. Improvedmulti-shot diffusion imaging using GRAPPAwith a compactkernel.Neuroimage2016;138:88–99.doi:10.1016/J.NEUROIMAGE.2016.05.079.
7.DongZ,WangF,MaX,ZhangZ,DaiE,YuanC,GuoH. InterleavedEPIdiffusion imagingusingSPIRiT-based reconstruction with virtual coil compression. Magn. Reson. Med. 2018;79:1525–1531. doi:10.1002/mrm.26768.
8.ZhangZ,HuangF,MaX,XieS,GuoH.Self-feedingMUSE:ArobustmethodforhighresolutiondiffusionimagingusinginterleavedEPI.Neuroimage2015;105:552–560.doi:10.1016/J.NEUROIMAGE.2014.10.022.
9.ChenNK,GuidonA,ChangHC,SongAW.Arobustmulti-shotscanstrategyforhigh-resolutiondiffusionweighted MRI enabled by multiplexed sensitivity-encoding (MUSE). Neuroimage 2013;72:41–47. doi:10.1016/J.NEUROIMAGE.2013.01.038.
10. Chu M-L, Chang H-C, Chung H-W, Truong T-K, Bashir MR, Chen N. POCS-based reconstruction ofmultiplexed sensitivity encoded MRI (POCSMUSE): A general algorithm for reducing motion-relatedartifacts.Magn.Reson.Med.2015;74:1336–1348.doi:10.1002/mrm.25527.
11.ManiM,JacobM,KelleyD,MagnottaV.Multi-shotsensitivity-encodeddiffusiondatarecoveryusingstructured low-rank matrix completion (MUSSELS). Magn. Reson. Med. 2017;78:494–507. doi:10.1002/mrm.26382.
12. Zhang Z, Huang F, Ma X, Xie S, Guo H. Self-feeding MUSE: A robust method for high resolutiondiffusion imaging using interleaved EPI. Neuroimage 2015;105:552–560. doi:10.1016/J.NEUROIMAGE.2014.10.022.
13. Chu M-L, Chang H-C, Chung H-W, Truong T-K, Bashir MR, Chen N. POCS-based reconstruction ofmultiplexed sensitivity encoded MRI (POCSMUSE): A general algorithm for reducing motion-relatedartifacts.Magn.Reson.Med.2015;74:1336–1348.doi:10.1002/mrm.25527.
14.ManiM,MagnottaV,KelleyD, JacobM.Comprehensive reconstructionofmulti-shotmulti-channeldiffusion data usingmussels. In: 2016 38thAnnual International Conference of the IEEE Engineering inMedicineandBiologySociety(EMBC).IEEE;2016.pp.1107–1110.doi:10.1109/EMBC.2016.7590897.
15.LiuC,MoseleyME,BammerR.SimultaneousphasecorrectionandSENSEreconstructionfornavigatedmulti-shot DWI with non-cartesian k-space sampling. Magn. Reson. Med. 2005;54:1412–1422. doi:10.1002/mrm.20706.
23
16.PipeJG,FarthingVG,ForbesKP.Multishotdiffusion-weightedFSEusingPROPELLERMRI.Magn.Reson.Med.2002;47:42–52.doi:10.1002/mrm.10014.
17.KoopmansPJ,PoserBA,BreuerFA.2D-SENSE-GRAPPAforFast,Ghosting-RobustReconstructionofIn-Plane and Slice Accelerated Blipped-CAIPI-EPI. In: International Society for Magnetic Resonance inMedicine23rdScientificMeeting,Toronto,Canada.;2015.p.2410.
18.OngF,ChengJ,LustigM.SupportingInformationfor :GeneralPhaseRegularizedReconstructionusingPhaseCycling.2017:1–3.
19. Blaimer M, Gutberlet M, Kellman P, Breuer F, Kostler H, Griswold MA. Virtual coil concept forimprovedparallelMRIemployingconjugatesymmetricsignals.Magn.Reson.Med.2009;61:93–102.
20. Blaimer M, Heim M, Neumann D, Jakob P, Kannengiesser SA, Breuer F. Comparison of phase-constrainedparallelMRIapproaches:Analogiesanddifferences.Magn.Reson.Med.2016;75:1086–1099.
21.BlaimerM,JakobP,BreuerF.Regularizationmethodforphase-constrainedparallelMRI.Magn.Reson.Med.2014;72.1:166–171.
22.SchmiedeskampH,StrakaM,NewbouldRD,ZaharchukG,Andre JB,Olivot J-M,MoseleyME,AlbersGW, Bammer R. Combined spin- and gradient-echo perfusion-weighted imaging. Magn. Reson. Med.2012;68:30–40.doi:10.1002/mrm.23195.
23.RonnebergerO,FischerP,BroxT.U-Net:ConvolutionalNetworksforBiomedicalImageSegmentation.In: Medical image computing and computer-assisted intervention : MICCAI. Springer, Cham; 2015. pp.234–241.doi:10.1007/978-3-319-24574-4_28.
24.OngF,ChengJY,LustigM.Generalphaseregularizedreconstructionusingphasecycling.Magn.Reson.Med.2018;80:112–125.doi:10.1002/mrm.27011.
25. Pruessmann KP,Weiger M, Scheidegger MB, Boesiger P. SENSE: Sensitivity encoding for fast MRI.Magn.Reson.Med.1999;42:952–962.
26.Haldar JP. Low-RankModeling of Local k-SpaceNeighborhoods (LORAKS) for ConstrainedMRI. IEEETrans.Med.Imaging2014;33:668–681.doi:10.1109/TMI.2013.2293974.
27.JinKH,YeJC.Sparse+LowRankDecompositionofAnnihilatingFilter-basedHankelMatrixforImpulseNoiseRemoval.2015.
28. Shin PJ, Larson PEZ, OhligerMA, EladM, Pauly JM, Vigneron DB, LustigM. Calibrationless parallelimaging reconstruction based on structured low-rank matrix completion. Magn. Reson. Med.2014;72:959–970.doi:10.1002/mrm.24997.
29. Samsonov AA, Kholmovski EG, Parker DL, Johnson CR. POCSENSE: POCS-based reconstruction forsensitivity encoded magnetic resonance imaging. Magn. Reson. Med. 2004;52:1397–1406. doi:10.1002/mrm.20285.
30. Beck A, TeboulleM. A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems.Soc.Ind.Appl.Math.2009;2:183–202.doi:10.1137/080716542.
31.LobosRA,KimTH,HogeWS,HaldarJP.Navigator-FreeEPIGhostCorrectionWithStructuredLow-RankMatrix Models: New Theory and Methods. IEEE Trans. Med. Imaging 2018;37:2390–2402. doi:10.1109/TMI.2018.2822053.
24
32.ZhangK,ZuoW,ChenY,MengD,ZhangL.BeyondaGaussiandenoiser:ResiduallearningofdeepCNNforimagedenoising.IEEETrans.ImageProcess.2017;26:3142–3155.doi:10.1109/TIP.2017.2662206.
33.HeK,ZhangX,RenS,SunJ.DeepResidualLearningforImageRecognition.In:ProceedingsoftheIEEEconferenceoncomputervisionandpatternrecognition.;2016.pp.770–778.
34.SrivastavaN,HintonG,KrizhevskyA,SutskeverI,SalakhutdinovR.Dropout:ASimpleWaytoPreventNeuralNetworksfromOverfitting.J.Mach.Learn.Res.2014;15:1929–1958.
35.MaasAL,HannunAY,NgAY.Rectifiernonlinearities improveneuralnetworkacousticmodels. ICMLWork.DeepLearn.AUDIO,SPEECHLang.Process.2013;30:3.
36. Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internalcovariateshift.Int.Conf.Mach.Learn.2015:448–456.
37.CholletF.https://keras.io.2015.
38.AbadiM,AgarwalA,BarhamP, et al. TensorFlow: Large-ScaleMachine LearningonHeterogeneousDistributedSystems.2016.
39.KingmaD,BaJ.Adam:Amethodforstochasticoptimization.arXivPrepr.2014:arXiv:1412.6980.
40.BilgicB,Chatnuntawech I,ManhardMK, TianQ, LiaoC,Cauley SF,HuangSY, Polimeni JR,Wald LL,Setsompop K. Highly Accelerated Multishot EPI through Synergistic Machine Learning and JointReconstruction.2018:ArXiv:1808.02814.
41. Bilgic B, Chatnuntawech I,ManhardM, et al.NEATR-SMS forHighly AcceleratedMulti-Shot EPI. In:ISMRMWorkshoponMachineLearningPart2.;2018.
42. Aggarwal HK, ManiMP, JacobM.Multi-Shot Sensitivity-Encoded DiffusionMRI usingModel-BasedDeepLearning(MODL-MUSSELS).2018:ArXiv:1812.08115.
43.ManiM,JacobM,McKinnonG,YangB,RuttB,KerrA,MagnottaV.SMSMUSSELS:ANavigator-freeReconstruction for Simultaneous MultiSlice Accelerated MultiShot Diffusion Weighted Imaging.2019:ArXiv:1901.01893.
44. Uecker M, Lai P, Murphy MJ, Virtue P, Elad M, Pauly JM, Vasanawala SS, Lustig M. ESPIRiT-aneigenvalue approach to autocalibrating parallelMRI:Where SENSEmeetsGRAPPA.Magn. Reson.Med.2014;71:990–1001.doi:10.1002/mrm.24751.
45. Uecker M, Ong F, Tamir JI, Bahri D, Virtue P, Cheng JY, Zhang T, Lustig M. Berkeley AdvancedReconstructionToolbox.In:ProceedingsoftheInternationalSocietyforMagneticResonanceinMedicine.;2015.p.2486.
46.PolimeniJR,BhatH,WitzelT,BennerT,FeiweierT,InatiSJ,RenvallV,HeberleinK,WaldLL.Reducingsensitivity losses due to respiration and motion in accelerated echo planar imaging by reordering theautocalibrationdataacquisition.Magn.Reson.Med.2016;75:665–679.doi:10.1002/mrm.25628.
47. Hu Y, Levine EG, Tian Q, Moran CJ, Wang X, Taviani V, Vasanawala SS, McNab JA, Daniel BA,Hargreaves BL. Motion-robust reconstruction of multishot diffusion-weighted images without phaseestimationthroughlocallylow-rankregularization.Magn.Reson.Med.2018.doi:10.1002/mrm.27488.
48. Dabov K, Foi A, Katkovnik V, Egiazarian K. Image Denoising by Sparse 3-D Transform-Domain
25
CollaborativeFiltering.IEEETrans.ImageProcess.2007;16:2080–2095.doi:10.1109/TIP.2007.901238.
49. JenkinsonM,BannisterPR,BradyM,SmithSM. ImprovedOptimizationfortheRobustandAccurateLinear Registration and Motion Correction of Brain Images. Neuroimage 2002;17:825–841. doi:10.1006/NIMG.2002.1132.
50. Hammernik K, Klatzer T, Kobler E, RechtMP, Sodickson DK, Pock T, Knoll F. Learning a variationalnetwork for reconstruction of accelerated MRI data. Magn. Reson. Med. 2018;79:3055–3071. doi:10.1002/mrm.26977.
51.AggarwalH,ManiM,JacobM.MoDL:ModelBasedDeepLearningArchitectureforInverseProblems.IEEETrans.Med.Imaging2018:1–1.doi:10.1109/TMI.2018.2865356.
52. Adler J, ÖktemO. Solving ill-posed inverse problems using iterative deep neural networks. InverseProbl.2017;33:124007.doi:10.1088/1361-6420/aa9581.
53.AdlerJ,ÖktemO.LearnedPrimal-dualReconstruction.arXiv:1707.06474[math.OC]2017.
54.LeeJ,HanY,YeJC.k-SpaceDeepLearningforReference-freeEPIGhostCorrection.arxiv:1806.001532018.
55.HanY,YeJC.k-SpaceDeepLearningforAcceleratedMRI.arxiv:1805.037792018.
56.AdlerJ,OktemO.LearnedPrimal-DualReconstruction.IEEETrans.Med.Imaging2018;37:1322–1332.doi:10.1109/TMI.2018.2799231.
57.HogeWS,PolimeniJR.Dual-polarityGRAPPAforsimultaneousreconstructionandghostcorrectionofechoplanarimagingdata.Magn.Reson.Med.2016;76:32–44.doi:10.1002/mrm.25839.
58.RenvallV,WitzelT,WaldLL,PolimeniJR.Automaticcorticalsurfacereconstructionofhigh-resolutionT1echoplanarimagingdata.Neuroimage2016;134:338–354.doi:10.1016/J.NEUROIMAGE.2016.04.004.
59.CohenO,Polimeni JR.Optimized inversion-timeschedules forquantitativeT1measurementsbasedonhigh-resolutionmulti-inversionEPI.Magn.Reson.Med.2017.doi:10.1002/mrm.26889.
60.DeCoeneB,HajnalJV.,GatehouseP,LongmoreD,WhiteSJ,OatridgeA,PennockJM,Young ianR,BydderGM.MRof theBrainUsingFluid-Attenuated InversionRecovery (FLAIR)PulseSequences.Am. J.Neuroradiol.1992;13:1555–1564.
61.PoserBA,BilgicB,GagoskiBA,UludagK,StengerVA,WaldLL,SetsompopK.Echo-planarimagingwithwave-CAIPIacquisitionandreconstruction. In: InternationalSocietyforMagneticResonanceinMedicine25thScientificMeeting,Hawaii.;2017.p.1198.
62.BilgicB,CauleySF,ChatnuntawechI,ManhardMK,WangF,HaskellM,LiaoC,WaldLL,SetsompopK.CombiningMRPhysicsandMachineLearningtoTackleIntractableProblems.In:Proceedingsofthe26thAnnualMeetingISMRM.;2018.
63. Haskell MW, Cauley SF, Wald LL. TArgeted Motion Estimation and Reduction (TAMER): DataConsistencyBasedMotionMitigationforMRIusingaReducedModelJointOptimization.IEEETrans.Med.Imaging2018:1–1.doi:10.1109/TMI.2018.2791482.
26
FIGURECAPTIONS
Fig1.SMS-NEATRisacombinedmachinelearningandphysics-basedreconstructiontechniqueforhighly-
acceleratedmsEPIacquisition.WedevelopedSMS-MUSSELSalgorithmtoprovideaninitialsolution,which
may suffer from artifacts due to high acceleration (RinplanexMB=8x2 with 2-shots). Starting from this,
residual learning with U-Net architecture provides an interim image with minimal artifacts. Given this
solution,phasecyclingalgorithmisusedforestimatingshot-phases,whicharethenutilizedassensitivity
variationsinafinaljointvirtualcoil(JVC)SENSEreconstruction.
Fig 2. U-Net architecture is used to learn themapping between patches of shot-images reconstructed
with SMS-MUSSELS, and their difference to reference data. Both the input and output have been
decomposed into real and imaginary components to enable complex-valued processing for SAGE
reconstruction.64×64patchesfromalltheshotsarepresentedasinputtoa5-levelnetwork,wherethe
first level uses 64 convolutional filters. To help provide scale invariance, max pooling operators
downsamplethepatchesaftereachlayer.Atthesametime,thenumberoffiltersaredoubledtoretain
thetotalnumberofkernelweightsateachlevel.
Fig3.RinplanexMB=8x2-foldacceleratedSAGEmsEPIacquisitionwith2-shots fromatrainingdataset.One
SMS slice group and two echoes out of a total of five are depicted. Using a POCS-like solver for SMS-
MUSSELS optimization led to residual aliasing/ghosting artifacts (arrows). FISTA update rules improved
convergenceandimagequalityofSMS-MUSSELS,andmitigatedthesestructurederrors.
Fig4.SAGE testdatasetatRinplanexMB=8x2-foldaccelerationusing2-shots.The firstand lastechoesare
shown for a single SMS slice group. SMS-MUSSELS with FISTA (left) was successful in reconstructing
images despite the high acceleration with 10.8% error. The bottom row shows root-sum-of-squares
combination of error images across the five echoes. U-Net denoising of SMS-MUSSELS reconstruction
providedimprovement(8.3%,middle),andwasusedforinitializingSMS-NEATRforadditionalqualitygain
(8.1%,right).
Fig5.T2andT2*parametermapsobtainedbyBlochequationfittingtothefive-echoSAGEreconstruction.
This2-shotacquisitionatRinplanexMB=8x2-foldaccelerationprovideswhole-braincoveragein8.3secwith
lowgeometricdistortion.WhileSMS-MUSSELSparametermapsappearednoisy(left),theseartifactswere
mitigatedintheSMS-NEATRestimates(right).
Fig6.DiffusionmsEPIacquisitionatRinplanexMB=9x2accelerationwith5-shotsfromthetestsubject.One
SMSslicegroupisshownforthiswhole-brainacquisition.SMS-MUSSELSsufferedfromaliasing/ghosting
artifactsinthisharderreconstructionproblem.BM3DandU-Netdenoisingcouldmitigatenoise,butthe
structuredartifactspersisted(arrows).SMS-NEATRwasabletofurthermitigatetheseerrorsto improve
imagequality,whileavoidingpotentialover-smoothingBM3DandU-Netmaysufferfrom.
Fig7.SMS-NEATRreconstructionforsixdirectiondiffusiondata,aswellasaverageDWI,colorFAandMD
mapsandroot-sum-of-squareserroracrossthedirectionsarepresented.
27
SUPPORTINGINFORMATIONFIGURECAPTIONS
SupportingInformationFigS1.DependenceofthereconstructionperformanceofSMS-MUSSELSonthek-
spacewindowsize𝑟,andtherankconstraintasrepresentedbytheeffectivenumberofshots(𝑁_``)foraslicegroupfromaSAGEtrainingdataset.Theoptimalparametersettingwas𝑟 = 5and𝑁_`` = 1forthe2-shotSAGEreconstructionatRinplanexMB=8x2acceleration.
SupportingInformationFigS2.SAGEmsEPIreconstructionresultsfromthetestdatasetatRinplanexMB=8x2
accelerationusing2-shots.Thefirstandlastechoesaredisplayedoutofatotaloffiveechoesbelongingto
this SMS slice group. SMS-MUSSELS yielded 10.8% RMSE (left), and was also used to initialize phase-
cycling and JVC-SENSE reconstruction without machine learning (9.2% error, middle). Using U-Net to
refinetheSMS-MUSSELSresultandjumpstartSMS-NEATRprovidedfurtherimprovementat8.1%RMSE.
Supporting Information Fig S3. The SMS-MUSSELS reconstruction for the slice group in Fig S2 was
denoisedusingBM3DandU-Net,wheredeeplearningprovedtobeadvantageous(BM3D:9.3%versusU-
Net: 8.3% error). Utilizing each of these denoised outputs to jumpstart SMS-NEATR led to similar
reconstructions,andU-Netinitializationhadthebestoverallperformance(8.1%RMSE).
Supporting Information Fig S4. Bloch equation based signal modeling for the five echoes in the SAGE
acquisitionallowsforT2andT2*parametermapping.Denoisingthe2-shotSMS-MUSSELSreconstruction
atRinplanexMB=8x2acceleration,corresponding toan8.3secwhole-brainacquisition,usingBM3DandU-
Net led to improvements in thequalityof thesequantitativemaps.U-Netappearedmore successful in
mitigatingthenoiseamplificationinthemiddleoftheFOVthantheconventionalBM3Dfiltering.
SupportingInformationFigS5.Slidingwindow(summationacrossshotsink-space)combinationof8-shots
of SAGE data acquired at Rinplane=8 acceleration per shot. Due to physiological shot-to-shot phase
variations, the combined fully-encoded images exhibit ghosting artifacts. This becomesmore severe at
laterTEs,but ismitigatedat the lastecho,which isa spinecho image.Thebottomrow is scaledup to
betterdemonstratetheartifacts.
Supporting InformationFigS6.Slidingwindowcombinationof9-shotsofDWIdataacquiredatRinplane=9
accelerationpershot.Fiveslicesfromawhole-brainacquisitionaredepicted.Duetoshot-to-shotphase
variations stemming from motion under the diffusion encoding gradients, there are severe ghosting
artifactsintheseotherwisefully-encodedimages.
SupportingInformationFigS7.Slidingwindowcombinationofdatafromthesameacquisitionsessionas
FigS6,but this time thediffusiongradientshavebeenswitchedoff (b=0).Thesespinecho images look
relativelydevoidofghostingartifacts,butthescaled-upimagesinthebottomrowrevealthattheseerrors
persist.We anticipate that headmotion contributed to these artifacts, since it took around 46 sec to
sample9-shotsatTR=5.1sec.
28
Supporting Information Fig S8. Employing TV-regularization in JVC-SENSE led to similar results as L2
penalty.ReconstructionobtainedwiththeTVparametervaluethatyieldedtheoptimalRMSEvalue(𝜆z{ =
10Q+)appearedover-smooth,hencereducingtheregularizationto𝜆z{ = 3 ∙ 10Q|providedabettertrade-
offbetweenRMSEperformanceandimagesharpness.Magnitude-basedU-NetwasusedtoinitializeSMS-
NEATRinthesereconstructions.
SupportingInformationFigS9.Usingcomplex-valueddeeplearningtoinitializeJVC-SENSEyieldedsimilar
quality reconstructions asmagnitude-based U-Net processing. JVC-SENSE could flexibly utilize L2 or TV
regularizerstofurtherstabilizethereconstructionwithcomparableresults.
Supporting Information Fig S10. Both complex- andmagnitude-valued U-Net processing could denoise
diffusionimages,albeitatthecostofremainingartifacts(arrows)andespeciallyinthecaseofmagnitude-
valuednetwork,over-smoothing.UsingthesetojumpstartSMS-NEATRledtocrispimageswithmitigated
artifacts.
Supporting Information Fig S11. FISTA update rule helped stabilize SMS-MUSSELS reconstruction,
especially in later iterationswherePOCS still experienced large signalupdates.Different colors indicate
different echo images. Termination criterion was reaching less than 0.1% change between successive
imageestimatesandmaximumiterationnumberwas200.
Supporting Information Fig S12. Parameter maps from fully-sampled MUSSELS reconstruction,
correspondingtoan8-shot,66.4secondSAGEscan.
Supporting InformationFigS13.Parametererrormaps fromRinplane xSMS=8x2-foldacceleratedSMS-
MUSSELS,U-NetandSMS-NEATRreconstructionsrelativetothefully-sampleddata.
Supporting Information Fig S14. Diffusion images from 6-directions, average diffusion weighted image
(DWI),b=0,colorfractionalanisotropy(FA)andmeandiffusivity(MD)mapsfromthefully-sampledmsEPI
acquisitionwithMUSSELSreconstruction.
Supporting Information Fig S15. Diffusion images from 6-directions, root-sum-of-squares (RSoS)
combinationoferroracrossthe6-directions,averageDWI,colorFAandMDmapsfromtheaccelerated
SMS-MUSSELSreconstruction.
SupportingInformationFigS16.6-directiondiffusionimages,RSoSerroracrossthe6-directions,average
DWI,colorFAandMDmapsfromthemagnitude-valuedU-Netreconstruction.
SupportingInformationFigS17.Fully-sampleddiffusionacquisitionwithMUSSELSreconstructiondoesnot
exhibit visible Nyquist ghost artifacts. Ghost-correction was performed using 1-dimensional navigators
acquiredforeachsliceandeachshotindividually.
SupportingInformationFigS18.Fully-sampledSAGEacquisitionwithMUSSELSreconstructionandmulti-
shot FLEET calibration data do not exhibit ghost artifacts. Ghost-correction was performed using 1-