Frank SchmidtICE 5th of December 20121 1) Development of MAD-X with Space Charge and preliminary...
-
Upload
britton-dennis -
Category
Documents
-
view
216 -
download
0
Transcript of Frank SchmidtICE 5th of December 20121 1) Development of MAD-X with Space Charge and preliminary...
Frank Schmidt ICE 5th of December 2012 1
1) Development of MAD-X with Space Charge and preliminary Application to the Fermilab Booster
2) Benchmarking of PTC_ORBIT & SYNERGIA with Emphasis on Micro-Scale Stability
Visit at Fermilab (Aug – Nov 2012)Visit at Fermilab (Aug – Nov 2012)
F.S.F.S.
Frank Schmidt ICE 5th of December 2012 2
1) MAD-X development and applications Valery Kapin.
2) Orbit issues and possible remedies are being pursued together with Leonid Vorobiev. Continuous discussions with Valery Kapin and Giuliano Franchetti from GSI.
3) SYNERGIA installation on my Laptop and getting applications to work Eric Stern.
4) Collaboration with Sasha Valishev to introduce a the exact Hamiltonian into LIFETRAC (not discussed here).
AcknowledgementsAcknowledgements
Frank Schmidt ICE 5th of December 2012 3
Development of MAD-X with Space Charge and preliminary Application to the Fermilab Booster
References:
1. V. Kapin & Yu.Alexahin, “Space Charge Simulation Using MADX with Account of Synchrotron Oscillations”, Proc. RuPAC-2010, Protvino, Oct 27, 2010, pp. 204-206.
2. V. Kapin, “Space Charge Simulation Using MADX with Account of Longitudinal Motion”, FNAL Beamsdoc- 3582‑v2, Apr. 2011.
3. Y.Alexahin et al., “EFFECTS OF SPACE CHARGE AND MAGNET NONLINEARITIES ON BEAM DYNAMICS IN THE FERMILAB BOOSTER”, Proceedings of PAC07, Albuquerque, New Mexico, USA.
Task ITask I
Frank Schmidt ICE 5th of December 2012 4
MAD-X V3 with Space Charge for MAD-X V3 with Space Charge for debuncher (Valery Kapin 2010)debuncher (Valery Kapin 2010)
• Utilizing the BB elements to create a frozen space charge model but adapting emittances and Twiss parameters for the sigma determination.
• Using the MAD-X Macro technique for all operations
• Phase-I: Lattice Preparation
• Splitting the elements (at least once) and introduce SC kicks in between
• Making sure that the lattice is stable including SC and proper sigma values ( next slide)
• Transferring thick lattices into thin ones (symplectic and only way to track in MAD-X)
• Phase-II: Running (turn-by-turn due to Macro procedures)
• Time varying fields
• Emittance and TWISS calculation during run
• Output (losses, emittances, etc)
Frank Schmidt ICE 5th of December 2012 5
Procedure to determine Beam SizesProcedure to determine Beam Sizeswith large SC Tune Swings with large SC Tune Swings
• TWISS fails for integer and/or half integer tunes • Trick to avoid it: turn on SC tune-shift in small steps and let tunes converge at each step
• Lines represent Laslett formula
2
2.1
2.2
2.3
2.4
2.5
1
1.1
1.2
1.3
1.4
1.5
0 5 10 15 20
Qx_formulaQx_num_I/2Qx_num_I
Qy_formulaQy_num_I/2Qy_num_I
N_iter
Reference:
Valery Kapin talk at GSI 2009
Frank Schmidt ICE 5th of December 2012 6
Lattice with variable ParametersLattice with variable ParametersM.J.Syphers, “Status of Mu2e Operating Scenario”, Feb, 2010 Beams-doc-3556
Mu2e experiment Accelerator issue: 8 GeV Accumulator and Debuncher storage rings are used Compact bunches (<200ns, <0.025) are prepared in Accumulator ring Bunches are transferred to Debuncher ring and extracted out to the experiment using 3-rd
order resonant extraction with spill times ~150ms.
A stable, slow spill with a very intense proton beam is a big challenge.
Frank Schmidt ICE 5th of December 2012 7
• The MACRO technique of MAD-X is working fine but typically one ends up with very complex logic so that even Valery had to work hard to re-understand them after a delay of 2 years! Therefore quite unpractical for new applications.
• Macros are inherently slow!
• Therefore the idea was to reduce the use of Macros as much as possible and to modify the code to do most of the work directly within MAD-X.
• In detail:
• Time varying multipoles
• Time varying phase trombone (Forest/Schmidt: Don’t use them!)
• Time varying cavity voltage (application for Booster lattice)
• Include the sigma determination at each turn
• Allow for stop&go for the MAD-X tracking routine to allow intermediate TWISS calculation at start and the location of the SC elements
• Input via several TFS tables.
Task for MAD-X V5sp for the Task for MAD-X V5sp for the Booster as a new ApplicationBooster as a new Application
Frank Schmidt ICE 5th of December 2012 8
• I have left the MAD-X project 18 months ago.
• Laurent Deniau at CERN has taken over till then.
• I have been a manager of the code with limited involvement in the actual code writing other than the PTC part. In particular, the core of the code in C has been written exclusively by Hans Grote.
• In a state of emergency I had to be ready to fix the code quickly.
• MAD-X is not well suited for this task I since deferred evaluation
does not easily allow to put element parameters back into the database.
• Various C routines were needed but Hans did not have to help!
MAD-X Status MAD-X Status
Frank Schmidt ICE 5th of December 2012 9
N_macro_surv vs Turn Number N_macro_surv vs Turn Number for the Debuncher for the Debuncher
Timing on CERN computer
• Macro version ~20-24h
• MADX-5sp
~2-4h
Frank Schmidt ICE 5th of December 2012 10
Ex,y_rms vs Turn Number for the Ex,y_rms vs Turn Number for the DebuncherDebuncher
Frank Schmidt ICE 5th of December 2012 11
Longitudinal Emittance vs Turn Longitudinal Emittance vs Turn Number for the DebuncherNumber for the Debuncher
Frank Schmidt ICE 5th of December 2012 12
• MAD-X clean-up changes of the lattice. A small horizontal non-closure is tolerated for the time being.
• The cavities had to be replaced by zero-length cavities.
• The first order edge focusing of Bends has to be simulated with the DIPEDGE elements.
• The booster has a vertical bump and I had to fix a bug to allow for the vertical DIPEDGE element in MADX-5sp.
• Phase-I has been passed including the very large SC tuneshift of ~0.4 and a successful crossing of the half integer (0.48).
• Phase-II has been started and some turns have been tracked. However, it has become clear that for such a large SC tuneshift one has to carefully set the initial parameters like tunes etc.
Progress for the BoosterProgress for the Booster
Frank Schmidt ICE 5th of December 2012 13
Benchmarking of PTC_ORBIT & SYNERGIA with Emphasis on Micro-Scale Stability
References:
1. J.A. Holmes et al., “ORBIT: BEAM DYNAMICS CALCULATIONS FOR HIGH-INTENSITY RINGS”, Proceedings of EPAC 2002, Paris, France.
2. E.Forest, A.Molodozhenstev, A.Shishlo, J.Holmes, “Synopsis of the PTC and ORBIT Integration”, KEK Internal, 2007-4, November 2007.
3. L.G. Vorobiev and K.Hirata, “A symplectic Poisson Solver Based on Fast Fourier Transformation”, KEK Report 95-12.
4. L.G. Vorobiev, “Space Charge Templates for High-Current Beam Modeling”, FERMILAB-PUB-08-236-APC.
Task IITask II
Frank Schmidt ICE 5th of December 2012 14
H-DETUNING GSI SIS18H-DETUNING GSI SIS18
Giuliano Franchetti
Valery Kapin (2008)Shinji Machida
Giuliano Benchmark WEB page:
http://web-docs.gsi.de/~giuliano/
Frank Schmidt ICE 5th of December 2012 15
H-DETUNING PTC-ORBITH-DETUNING PTC-ORBIT
Frank Schmidt ICE 5th of December 2012 16
Fake Tune Evolution PTC-ORBITFake Tune Evolution PTC-ORBIT
Turns
Qx
Frank Schmidt ICE 5th of December 2012 17
Amplitude blow-up PTC-ORBITAmplitude blow-up PTC-ORBIT
Turns
X [
0.1
3]
RED: zero Amplitude
GREEN: 0.1 sigma Amplitude
Frank Schmidt ICE 5th of December 2012 18
Small Amplitude blow-up SYNERGIASmall Amplitude blow-up SYNERGIA
Turns
X [
0.1
3]
Frank Schmidt ICE 5th of December 2012 19
SYNERGIA CodeSYNERGIA Code
Synergia is an accelerator simulation package for combining advanced single-particle optics with computationally-intensive
collective effects.
https://compacc.fnal.gov/projects/wiki/synergia2
Synergia is designed to take advantage of computing resources ranging from desktops to supercomputers.
Frank Schmidt ICE 5th of December 2012 20
Synergia PersonnelSynergia Personnel
Synergia is developed and maintained by the Computational Physics for Accelerators group in
Fermilab's Scientific Computing Division
James Amundson, Paul Lebrun, Qiming Lu, Alex Macridin, Leo Michelotti (CHEF), Chong Shik Park,
Panagiotis Spentzouris and Eric Stern
With development contributions from Tech-X:Steve Goldhaber
Frank Schmidt ICE 5th of December 2012 21
Physics Of SynergiaPhysics Of Synergia
Single-particle optics are provided by CHEF Direct symplectic tracking
Magnets, cavities, drifts, etc. (and/or) arbitrary-order polynomial maps
Apertures Collective (single and multiple bunches)
Space charge (3D, 2.5D, semi-analytic, multiple boundary conditions)
Wakefields Can accommodate arbitrary wake functions
Frank Schmidt ICE 5th of December 2012 22
Synergia FeaturesSynergia Features
Python-driven simulations Pure C++ simulations are possible All computationally intensive calculations in C++ One example: arbitrarily complex ramping, including feedback, is
straightforward Checkpointing
Simulations can be stopped and restarted arbitrarily Parallel scalability
Has run efficiently on 2^17 = 131,072 cores(!), simulating over 10^10 particles.
Can be extended in either Python or C++ User-defined Diagnostics New collective operators Etc.
Frank Schmidt ICE 5th of December 2012 23
Plans for SynergiaPlans for Synergia
More collective effects Electron cloud
In progress Beam-beam?
Cern related Features Double RF implementation Time varying fields Acceleration Foils Observer Particles
• Parsers for more lattice formats
• Mad8 finished• MadX close to done• others as requested
• Full documentation• End-user, API and examples
• Fixed API• Very close
Frank Schmidt ICE 5th of December 2012 24
““Unfair” speed comparison between Unfair” speed comparison between PTC-ORBIT and SYNERGIAPTC-ORBIT and SYNERGIA
• I have made a speed comparison of the SIS18, which is just a linear lattice plus space charge.
• For the same number of space charge kicks, the same grid bins and the same number of macro-particles. I find Synergia ~5 times faster than PTC-ORBIT.
• To avoid to be “unfair” one has to make a more careful analysis for various cases and with non-linearities.
• PTC allows a very general treatment of space charge without any need to split elements. This might be inherently inefficient.
• It also could be that up to now PTC-ORBIT is not yet fully optimized. Presently, we have started an attempt to profile the code to understand where the time is being spent. To identify remedies will take some time.
Frank Schmidt ICE 5th of December 2012 25
H-DETUNING SYNERGIA H-DETUNING SYNERGIA 32 Turns32 Turns
0.1
Qx
Frank Schmidt ICE 5th of December 2012 26
H-DETUNING SYNERGIA H-DETUNING SYNERGIA 1024 Turns1024 Turns
0.1
Qx
Frank Schmidt ICE 5th of December 2012 27
Sudden shift in Tune SYNERGIASudden shift in Tune SYNERGIA
Turns
X[m
]
Frank Schmidt ICE 5th of December 2012 28
Evolution of Linear Invariant Evolution of Linear Invariant SYNERGIASYNERGIA
Co
ura
nt S
nyd
er
Inva
rian
t
Frank Schmidt ICE 5th of December 2012 29
Analysis of the situation with PIC Analysis of the situation with PIC codescodes
• Originally, I had planned to follow the complete benchmarking procedure for PTC-ORBIT as prescribed by Giuliano. In view of a possible upgrade of ORBIT to PTC-ORBIT at Fermilab.
•Following this surprising results I checked out with Leonid every possible source of error that I might have done. In fact, he repeated the same studies with plain ORBIT and we find the same results.
• The conclusion is: that for PTC_ORBIT the benchmarking can simply not be done.
• We then contacted the author Jeff Holmes and his team and let them be the first to be
informed about this problem.
• A couple of weeks later I have received an Email by Jeff with a confirmation of our finding: “I have been doing a bit of work on the individual particle tracking with space charge. I set up a uniform focusing channel and a KV distribution. I observe the same numerical diffusion behavior seen by you and Leonid. If I symmetrize the distribution in the x-y planes, the diffusion is much slower and delayed, but eventually it does occur (due to roundoff?).”
• Lastly, Synergia behaves much better but is also suffering from this instability.
Frank Schmidt ICE 5th of December 2012 30
Remedies and DiscussionsRemedies and Discussions
• During the discussions with Jeff we have agreed to investigate possible remedies within the next 6 months.
• Independently, Leonid is reviving his decade old very careful “clouds in PIC” analysis and he also has a much improved Poisson solver.
• The much smoother distribution (see for comparison next 2 slides) did not in itself lead to a substantial improvement. But we still are hopeful that the full replacement of the space charge classes in ORBIT will bring the desired improvement.
• To a lesser extend SYNERGIA suffers also from this problem and we should be looking into that as well.
• Leonid also has a radically different approach that is to use the so-called space charge templates (see reference above), i.e. NOT based on PIC. It remains to be seen how this will work out!
• Following all these findings and the excitement I am happy to invite you to a space charge workshop to be held in April 2013 at CERN where we will discuss these issues in depth and hopefully with a better understanding!
• For me the crucial question is if PIC codes can be reliably used for 1’000 - 10’000 turns or more despite this micro-scale instability?
Frank Schmidt ICE 5th of December 2012 31
Shapes of Distributions (Leonid)Shapes of Distributions (Leonid)
Frank Schmidt ICE 5th of December 2012 32
Original ORBIT DensityOriginal ORBIT Density
Frank Schmidt ICE 5th of December 2012 33
Reversed Parabolic Density Reversed Parabolic Density (#3 in Shape Slide)(#3 in Shape Slide)
Frank Schmidt ICE 5th of December 2012 34
• MADX-5sp is going to be used at CERN as our frozen model tool for space charge studies. Any developments, bugfixes etc will be provided to Fermilab. Most probable Valery Kapin and Yuri Alexahin will use it here and we will certainly help in case of problems and/or further developments.
• Optimizing the ORBIT code: Leonid Vorobiev has started to replace the space charge classes in ORBIT. Our plan is to continue our collaboration remotely. But there has been discussions with his future employer to invite Leonid for an extended period to CERN to finish this work in the most efficient way. In fact, since we are investigating a possible speed-up of PTC-ORBIT we could at the same time make his implementation compatible with MPI.
Planned Collaborations IPlanned Collaborations I
Frank Schmidt ICE 5th of December 2012 35
• ORBIT authors: Leonid and myself have approached Jeff Holmes and his team to present out findings about the unstable particle motion on the micro-scale level. We have agreed with them to find a solution or at least achieving a dramatic improvement within the next 6 months. It could very well be that the MPI version of Leonid’s implementation will be the best that can be done.
• SYNERGIA: We have prepared in writing a program for a long-term collaboration for a possible use of this code for the LIU (LHC Injector Upgrade) at CERN. Missing code features have been identified and have been discussed with the code developers. Special attractions are the modern structure with a nice PYTHON interface and the high performance compared to other codes. We have also agreed to perform the full benchmarking cycle that has been prepared by Giuliano Franchetti at GSI.
Planned Collaborations IIPlanned Collaborations II
Frank Schmidt ICE 5th of December 2012 36
ReserveReserve
Frank Schmidt ICE 5th of December 2012 37
Synergia PersonnelSynergia Personnel
Synergia is developed and maintained by the Computational Physics for Accelerators group in
Fermilab's Scientific Computing Division
James Amundson, Paul Lebrun, Qiming Lu, Alex Macridin, Leo Michelotti (CHEF), Chong Shik Park,
Panagiotis Spentzouris and Eric Stern
With development contributions from Tech-X:Steve Goldhaber
Frank Schmidt ICE 5th of December 2012 38
Physics Of SynergiaPhysics Of Synergia
Single-particle optics are provided by CHEF Direct symplectic tracking
Magnets, cavities, drifts, etc. (and/or) arbitrary-order polynomial maps
Apertures Collective (single and multiple bunches)
Space charge (3D, 2.5D, semi-analytic, multiple boundary conditions)
Wakefields Can accommodate arbitrary wake functions
Frank Schmidt ICE 5th of December 2012 39
Synergia Code structureSynergia Code structure
A simulation consists of propagating a Bunch (or bunches) through a Lattice.
Inputs: machine lattice, initial bunch parametersOutputs: user-selected Diagnostics.
6d means 6d std deviations 6x6 covariance matrix 6x6 correlation matrix Individual particle tracks Dump of all particles Losses with locations in lattice
Some available Diagnostics
Actions may specify Diagnostics every n steps, every m turns, specified sets of turns or sets of steps, etc. User-defined logic is also possible.
Frank Schmidt ICE 5th of December 2012 40
Synergia FeaturesSynergia Features
Python-driven simulations Pure C++ simulations are possible All computationally intensive calculations in C++ One example: arbitrarily complex ramping, including feedback, is
straightforward Checkpointing
Simulations can be saved and restarted arbitrarily Parallel scalability
Has run efficiently on 2^17 = 131,072 cores(!), simulating over 10^10 particles.
Can be extended in either Python or C++ User-defined Diagnostics New collective operators Etc.
Frank Schmidt ICE 5th of December 2012 41
Status of SynergiaStatus of Synergia
Synergia is in production use at Fermilab Main Injector space charge Mu2e resonant extraction Booster with SC and wakefields
End-user release in near future
Frank Schmidt ICE 5th of December 2012 42
Potential with zero Boundary Potential with zero Boundary Condition Assumption (Leonid)Condition Assumption (Leonid)