SOFTWARE SUSTAINABILITY AND TOOLKITS: HOW …...26 David Sagan KEK September 18 -19, 2019...
Transcript of SOFTWARE SUSTAINABILITY AND TOOLKITS: HOW …...26 David Sagan KEK September 18 -19, 2019...
SOFTWARE SUSTAINABILITY AND TOOLKITS:
HOW TO PROGRAM FOR THE ACCELERATOR SIMULATION PROBLEMS OF TOMORROW
David Sagan
Cornell Laboratory for Accelerator Based Sciences and Education
Linac and Storage Ring Programs No Longer Used:
■ AGS,■ ALIGN■ COMFORT■ DESIGN■ DIMAD■ GUINEA-PIG■ HARMON■ LEGO■ LIAR■ MAGIC
■ MARYLIE■ PATRICIA■ PETROS■ PLACET■ RACETRACK■ SYNCH■ TRACY ■ TRANSPORT■ TURTLE■ UAL
David Sagan KEK September 18-19, 20192
How many man-years of effort went into these programs?How can we, as an accelerator physics community do better?How can software be developed that is useful and is used year after year?
David Sagan KEK September 18-19, 20193
Outline of the Talk
■ Software Sustainability – What is it?
■ Software Toolkits – What is a toolkit? How do toolkits help sustainability?
■ Bmad – An example of a toolkit.
SOFTWARESUSTAINABILITY
David Sagan KEK September 18-19, 20194
What is Software Sustainability?
David Sagan KEK September 18-19, 20195
A definition of Software Sustainability:
"the capacity of the software to endure. In other words, sustainability means that the software will continue to be available in the future, on new platforms, meeting new needs.”
From: Daviel Katz https://danielskatzblog.wordpress.com/2016/09/13/defining-software-sustainability/
Readability / Understandability
Extensibility / Evolvability
Portability / Installability
Scalability
Speed
Learnability
Documentation
Modularity / Reusability
Correctness / Testability
David Sagan KEK September 18-19, 20196
What Goes Into Software Sustainability?
■ Intrinsic: The characteristics of the software itself. ■ Extrinsic: The environment in which the software is developed and/or used.
Accessibility / Openly Available?--------
Usefulness---
Licensing-
Funding
Community-
Version-Control---
Actively Maintained--------
Intrinsic Extrinsic
David Sagan KEK September 18-19, 20197
SOFTWARE EVALUATION: CRITERIA-BASED
NOVEMBER 2011
Software Evaluation: Criteria-based Assessment Mike Jackson, Steve Crouch and Rob Baxter
Criteria-based assessment is a quantitative assessment of the software in terms of sustainability, maintainability, and usability. This can inform high-level decisions on specific areas for software improvement. A criteria-based assessment gives a measurement of quality in a number of areas. These areas are derived from ISO/IEC 9126-1 Software engineering — Product quality1 and include usability, sustainability and maintainability.
The assessment involves checking whether the software, and the project that develops it, conforms
And there exists Standards for Code Quality
Example of Non-Sustainable Code
David Sagan KEK September 18-19, 20198
”Piled Higher and Deeper" by Jorge Cham www.phdcomics.com. Used with permission.
"Software decays relatively quickly if it is not maintained and this is especially true for software used in research"
Mario Rosado de Souza, Robert Haines, Markel Vigo, Caroline Jay, "What Makes Research Software Sustainable? An Interview Study With Research Software Engineers."
SOFTWARETOOLKITS
David Sagan KEK September 18-19, 20199
Problem: Developing simulation code takes time.
David Sagan KEK September 18-19, 201910
⾞輪の再発明
In general, accelerator simulation problems have some common needs like reading in lattice information or calculating orbits.
Writing code from scratch to do this is like reinventing the wheel.
How can we do better?
What does it take to create programs efficiently?
AcceleratorPhysicist
Actual SimulationProgram
Desired Simulation Program
What is a Software Toolkit?
David Sagan KEK September 18-19, 201911
Definition of a Toolkit:
An integrated set of modularsoftware routines that are used to develop and maintain applications or databases.
A Toolkit is like a bunch of Lego blocks
David Sagan KEK September 18-19, 2019 12
Advantages of a toolkit:• Cuts down on the time needed to develop programs.• Cuts down on programming errors (via module reuse).• Standardizes sharing of lattice information between
programs. • Increased safety: Modular code provides a firewall. For
example, a buggy module introduced into the toolkit will not affect programs that do not use it.
Toolkit Dynamic Aperture Program
Control System Programs
Lattice Design Program
Etc.
IBS Simulation Programs
A toolkit, by making the software more useful, helps make the software sustainable.
Accelerator Simulation
Toolkits
David Sagan KEK September 18-19, 201913
Accelerator Toolbox
Bmad
COSY INFINITY
____ __ ______ ______ / \/ \ / _ \ / _ \ / __ / / /_/ / / /_/ / /__/ /_/ /__/ /_/ /_____ / MAD-NG
Merlin/Merlin++
FPP/PTC
Matlab based. Used extensively at light sources for such things as orbit response matrix analysis.
Includes modules for spin tracking, low energy space charge, etc. Has interface to PTC
In development by Laurent Denieu (CERN). Replacement for MAD-X.
Recent developments oriented towards simulating wakefields and particle/matter interactions.
Uses Differential Algebra (DA) for tracking.
Has Differential Algebra (DA), symplectic tracking, power series map, normal form analysis, etc.
BMAD
David Sagan KEK September 18-19, 201914
In the Beginning…Brief History:
§ Born at Cornell in mid 1990’s
§ Started life as modest project: Just wanted to calculate Twiss functions and closed orbits within control system programs.
§ Initially Bmad used a subset of the MAD lattice syntax. Hence the name: “Baby MAD” or “Bmad” for short.
David Sagan KEK September 18-19, 201915
And Baby Grows Up...
Currently:§ >100,000 lines of code§ ~1,000 routines
David Sagan KEK September 18-19, 201916
And Bmad can do Much
More:
§ Lattice design § X-ray simulations§ Spin tracking§ Wakefields and HOMs§ Beam breakup (BBU) simulations in ERLs§ Intra-beam scattering (IBS) simulations§ Coherent Synchrotron Radiation (CSR)§ Touschek Simulations§ Frequency map analysis§ Dark current tracking§ Etc., etc.
David Sagan KEK September 18-19, 201917
Overview§ Written in object oriented Fortran 2008. § With certain restrictions, Bmad can be run
multi-threaded.§ Lattice files use a MAD like syntax.§ Well documented (Manual is >500 pages).§ Open Source:
http://www.classe.cornell.edu/bmad/ § And Bmad is indeed a toolkit:
David Sagan KEK September 18-19, 201918
type (lat_struct) latcall bmad_parser (‘lat.bmad’, lat)
Data Lego block
Action Lego block
Bmad Features
David Sagan KEK September 18-19, 2019 19
Bmad has a number of features that over the years
have proven useful.Among
these are:
Superposition – Define overlapping elements.
Controller elements – Elements controlling attributes of other elements.
Multiple connected beam lines in one lattice – Example: Injection line attached to a ring.Multipass lines – Multiple beams sharing a common line such as the IR region in a dual ring colliding beam machine.
Custom elements and custom particle tracking.
Define beam chamber walls.
Example of how a lattice can contain multiple attached beam lines and this can be used to describe complicated machine geometries
6 MeV
42, 78, 114, 150 MeV
Mirror Merger
Injector Merger
Diagnostics
Bmad Ecosystem
20 David Sagan KEK September 18-19, 2019
Due to its flexibility, Bmad has been used in a number of programs including:•Tao: General purpose design and simulation.•Synrad3d: 3D tracking of synch photons, including
reflections, within the beam chamber walls.•BBU: Beam breakup instability simulations.•dark_current_tracker: Dark current electron simulation.•ibs_sim: Analytic intra-beam scattering (IBS) calculation.•touschek_track: Tracking of Touschek particles.•freq_map: Frequency map analysis.•MOGA: Multi-Objective Genetic Algorithm optimization.•Lux: Photon tracking in X-ray beam lines.•etc...
Code reuse: Modules developed for one program can, via Bmad, be used in other programs.
Tune Scan for CESR Ring Upgrade
Circles = design tunes, Diamonds = current tunes.
BBU threshold current for CBETA as a function of the phase advance between cavities.
Tao: Tool for Accelerator Optics
■ Problem: Bmad is not a program so it cannot be used “out of the box.” for simple calculations.
■ Solution: Develop Tao - a general purpose simulation & design program
• Twiss and orbit calculations.• Nonlinear optimization.• Lattice design.• Etc.
■ Additionally: Tao’s object oriented coding makes it relatively easy to extend it.
■ For example: Can add custom commands to interface Tao with a control system (EG: CBETA ERL).
21 David Sagan KEK September 18-19, 2019
Ø Bmad with Tao gives the flexibility of a Toolkit with the convenience of a program.
Example: Modifying the CESR ring to stay within the existing building.
Example: Element misalignment in the CBETA ERL followed by a steering correction.
s (m)
Orb
it (
mm
)
Energy (MeV)
150
114
78
42
Energy (MeV)
150
114
78
42
s (m)
Orb
it (
mm
)
Orbits w/ misalignment
Orbits after correction
Synrad3D was created to calculate the initial distribution of photoelectrons for electron cloud studies. It has also been used for studying the efficiency of beam masks.
Simulation Procedure:1. Calculate beam orbit2. Emission of photons from the beam3. 3D tracking of photons to the vacuum chamber wall4. Scattering or absorption at the chamber wall5. If not absorbed, further tracking until absorption
Synrad3D Program
22 David Sagan KEK September 18-19, 2019
Synrad3D Development
23 David Sagan KEK September 18-19, 2019
Electron cloud distribution vs position after 10 bunch train
At the start of development, Bmad provided code for:• Lattice parsing• Calculation of the closed orbit• Photon generation• Defining the vacuum chamber wall
Synrad3d development involved creating code for:• Tracking of photons.• Reflection/absorption at the chamber wall.
èThe use of Bmad saved considerable development time.
Also: The code for defining the vacuum chamber wall came from dark electron simulation development.
èCode developed for one purpose can be used in other places saving development time.
Synrad3d: SuperKEKB Simulation
0
50
100
150
200
250
0 100 200 300 400
Simulations by Jim CrittendenSuperKEKB wall profile courtesy Takuya IshibashiProblem: Photoelectrons can be trapped in the field of a quadrupole. The electrons will react with the beam and if the density is high enough, this can cause problems.Simulation to determine the number of photons absorbed near a particular quadrupole
EntriesMeanRMS
3521 50.92 75.89
0
100
200
300
400
500
600
700
800
0 200 4000
25
50
75
100
125
150
175
200
3015.2 3015.3 3015.4 3015.5 3015.6
Photon numberEnergy Spectrum Angular distribution
24 David Sagan KEK September 18-19, 2019
25 David Sagan KEK September 18-19, 2019
12 cm
Problem: SuperKEKB lattices have a chicane with px ~ 0.03.
But MAD8/X uses the paraxial approximation which assumes that px, py << 1.
So in this case a MAD driftelement is a poor approximation for the chicane drifts.
SAD To MAD Lattice
Translation for
SuperKEKB
26 David Sagan KEK September 18-19, 2019
Translation Solution
Translation is done in two steps:1. SAD -> Bmad2. Bmad -> MAD
In the Bmad -> MAD step, Bmad uses Etienne’s PTC Toolkit to construct a 2nd order map for the chicane drifts which is then instantiated as a MAD matrix element.
Since Bmad and PTC are toolkits, the translation code is integrated into a unified program.
LATTICE TRANSLATION BETWEEN ACCELERATOR SIMULATION
D. Zhou , H. Koiso, A. Morita, Y. Ohnishi, K. Oide, H. Sugimoto, KEK, Tsukuba, JapanM.E. Biagini, INFN/LNF, Frascati, Italy
N. Carmignani †, S.M. Liuzzo, ESRF, Grenoble, FranceD. Sagan ‡, Cornell University, Ithaca, NY, USA
CODES FOR SuperKEKB
Proceedings of IPAC2016 , Busan, Korea WEPOY040
o
pointchamberBeam vacuum
Tangential reference line1.25
Muon orbit
Beam line
Injection
G-2 Simulation Program■ Dave Rubin at Cornell has been
developing a simulation program to simulate the Muon g-2 experiment at Fermilab.
■ Need to track the polarized muons with:• Injection line into a storage ring.• Three dimensional field of the injection
line.• Scattering of muons as they cross the
inflector wall• Electrostatic quadrupoles• Muon decay• Tracking of electron decay product
27
Polarized Muons
David Sagan KEK September 18-19, 2019
G-2 Simulation■ At the start of program development Bmad provided:
– Ability to define the geometry of the injection line and storage ring.
– Ability to define the geometry of the inflector wall– Ability to define custom fields for the injection line
and the ability to simulate electrostatic quadrupoles
■ Needed to develop for the program:– Scattering of muons through the inflector wall– muon decay.– etc.
28
è Bmad reduced the development time for creating the program and provides a flexible framework for future program modifications.
David Sagan KEK September 18-19, 2019
CONCLUSIONS
29 David Sagan KEK September 18-19, 2019
Some Thoughts
David Sagan KEK September 18-19, 2019 30
Main advantage of a software toolkit: The flexibility so that programs can be developed in less time and with fewer bugs.
Main disadvantages of a toolkit: Increased work for the toolkit developers in terms of documentation and stricter code quality requirements.
Personal opinion: The advantages of a toolkit far outweigh the disadvantages for accelerator simulation software.
In particular, Bmad has been successful due to it’s modular, object-oriented design which allows it to be adapted to ever changing simulation needs. The quality and quantity of Bmad based programs would not be possible if Bmad where not a toolkit.
Bmad has been used at Cornell, CERN, KEK, DESY, ANL, BNL, Cockcroft Institute, for simulating CESR, ILC, LCLS2, SuperKEK-B, CBETA, ALS-U, Fermilab g-2 muon project, etc.
Bmad is always in continual development with about 1.5 FTEs. Supported by Cornell Laboratory for Accelerator-based Sciences and Education (CLASSE).
Bmad is evolving towards being able to do “complete” start-to-end simulations.
Thanks to ContibutorsDan AbellIvan BazarovMoritz BeckmannMartin BerzJoel BrockDesmond BarberSarah BuchanAvishek ChatterjeeJing Yee CheeChristie ChiuJoseph ChoiRobert CopeJim Crittenden
Attilio De FalcoGerry Dugan Michael EhrlichmanKen FinkelsteinEtienne ForestMike ForsterHans GroteRichard HelmsGeorg HoffstaetterChris MayesKarthik NarayanKatsunobu OideMark Palmer
Tia PlautzMatt RandazzoMatt RendinaDave RubinMichael SaelimFrank Schmidt Jim ShanksJeff SmithJeremy UrbanMark WoodleyDemin Zhou
31 David Sagan KEK September 18-19, 2019
END
32 David Sagan KEK September 18-19, 2019
Programming with Bmad
David Sagan KEK September 18-19, 2019 33
For programmers, to maximize Bmad’s usefulness, Bmad is designed to be modular and object oriented
from the ground up:
There are two types of Bmad users:
The majority are non-programmers who use Bmad based programs.
A minority are programmers who develop
Bmad based programs.
type (lat_struct) latcall bmad_parser (‘lat.bmad’, lat)
Tao Examples
■ Bmad with Tao gives the flexibility of a Toolkit with the convenience of a program.
34
■ Example: Designing or modifying a machine to be/stay within an existing building.
■ Example: Element misalignment in CBETA followed by steering correction.
s (m)
Orb
it (
mm
)
Energy (MeV)
150
114
78
42
Energy (MeV)
150
114
78
42
s (m)
Orb
it (
mm
)
Orbits w/ misalignment
Orbits after correction
David Sagan KEK September 18-19, 2019