Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation
description
Transcript of Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation
![Page 1: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/1.jpg)
Status and future perspectives of the
analysis language (eDSL) for HWC and LHC Operation
Kajetan FuchsbergerTE-MPE TM, 2014-02-27
On behalf of the TE-MPE-MS Software Team:D. Anderson, M. Audrain, M. Dragu, J.-C. Garnier,
A. Gorzawski, M. Koza, K.H. Krol, A. Moscatelli, K.Stamos,P.C. Turcu
![Page 2: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/2.jpg)
2Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
Content
Motivation
Status & Usage
Behind the Scenes
Outlook & Visions
![Page 3: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/3.jpg)
3Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
Warning!Vi
siona
rity
Tem
pera
ture
100°
50°
0°
Visionarity Temperature might be High!
(But not unrealistic ;-)
![Page 4: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/4.jpg)
4Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
Content
Motivation
Status & Usage
Behind the Scenes
Outlook & Visions
![Page 5: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/5.jpg)
5Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
eDSL ?
eDSLDomain
Specific Language
embedded
Hwc; Systems; Accelerators; Signals;
![Page 6: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/6.jpg)
6Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
Why a DSL at all? Currently:
E.g. writing a PM module (or an analysis for a test) is pure coding (Java/LV). A lot of knowledge required, which distracts from the real logic: Where to get the data from? What tools to use to display it? What algorithms to apply?
A lot of code duplication. E.g.: E.g. ArrayUtils in PM modules Every application again connects in its own way to the Logging DB,
PM, reads files....
DSL focuses on the logic of the domain:
![Page 7: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/7.jpg)
7Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
Why (Java) embedded? Integration within the Accelerator SW Architecture Java
We are not language designers, but software developers.
Reuse: All the native language features of Java, while providing the
required readability for non-programmers. Tools for writing and verification of analysis scripts (eclipse). Strongly typed; with the help of an external library, it was
easy to enforce the correct usage of physical units.
It provides the necessary possibility to constrain the user input to the available functionality.
![Page 8: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/8.jpg)
8Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
Where could it be used? Test Analysis within AccTesting (origin)
PM modules
Pre/Post operational checks
periodic calls + reports: E.g. Splice measurements LHC statistics
…?
Any Java Application!
Visio
narit
y Te
mpe
ratu
re
100°
50°
0°
![Page 9: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/9.jpg)
9Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
What about a better name?
ZeldaZuper Easy Language
for Data Analysis
Brainstorming….
GENIAL
Generic Integrated Analysis Language
Analysis Language for Operation and Hardware
Commissioning
![Page 10: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/10.jpg)
10Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
Content
Motivation
Status & Usage
Behind the Scenes
Outlook & Visions
![Page 11: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/11.jpg)
11Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
Actual Status
Courtesy: M. Audrain
Visio
narit
y Te
mpe
ratu
re
100°
50°
0°
![Page 12: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/12.jpg)
12Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
Full Automation of PNO.d1 and PCC.1
See Scott & Zinur‘s presentation
Executed ~2000 times within an HWC campaign
Visio
narit
y Te
mpe
ratu
re
100°
50°
0°
![Page 13: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/13.jpg)
13Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
What are we missing? More signals from post mortem
(E.g. QPS buffers)
Circuit parameters (deferred scalars)
Calculations:
All logging variables (code generation)
Visio
narit
y Te
mpe
ratu
re
100°
50°
0°
![Page 14: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/14.jpg)
14Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
How to test? - Framework On our (SW) side:
http://sonar.cern.ch See also Cristina‘s recent Presentation:https://indico.cern.ch/event/293574/
![Page 15: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/15.jpg)
15Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
How to test? – Module Developer
Use a set of known cases for which the module passes or fails.
To be automated (bamboo)!
![Page 16: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/16.jpg)
16Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
Systematic Checks against old data
![Page 17: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/17.jpg)
17Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
Integration in AccTestingTo come:Add possibility to display arbitrary signals
![Page 18: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/18.jpg)
18Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
(mis ;-) use for data extraction Do not care, where data comes from!
Visio
narit
y Te
mpe
ratu
re
100°
50°
0°
![Page 19: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/19.jpg)
19Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
Content
Motivation
Status & Usage
Behind the Scenes
Outlook & Visions
![Page 20: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/20.jpg)
20Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
Behind the scenes
request
„compile“
Data
Result
Only extract necessary data!
![Page 21: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/21.jpg)
21Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
The Tree …
Courtesy: Kamil Krol
![Page 22: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/22.jpg)
22Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
Resolving….
Main Principle: Ask the Modules, to resolve the biggest subtree possible in one go.
![Page 23: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/23.jpg)
23Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
Content
Motivation
Status & Usage
Behind the Scenes
Outlook & Visions
![Page 24: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/24.jpg)
24Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
A more abstract view Requirements for data Analysis:
Data incompleteness (e.g. a beam positions along the ring: added/removed BPMs)
Data invalidity: (e.g BPMs might be misbehaving) Mathematical operations: (e.g.
average/addition/scaling) Physical units: (Different units in storage ->
conversion left to the user) Error propagation: left to the user
Visio
narit
y Te
mpe
ratu
re
100°
50°
0°
![Page 25: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/25.jpg)
25Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
Anything out there?
Courtesy:K. Stamos, D. Anderson
Visio
narit
y Te
mpe
ratu
re
100°
50°
0°
![Page 26: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/26.jpg)
26Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
Tensorics® Everything is a tensor.
Common Operations.
Direct and deferred Usage.
Choosable strategies(with useful defaults): Error propagation Data incompleteness Units
To be released as Open Source soon.
Courtesy:A. GorzawskiVi
siona
rity
Tem
pera
ture
100°
50°
0°
![Page 27: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/27.jpg)
27Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
The dispatcher – possible Improvements
Caching
All operations are Stateless (basically mathematical functions) easy to parallelize.
E. Montesnitsalis: UFO search as an example Use Case
Visio
narit
y Te
mpe
ratu
re
100°
50°
0°
![Page 28: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/28.jpg)
28Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
Blockers & Wishes ;-) Java 8: Closures!
Would allow us to make the language more dynamic and easily extensible.
Will be production released on 18th March. Unfortunately blocked by CO. Will not come
before end of Run 2! Cluster!
To evaluate different options (parallelization, Scalability)
Useful also for different other R&D: PM storage, distributed file systems, test environments, R&D for future protection systems.
Visio
narit
y Te
mpe
ratu
re
100°
50°
0°
![Page 29: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/29.jpg)
29Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
Only measured data? Circuit Models?
Accelerator Models (MadX)?
Acquisitions (live data)
…
Beam Commissioning?Visio
narit
y Te
mpe
ratu
re
100°
50°
0°
![Page 30: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/30.jpg)
30Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
Summary & Outlook Ready (from language point of view) to analyse PNO.d1 and
PCC.1
Next features: More Signals, Circuit parameters (scalars), Calculations.
Switch to Tensorics® when ready.
Optimization (Caching, Parallelization…) to come?… as soon as we can get some cluster!
Can be use in other fields (Models? Big Data analysis? Supervision? Statistics…)
Wishes/dreams: Java8 and a Cluster ;-)
Visio
narit
y Te
mpe
ratu
re
100°
50°
0°Visio
narit
y Te
mpe
ratu
re
100°
50°
0°Visio
narit
y Te
mpe
ratu
re
100°
50°
0°Visio
narit
y Te
mpe
ratu
re
100°
50°
0°Visio
narit
y Te
mpe
ratu
re
100°
50°
0°
![Page 31: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/31.jpg)
31Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
Growing a Language http://www.youtube.com/watch?v=_ahvzDzKdB0
Guy Steele ‚Growing a Language‘(Keynote OOPSALA Conference, 1998)
„If one lets the users help do the work, growth can be quick!“
„if many persons work side by side and the best work is added with care and good taste, a great deal can be added in short time.“
![Page 32: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/32.jpg)
32Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
Thanks a lot for your Attention!
![Page 33: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/33.jpg)
33Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
Resolving…. Ask each resolver: can resolve ?
If a resolver can resolve the given node remember.
If none of the resolvers can resolve the node, then the same procedure is recursively invoked with all the children of the node (e.g. node B in the example).
Else (at least one potential candidate resolver), the dispatcher stops the iteration through the tree in this branch and does not try to resolve the child nodes anymore (in the initial state of the example, this would happen for node C).
As soon as the iterations through all branches are the dispatcher selects one of the candidate resolvers per node, invokes it and rebuilds the tree with the resolved node.
After all resolvers return, the whole algorithm is looped starting at item (1), until all nodes (including the root node) are resolved.
![Page 34: Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation](https://reader036.fdocuments.in/reader036/viewer/2022062323/568161c1550346895dd1a712/html5/thumbnails/34.jpg)
34Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation
The Tree …
Courtesy: Kamil Krol