Analysis, & future direction

A FRAMEWORK FOR OFFLINE VERIFICATION OF BEAM INSTRUMENTATION SYSTEMS

MOTIVATION• BI systems need regular checks to confirm their beam readiness

• Detecting deterioration in performance and identifying physical problems / anomalies

• Existing checks deployed in LHC sequencer tasks are generally comprehensive but…

• Scope limited to LHC – No injector chain systems

• Operator-centric tests aren’t directly accessible to hardware / software experts

• As of 2013, several ad-hoc solutions emerged

• Code-sharing / duplication in Java based applications

• Manual data extraction and analysis using Timber

• Many Python scripts from the BL section

• etc…

PROBLEMS WITH CURRENT PYTHON SCRIPTS• Often, developers writing analysis code will ultimately leave CERN

• Need to ensure the code is easy to maintain after they leave!

• APIs would enforce standards to facilitate this long-term maintenance.

• Also, choice of development language is an issue…

• Python not officially supported in BE and not common in BI SW

• In reality Python is used in many places and works

• But some maintenance is needed from BI SW to add the additional libraries used by BL for example

• When moving O/S (i.e. SLC5->SLC6), customization of bdidev1 needed

• Code is not easily shared from BI SW Expert apps as these are always in Java or C++

• Moving to Java as analysis language also has issues

• Learning curve steeper for equipment specialists

• Heavier than a scripting language

CURRENT ENVIRONMENT

Detectors

Linux Front End

Oracle

NFS FilesExpert GUIsS

Online Data flow

Offline analysis(Python scripts as cron tasks)

Scripts.py

Extracted with system() call to java app

PyROOT

matplotlib

SciPyPDF

report

Text repor

t + imag

es

*

pdflatex

Email specified users

Problems

• Clumsy data extraction

• Little code re-use• PDF creation very

verbose• PNG creation very

verbose• Report distribution

by email not always suitable

• No direct access to historical reports

Report Service

Informative Report

Published on Website Email

Action Report

Email

Reporting APIs

Report Renderers

PDFHTML

Report Transport

eMailHTTP

Data APIs

Data ContainersReport Data

TitleRow(s) X Data or XY Data Title Index Type (time, scalar, etc.) Colour ….

Report TextContentTitleType (HTML or Plain)….

Data RenderersTableGraphText

Data Source APIs

Source Factories

OracleNFS File

Source Interfaces

Binary

XMLDictionary

LANGUAGE NEUTRAL API PROPOSAL

DATABASE ALTERNATIVE

CERN Accelerator Data Services

Domain Specific

Language (DSL) scripts

LSA Settings Database

Accelerator Logging Service

Generated PL/SQL or

Oracle Enterprise R

code

Triggered by accelerator events or preset times

Results of analysis stored back in database

Results viewed on standard CERN data visualisation

tools

Post Mortem System

Datasource

s

ANALYSIS CURRENTLY OPERATIONAL• Java

• Analysis is generally embedded within an Expert GUI

• In some cases, for the BLM modulation tests for example, the code is re-used in a sequencer task

• No pure analysis scripts exist outside the expert GUIs and sequencer

• Python

• Analysis is on a script-by-script basis

• Some code sharing

• 43 .py files. Many probably not used any more

EXAMPLE JAVA ANALYSIS

Expert GUIs written in Java produce visual analysis of BLM connectivity checks

Code performing analysis shared with an LHC sequencer task

EXAMPLE PYTHON ANALYSISPDF Report on BLM threshold changes

Image of BLM CRC errors vs temperature

Image of surface building temperatures and ‘nominal’ range in

green

PROPOSED API IMPLEMENTATION PRODUCTS

NEXT STEPS…• Awaiting direction from database team for future of the DSL proposal

• A large portion of the BLM analysis could be transferred to this solution

• DSL scripts would be executed on the database, therefore they would be much more performant

• Machine event triggered script execution as well as time based execution very interesting

• Post-mortems, XPOC, etc

• For the other analysis, need to decide on future language

• Java or Python or both

• A full implementation of the API should be made

• Further investigations into new ideas such as web-site report distribution need to be made

• Historical analysis retrieval etc

• Discussions are currently ongoing as to who will carry out the implementation of the APIs and also timescales etc…