ISIS: towards a 21st century facility computing environment (slowly) Robert McGreevy Science and...
-
Upload
loren-jacobs -
Category
Documents
-
view
216 -
download
0
Transcript of ISIS: towards a 21st century facility computing environment (slowly) Robert McGreevy Science and...
ISIS: towards a 21st century facility computing environment
(slowly)
Robert McGreevy
Science and Technology Facilities Council
ISIS, Rutherford Appleton Laboratory, Harwell Science and Innovation Campus, Didcot, OX11 0QX, UK
3rd November 2008
Source
Instruments
Sample environment
Data treatment
Scientific results
Source
Instruments
Sample environment
Data treatment
Scientific results
30 years ago …
ISIS Target Station 1
ISIS Target Station 2
StructuresNIMROD Intermediate
range order in liquidsWISH High-resolution
magnetic structureSANS2D Large
molecule structure in multi-component
systems
ReflectometryINTER Air/ liquid/ solid interface interactionsOFFSPEC Structures of membrane, protein and liquid interfacesPOLREF Interface measurements in magnetic sensor devices
DynamicsLET High-resolution measurement of material energy scales
TS2 Instruments Phase 1
StructuresNIMROD Intermediate range order in liquidsLMX Large molecule single crystal diffractionZOOM Focusing small angle scatteringIMAT Imaging / diffraction
Structure/DynamicsLARMOR SESANS, High resolution diffraction, MIEZE, MISANS
TS2 Instruments Phase2
Understanding
Data analysis
Data visualisation
Data treatment
Data diagnostics
Data acquisition
Measurement
The data chain
Data curation Data (scientific)
Data (raw)
Data (publication)
Data (metadata) Software
Hardware
Understanding
Data analysis
Data visualisation
Data treatment
Data diagnostics
Data acquisition
Measurement
Data curation Data (scientific)
Data (raw)
Data (publication)
Data (metadata)Fac
ility
User
Software
Hardware
The data chain
IdeasKnowledge
andUnderstanding
Exploitation(Economic Impact)
GRIDHPC
ES
S
XFE
LILL
ES
RF
IRU
VX
HPC
DA
TA
Time to Market
Proposal system
Fromsingle instrument, single experiment
tomulti-instrument + support facilities + computing
Damian Flannery, Tom Griffin, Hannah Guest
Experiment (controls, electronics, data formats)
Freddie Akeroyd (NeXus)
User Interface, DAE-III
Data catalogue, Data curation, Data Policy
Damian Flannery (ICAT)
Single sign-on
Data reduction/analysis – MANTID
Nick Draper
Data analysis – software ‘instrument scientists’
EPSR, Single crystal inelastic, RMC …
Data analysis – modelling/simulation
Simulation – source and instruments
Tom Griffin (GRID-MP)
SCARF (UK GRID) - CASTEP, Tobyfit
McStas
StructuresNIMROD Intermediate range order in liquidsLMX Large molecule single crystal diffractionZOOM Focusing small angle scatteringIMAT Imaging / diffraction
Structure/DynamicsLARMOR SESANS, High resolution diffraction, MIEZE, MISANS
TS2 Instruments Phase2
Software infrastructure
MANTID Bio-SANS
Single crystalImaging/enginering
Gateway Centres
Imaging Solutions Centre, Hartree Centre
Collaboration
European Data Infrastructure for Neutron and Photon Sources
STFC, ESRF, ILL, DLS, PSI, DESY, ELETTRA, SOLEIL, ALBA, BESSY
What will Mantid do?
Mantid– Manipulation and Analysis Toolkit for ISIS Data
Aims– To provide a framework for Data Analysis that is not
instrument or technique/dependent.– To support multiple target platforms (Windows, Linux).– The framework must be easily extensible by Instruments
Scientists/Users.– The framework must be freely redistributable to visiting
scientists.– The framework should provide low-level functionalities
for Scripting, Visualization, Data transformation, Implementing Algorithms, Virtual Instrument Geometry.
Top Requirements
Easily extensible.– Support all current and future analysis.– Support current and future file formats.– Provide a simple but powerful objects and
services to support user created algorithm code.No user license costs.Supportable.Portable.
– Operating System (Windows, Linux).– Computing Power (Laptop, Server).
Architectural Design - Overview
Mantid Framework
Command line & Scripting interface
QTIPlot Graphing tool
RAW data files
NEXUS data files
Future Data analysis GUI
Instrument log files
APIStandard
Algorithms
User Defined Algorithms
Matlab Interface
DAE direct access
Development Process
Iterative development – Why?– Adaptability - the ability to rapidly respond to
changes in strategy, priorities, and plans – Value - continuous delivery of more useful
functionality – Visibility - stakeholder collaboration and
validation throughout the development life-cycle
– Risk - the reduction in overall project risk as a result of #1-3 above
Software led by Scientists
This is project is intended to support the needs of all of the beamlines.
Scientific Steering Committee– To ensure input from each group of instruments
• To ensure what we build fits what you need.• To ensure good communication between the
Mantid project and the scientists.• To allow each group to have equal input into the
project.– Prioritise the list of tasks to be done over the next
iterations.– Review the progress of each iteration.
Milestones– Architecture design completed October 2007
• Completed on time
– Prototype framework released: March 2008 • Released to a limited audience in May 2008• Purposefully delayed to allow for additional
functionality
– Functional framework + 2-3 complete applications released March 2009
Timeline
What Mantid can do now
Data Input/Export– Read Raw files and associated log files– Read Instrument definition files (Mantid XML format)– Nexus files (limited support – data only)
Internal Data Structures– 1D and 2D workspaces– Managed 2D workspaces for large volumes of data
Instrument Definition– Sufficient definition to allow conversion of units– Loading instrument definition from XML files or RAW files
What Mantid can do now
Algorithms– Simple binary mathematical operations
• +, -, *, /– Integration– Rebin– ConvertUnits
• TOF, , energy, d-Spacing, q, Q2, Energy transfer (meV and 1/cm)
– Grouping of detectors– Marking detectors as dead
User Interfaces
Early Prototypes– QTIPlot-Mantid
• A graphical plotting tool which interfaces directly to Mantid for data interaction and analysis.
– Python• Command line access and scripting
Proof of concept– Matlab Interface
QTIPLOT-MantidLoading a Raw File
QTIPLOT-MantidContour and 3D Plots
QTIPLOT-MantidLine graphs of individual detectors
Matlab Interface