Post on 24-Dec-2015
Elettra Virtual Collaboratory: the evolution of a Virtual Laboratory Software from a simple web application to the GRIDCC
Roberto Pugliese, Alessandro Busato, Alessio Curri, Enrico Mariotti, Daniele Favretto, Fulvio Billè, Roberto Borghes, Fabio Asnicar, Valentina Chenda,
Laura Del Cano, Lawrence Iviani, Michele Turcinovich
and the GRIDCC collaboration
Sincrotrone Trieste – ELETTRAInstruments and Sensors on the Grid
IEEE Conference on e-Science and Grid ComputingMelbourne, Australia, on 5-8 December 2005
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
Outline The Elettra Virtual Collaboratory (EVC)
EVC @ Work The BIOXHIT project
Virtual Collaborative System (VCS) The EUROTeV project
The Global Accelerator Network The Multipurpose Virtual Laboratory (MVL)
Evolving EVC to meet VCS and MVL requirements The GRIDCC project
The Multipurpose Collaborative Environment (MCE) Evolving EVC with MCE
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
What is a Collaboratory? The term “collaboratory” was coined by William
Wulf by merging the words collaboration and laboratory, and defined as “... Center without walls, in which researchers can perform their research without regard to geographical location - interacting with colleagues, accessing instrumentation, sharing data and computational resource, and accessing information in digital libraries”.
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
What is a Collaboratory? In particular, the core capabilities that constitute a
collaboratory can be seen as technologies to link: People to people (e.g., electronic mail, and tools for data
conferencing, such as VRVS) People to information (e.g., the World Wide Web and digital
libraries) People to facilities (e.g., status of remote instruments) to
enhance utilization by expanding access to resources
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
What is the Elettra Virtual Collaboratory (EVC)?
EVC is an example of virtual laboratory, a system which allows a team of researchers distributed anywhere in the world to perform a complete experiment on the equipped beamlines and experimental stations of Elettra.
User atELETTRA
Team Memberat Home Lab
RemoteCollaborator
EquipmentControl Data CPU
COLLABORATORY
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
What is Elettra?
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
EVC usage scenarios:Cristallography “by mail” EVC allows biologists to send by mail protein crystals
which will be ananlized at the Xray Diffraction beamline by the beamline staff.
Collected data and resultsare accessible via EVC andresults can be downloadedas soon they are available.
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
EVC in action: a web portal
EVC is based onthe “web portal”metaphor
All you need is abrowser
EVC supports four different usercategories: Visitors Normal users Project leaders Staff
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
Collaborating to an EVC project
Scientists workingto an EVC projectcan use manyproject relatedcollaborationtools
EVC presents anadaptive interfacechanging to suitethe categoryand expertiselevel of the user
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
Collaboration Tools: EVC chat
EVC chat is“project centered”:there is a differentchannel for eachproject
Usual chat featureare extended inorder to allowexchange of Drawings scientific images graphical
annotations
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
Collaboration tools:scientific visualisation
Scientists canbrowse, visualiseand process remotelyscientific datain real-timeas soon asthe data is collected
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
Collaboration Tools: telepresence
The different videostreams of the equipped experimental stationscan be selected and viewed eventhrough a slowconnection
Movable camerascan be controlledvia web by theproject leader
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
Collaboration tools remote computing
Legacy software isnormally not web enabled
EVC uses VNCto web enablelegacy apps. It is small and
simple, sharableand open
Can be tunnelledvia ssh
VNC can be used asa fast integration tool
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
Collaboration Tools: Remote Beamline Control and Supervision
Beamwatchpresents asynoptic viewof the beamlines
Autorised peoplecan thus operateremotely onthe beamlineintrumentation
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
EVC Architecture EVC has 2 main components:
the application server and
a set of nodes
The application server is running the portal application, the user and project database; the application server activates actions implemented by agents running in the nodes or requests services to external systems
node1 node2 nodek-1 nodek
application server DB
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
EVC facts EVC project started on June 2001 and finished on June 2003.
The first prototype was installed on the Xray Diffraction beamline of ELETTRA on June 2002
EVC is now operating on all the beamline and experimental station of ELETTRA and acts as the web interface to the Elettra Scientific Computing Environment (instruments, computing farms, storage)
EVC was presented at SMAU2002, NOBUGS2002 and SMAU2003, HCI2003, NOBUGS2004
EVC development staff is partecipating in manyEU founded projects under FP6(BIOXHIT, IA-SFS/JRA1, GRIDCC, EUROTeV/GAN)
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
BIOXHIT Virtual Collaboratory System
The BIOXHIT project which will develop an integrated platform for high-throughput structure determination
ELETTRA is developing the Virtual Collaboratory System a Virtual Organization (VO) connecting all the European laboratories doing research in the field of structural genomics.
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
BIOXHIT VCS scenario
SupportingServices
VO user
VO user
crystallisation
beamline 1
beamline 2
beamline 3
ProcessingFarm
ProcessingFarm
DataStorage
VCS will be used to implement a widley distributed Virtual Organisation (VO) connecting all the stations and Laboratories involved in the BIOXHIT project
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
VCS Node @ EMBL-Hamburg
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
EUROTeV/GANMVL In the EUROTeV project the design study of the
International Linear Collider ELETTRA is developing the Multipurpose Virtual Laboratory, the core tool to implement the Global Accelerator Network, a VO connecting all the international laboratories doing research in the field of Accelerators.
Remote control of an accelerator facility has the potential of revolutionizing the mode of operation and the degree of exploitation of large experimental physics facilities.
The first prototype of the system planned by April 2005 will allow the remote control of ELETTRA storage ring from DESY.
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
MVL @ Work
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
MVL @ Work
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
www.lightsources.org
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
VCS/MVL Virtual Organisation
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
Evolving EVC to implement VCS / MVL In this architecture every institute in the VO should have a
VCS Node. The node can support more stations (e.g. a data collection station, a cristallisation, a control room station, a movable station etc). Stations can share resources and tools.
The remote collaborator will use his PC equipped with a web browser and if the case with a projector.
All the communication (AS-to-AS and AS-to-LN) is done via webservices secured with X.509 certificates by mutual authentication
Legacy applications are integrated using VNC if they do not have a web interface or ssh tunnels and proxy or redirection if they already have a web interface.
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
Evolving EVC to implement VCS / MVL All the communication between the Application Servers
located in the distributed laboratories is done via webservices (Axis implementation).
All the Local Nodes run a Local Node Server. Comunication between the AS and the LN is done via webservices (gSOAP implementation).
The systems are also equipped by a Management Station which allows easy configuration and maintenance via web browser.
VCS and can be considered a sort of integration platform. Scripts stored in the AS database are transferred to the local nodes, executed and the results returned to the user via the AS.
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
The GridCC Project
Instruments Grid Computational Grid+
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
GRIDCC project in 3 steps Development of generic Grid middleware, based on existing building
blocks (Grid Services) which will allow the remote control and monitoring instrumentation such as distributed systems.
Testing of the middleware on challenging applications to validate it both in terms of functionality and quality of service: European Power Grid Geo-hazards Remote Operation of an Accelerator Facility High Energy Physics Experiment …
Dissemination of the new software technology to encourage a wide range of enterprises to evaluate and adopt our Grid-oriented approach to real-time control and monitoring of remote instrumentation.
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
The MCE and the GRIDCC landscape
SupportingServices
VirtualCtrl. Room
VirtualCtrl. Room
Diagnostics
Instrument 1
Instrument 2
Instrument 3
The MCE is a software to implement Virtual Control Rooms, i.e., multi-user, collaboration-supporting interfaces to a widely distributed control system with access to grid-enabled computing and data storage facilities
StorageStorageElementElement
ComputingComputingElementElement
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
GridCC ArchitectureInformation
MonitorService
ComputingElement
ComputingElement
ComputingElement
StorageElements
StorageElements
StorageElementGlobalProblemSolver
Virtual Control Room
Virtual Control Room
Security
Service AutSTGSPolR
Instrument
Element
Instrument
Element
Instrument
Element
Collaborative
Service
Exe
c.
Ser
vice
WfM
SW
MS
Agr
S
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
The Multipurpose Collaborative Environment
a groupware providing general purpose services and interfaces to support collaboration among researchers and operators, to control remote instrumentation, and other tasks related with experimental activities
will be used to implement the VCR for the different GRIDCC pilot applications through customization and integration with application-specific services.
based on a core groupware application (providing, e.g., authentication, management of the VO users and instruments) and a set of plug-ins: General purpose (e.g., chat, notebook, video conference) Specific to the particular application (e.g., accelerator control,
specific instrument control)
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
Design and Development Approach State of the art evaluation
Collaboration support tools for scientific experimental activities
Available technologies Derivation of general requirements
through use cases, interviews, exchanges with other related projects (e.g., EUROTeV)
Prototyping Discussions over interface sketches Incremental development of functional prototypes
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
Design Choices Web-based portal interface
no installation, porting, clients available almost anywhere Minimal system requirements: Web browser, JRE for applets
Exploit portlet technology Why? manageable integration of application-specific
functionalities within the MCE Current prototypes are based on a modified version of the
GridSphere framework, developed under the GridLab EU project Collaboration with the GridSphere team to extend and improve
the framework to our needs was established and some of our developments are already in GridSphere codebase
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
Login page
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
Personalisation
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
eLogbook
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
VO view: people browser and chat
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
Resource browser and Instrument control
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
More Instruments Control
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
Video/Phone Conference: VRVS/Skype
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
Desktop Sharing
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
Execution Services: job submission
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
Execution Services: File Access
Roberto Pugliesepugliese@elettra.trieste.it
Instruments and Sensors on the GridMelbourne, Australia, December 2005
Evolving EVC with MCE EVC project started in 2001 and has now more
than 4 years of operations EVC software has been improved in these years
moving from a simple single facility web application to a multi-facility integration platform based on webservices
We are currently refactoring EVC in order to migrate to the GRIDCC MCE middleware.