www.lightsource.ca

Science Studio and ANISE(Jan 2010 update)

Elder MatiasCanadian Light Source

www.lightsource.ca

Requirements

• New User Office Functionality– Proposal submission– Peer review– User Feedback Tracking– Experiment Management– User Training/ Safety Testing

• Remote Beamline Access• Integration with grid data-storage• Grid computing

www.lightsource.ca

Science Studio and ANISE

ProcessingService

DataServiceData

ServiceProcessing

Service

ANISE

Business ModelLayer

PersistenceLayer

DeviceProxies

Client ServicesLayerBrowser

BrowserLaboratory

Control Module

ServiceProxies

Science Studio

LabatoryControl Module

IOCs

Beamline

IOCs

Beamline

General, commonComponent

XRD Processing

XRF Processing

Science Studio specificComponent

www.lightsource.ca

System Architecture

WebApplication

BeamlineControlModule

DB SAN

JMS CA

VESPERS

HTTP

1. VESPERS Beamline2. EPICS control system3. Beamline Control Module (BCM)4. Web Application5. Database6. File Storage7. Web Interface

www.lightsource.ca

VESPERS Beamline

• VESPERS — Very Sensitive Elemental and Structural Probe Employing Radiation from a Synchrotron

• A bending magnet beamline on sector 6 at the Canadian Light Source synchrotron in Saskatoon, Saskatchewan.

• A hard x-ray microprobe with an energy range of 6 to 30keV.

• Techniques: X-Ray Fluorescence (XRF) & X-Ray Diffraction (XRD)

WebApplication

BeamlineControlModule

DB SAN

JMS CA

VESPERS

HTTP

www.lightsource.ca

EPICS Low-level Control System

• EPICS — Experimental Physics and Industrial Control System

• The standard control system at the CLS.

• EPICS consists of a network of Input-Output Controls (IOCs) which are connected to directly to devices.

• An IOC provides many Process Variables (PVs) which relate to either an input or output from a device and have a unique name.

• Channel Access (CA) is used to read or write to any PV without knowing which IOC provides the PV.

• More than 50,000 PVs in the CLS control system.

WebApplication

BeamlineControlModule

DB SAN

JMS CA

VESPERS

HTTP

www.lightsource.ca

Beamline Control Module (BCM)

• The BCM provides a high-level interface to the low-level control system (EPICS).

• Logical and physical separation of business logic and control logic.

• Virtual device abstraction that provides independence from low-level control system.

• Virtual devices can be logically organized into a device hierarchy.

• Basic devices can be combined to build more functional devices.

• Communication with external applications using two message queues (ActiveMQ).

WebApplication

BeamlineControlModule

DB SAN

JMS CA

VESPERS

HTTP

www.lightsource.ca

Web Application

• A J2EE Servlet application that provides a web-based interface Science Studio.

• Tools: Spring (MVC), iBATIS (ORM), JSecurity (Apache Ki), Apache Tomcat

• Divided into two parts: the Core application and the VESPERS beamline application.

• Core application is responsible for providing access to the business objects.

• VESPERS application is responsible for remote control of the VESPERS beamline.

WebApplication

BeamlineControlModule

DB SAN

JMS CA

VESPERS

HTTP

www.lightsource.ca

Database

• Metadata associated with the operation of a remote controlled beamline and the organization of experimental data collected on that beamline.

• A project is the top level organizational unit and is associated with a project team.

• A session defines a period of time allocated to a project team to conduct experiments.

• An experiment relates a sample and the technique being applied to that sample.

• A scan records the location of the acquired experimental data.

WebApplication

BeamlineControlModule

DB SAN

JMS CA

VESPERS

HTTP

www.lightsource.ca

Experimental Data Storage

• Experimental data is stored at the CLS.

• Common directory structure shared with other beamlines. • A large data storage facility is now operational at the University of

Saskatchewan as part of WestGrid.

WebApplication

BeamlineControlModule

DB SAN

JMS CA

VESPERS

HTTP

www.lightsource.ca

VESPERS Web Interface

• Rich web interface to Science Studio and the VESPERS beamline.

• Designed to be used over commodity broadband internet.

• Developed for the Firefox web browser without any additional plugins or extensions.

• Known to work with other browsers, but requires the Canvas HTML tag.

• AJAX is used for the VESPERS interface to provide device values in pseudo real time.

• ExtJS, a JavaScript framework, provides many advanced GUI elements.

WebApplication

BeamlineControlModule

DB SAN

JMS CA

VESPERS

HTTP

www.lightsource.ca

www.lightsource.ca

Beamline Setup

www.lightsource.ca

Experiment Setup

www.lightsource.ca

XRF (X-Ray Fluorescence)

www.lightsource.ca

Beamline Hutch Cameras

www.lightsource.ca

Experimental Data Viewer

www.lightsource.ca

www.lightsource.ca

ScienceStudio

www.lightsource.ca

Scanning

www.lightsource.ca

XRD Screen

www.lightsource.ca

(a) X ray Fluorescence spectra and mapping

(b) Laue X ray Diffraction and mapping Orientation Strain

www.lightsource.ca

VM strain

XRF Image XRD: orientation XRD:strain

Optical image

www.lightsource.ca

University of Western OntarioNan-Fabrication Facility

Using ScienceStudio

www.lightsource.ca

Nanofabrication Facility The Western Nanofabrication Facility is an open user facility at the

University of Western Ontario for the fabrication of micro and nano structures.

LEO 1540XB Scanning Electron Microscope (SEM) with an integrated Oxford x-ray system.

Oxford x-ray system is controlled by INCA.

INCA provides limited control of the SEM, and spectral analysis and quantification.

www.lightsource.ca

Nanofab Integration Design

www.lightsource.ca

Nanofab Implementation

www.lightsource.ca

Initial Test: INCA Remotely

www.lightsource.ca

User Office

www.lightsource.ca

User Office Modules• User Profile/Authentication

– User Profile Deployed– User Roles Deployed– Authentication (LDAP/AD) Deployed

• Information Model– Beamlines Done– End-stations Done– Techniques Done

• Notification Module Deployed

www.lightsource.ca

User Office Modules

• Proposal Submission In Progress• User Feedback Survey Prototype• Scheduling Module Prototype• Workflow Engine Evaluation• Sample Management

Requirements• Publication Tracking Requirements

www.lightsource.ca

User Office - Training Module• First User Office Module Deployed• User Profile

• 1,400 + user records/accounts created• Training Module

– 53 Training Modules Setup and past user records manually transferred

www.lightsource.ca

Calendar view in JQuery

www.lightsource.ca

User Office Workflow Example

Prototype Implementation1. CLS issues a call for proposals and gives deadline2. Beamline users submit proposals3. User Office administrator ends registration or extends deadline4. User Office administrator assigns proposals to user office reviewers5. Reviewers look at proposals and rank them6. User Office looks at ranking and chooses the proposals to accept7. Accepted proposals contact persons are notified8. Beamline User completes training (web service)9. After training is completed (simulated by a delay) the CLS is notified

www.lightsource.ca

User Login

www.lightsource.ca

Admin – Add Proposal

www.lightsource.ca

Admin - Setup for Cycle

www.lightsource.ca

Admin – Creating Roles

www.lightsource.ca

Grid Computing

www.lightsource.ca

HPC Computing in Canada

• Compute Canada– Responsible for High Performance Computing in Canada– Each regional grid is a member of Compute Canada

• ACEnet – Atlantic Canada• CLUMEQ - Quebec• SCINET - UofT• HPCVL – Queens, Royal Military Collage

St. Lawrence, Carlson, Ottawa, …• RQCHP - Quebec• SHARCNET - Ontario• WESTGRID – Western Canada

CLUMEQ Facility

www.lightsource.ca

Grid Data Storage?

• UofS is the host for the new WestGrid data storage facility

• Cost: $3.2 M• Includes on-line and

archival storage• Two sites on campus

Photo: tape backup unit holding 6,000 tape (each @1TB)

www.lightsource.ca

IBM Cell Processor (3.2 GHz)

www.lightsource.ca

www.lightsource.ca

User Configurable Light Paths

www.lightsource.ca

Lightpath Accelerator controls a software virtual cross-connect that commands UCLP.

Integrating LPA with User Configurable Light Paths

In effect, CA*Net4 is treated as a single lightpath cross-connect

real device real devicevirtual device

www.lightsource.ca

File Transfer Demo