The DAWNSci Eclipse science

usecases

Jun Aishima

Software Engineer

Diamond Light Source

Didcot, UK

1. What is a light source and why do science there?

2. What data do you get at a synchrotron?

3. What do you do to the data?

4. How do DAWN and DAWNSci Eclipse help with this?

5. How do we use DAWN and DAWNSci Eclipse to analyze our data?

6. How can you use DAWN and DAWNSci Eclipse to analyze your data?

SynchrotronAKA – cool word, but what does it

mean? syn·chro·tron/ˈsiNGkrəˌträn/

Noun:A cyclotron in which the magnetic field strength increases with the energy of the particles to keep their orbital radius constant.

“They are machines which produce very strong light used for many different type of scientific experiments and sometimes other things.”

Diamond Light Source, Didcot, Oxfordshire, UK

• (Video has been removed to allow the talk to fit into size guidelines – see http://www.diamond.ac.uk/Public/Multimedia/Animations.html to view the animation)

1. What is a light source and why do science there?

2. What data do you get at a synchrotron?

3. What do you do to the data?

4. How do DAWN and DAWNSci Eclipse help with this?

5. How do we use DAWN and DAWNSci Eclipse to analyze our data?

6. How can you use DAWN and DAWNSci Eclipse to analyze your data?

Detectors ofVarious flavors

1. What is a light source and why do science there?

2. What data do you get at a synchrotron?

3. What do you do to the data?

4. How do DAWN and DAWNSci Eclipse help with this?

5. How do we use DAWN and DAWNSci Eclipse to analyze our data?

6. How can you use DAWN and DAWNSci Eclipse to analyze your data?

…180 images

The collected data may look like this…

When what you really want is this…

1. What is a light source and why do science there?

2. What data do you get at a synchrotron?

3. What do you do to the data?

4. How do DAWN and DAWNSci Eclipse help with this?

5. How do we use DAWN and DAWNSci Eclipse to analyze our data?

6. How can you use DAWN and DAWNSci Eclipse to analyze your data?

DAWN - Acquisition and Analysis with RCP

@ Diamond Light Source

Client server technology

Communication withEPICS and hardware

Scan mechanism

www.opengda.org

Jythonand Python

Visualisation

Communicationwith external

analysis

Analysistools

Data read, write,convert

Metadata structure

Workflows

www.dawnsci.org

DAWN is a collection of generic and

bespoke ‘views’ collated into

‘perspectives’.

The perspectives and views can be

used in part or whole in either

the GDA or DAWN.

Acquisition Analysis

Collaborative Approach

Kichwa Coders Ltd. 

Science Working Group

External Users

Consultants

Members

Now an Eclipse project!https://projects.eclipse.org/projects/technology.dawnsci

‘Shoulders of Giants’

AKA – didn’t think of everything• RCP many of the core features, editors, toolbars, views, projects

• Ptolemy 2 (a version known as ‘Passerelle’) workflow and pipelining

• GEF for visualization of pipeline graphs

• Draw2D for 1D and 2D plotting (SWT XY Graph)

• Pydev for python/jython scripting layer used by the scientists

• HDF5 libraries for storing large data sets

• SWT/Jface – lazy viewers being used extensively for large trees and tables

• Apache, Eclipse-WST, springsource, JDK, and many more of course...

Did you see our poster yesterday, presented by Bahá?

• The DAWNSci Eclipse Project: How Eclipse Is Used for Science (Bahá El Kassaby)– Printed copies or PDF file available! Ask us!

Data in DAWNSci

Example : Loading and Manipulating Data

This example• Columns of ASCII data• Separated by tabs• Read as 1D arrays

Other format with a loader: srs, dat, flt, gff, mca, csv, xy, xye, txt, tif, tiff, cbf, img, ciff, mccd, edf, pgm, cor, bruker, jpg, jpeg, png, f2d, msk, mib, mar3450, pck3450, raw, mrc, gz, bz2, zip, h5, hd5, hdf5, nxs, nexus, hdf, mat

Extension point to add more...

Demonstration – Visual Tools

1. What is a light source and why do science there?

2. What data do you get at a synchrotron?

3. What do you do to the data?

4. How do DAWN and DAWNSci Eclipse help with this?

5. How do we use DAWN and DAWNSci Eclipse to analyze our data?

6. How can you use DAWN and DAWNSci Eclipse to analyze your data?

The Processing pipeline

• Previous systems– Required much effort by the developer– User experience was too generic – not optimized for this

purpose

• Simplify (coding), simplify (GUI), simplify (capabilities)

• Result: wide adoption and rapid development for several experimental techniques

• Developed by Jacob Filik at Diamond• Available in DAWN

Non-crystalline diffraction

Why Processing in DAWN?

Simple? Operation

Calculation

Errorhandling

Simple? Model

Allow drag and dropProvide a button to open file dialog box

Radio box in GUI

Number box in GUI with limits

Mouseover in GUI

Simple?

File list

Input

Output

List of processing steps

Model

Demonstration of pipeline

1. What is a light source and why do science there?

2. What data do you get at a synchrotron?

3. What do you do to the data?

4. How do DAWN and DAWNSci Eclipse help with this?

5. How do we use DAWN and DAWNSci Eclipse to analyze our data?

6. How can you use DAWN and DAWNSci Eclipse to analyze your data?

Create your own operations and pipelines!

• Create an operation – org.eclipse.dawnsci.analysis.api.operation extension point– Dataset operations – including lazy operations

• Create a model – if necessary– Any additional data required for the calculation– Reflection on data type and annotations are used to

build the GUI for users to enter parameters

A lot of existing, reusable operations

• Sector integration

– Typically, reduce a two-dimensional image into a one-dimensional dataset

• Normalization

– Remove frame-to-frame variation to put them all onto the same scale

• Subtraction

– Removing artifacts by subtracting background data from the sample data

• Averaging

– Multiple noisy points can be averaged

Future work on the Processing pipeline

• Support more sciences (XPDF)

• Add branching, merging, and other non-linear flows

• Run calculations via HPC

• Passerelle workflows

• More, faster, better processing steps

Try DAWN for yourself!

• DAWN 1.7.1, available now!

• http://www.dawnsci.org

• See directions on how to use DAWNSci – “Getting started with DAWNSci”

• Videos on how to do use slicing, data reduction, Python scripting, processing at https://www.youtube.com/user/DAWNScience

• Follow us on Twitter - https://twitter.com/dawnscience

• Read the article about us in the Eclipse newsletter - https://www.eclipse.org/community/eclipse_newsletter/2015/january/article3.php

• DAWN paper accepted for publication, doi: 10.1107/S1600577515002283

• Email us: scientificsoftware@diamond.ac.uk

+1 0 -1

Sign in: www.eclipsecon.org

Evaluate the sessions