JLab Lattice Portal – JLab Lattice Portal – Data Grid Web ServiceData Grid Web Service

Ying Chen, Chip WatsonThomas Jefferson National Accelerator Facility

Jlab Lattice PortalJlab Lattice Portal

Jefferson Lab (JLab) and its collaborators at MIT are in the process of developing a web portal for the Lattice Hadron Physics Collaboration.

This portal will eventually provide access to Linux clusters, disk caches, and tertiary storage located at JLab, MIT, and other universities.

Jlab Lattice PortalJlab Lattice Portal

Jlab Lattice Web Portal infrastructure capabilities:

- Publish/locate a data set.

- Submit/monitor/control a batch job.

- Move a data set to/from the compute resource.

- Authenticate/authorize and track the use of the resources.

- ...

Why Grid Web Service ?Why Grid Web Service ?

Disk Mgr

Tape Mgrusers

Managed diskUnmanaged disk

tertiary storageusers

users

login

login

login

Data Grid Node

Fire wall

Data Grid Web ServiceData Grid Web Service

Hides the complexity of the underlying systemusers don’t have to deal with the disk manager or tape manager etc.

Provides the service on the webusers don’t have to login to do the work

System Architecture OverviewSystem Architecture Overview

Grid Service

Disk Mgr

Tape Mgr

users

Managed disk

Unmanaged disk

tertiary storage

users

users

Web browser

web browser

Fire wall

Web Based Grid ServiceWeb Based Grid ServiceJlab Data Grid Web Service - a java

servlet

- uses apache with model-ssl as secure web server and tomcat as servlet engine

- service request is an xml format string

- the result/response is also in xml format

- a certificate is required for the privileged services

Service Request/ResponseService Request/ResponseRequest:

<info name="version“ />

Response:<grid name="LQCD at Jlab“ url="lqcd.jlab.org/servlet/grid.data.Service">

<info name="version"> <info>jlab data grid version '0.1'</info>

</info></grid> ---------------------- or -------------------<grid name=“LQCD at Jlab” url=“lqcd.jlab.org/servlet/grid.data.Service”>

<info nam=“version”> <error>Attribute “name” missing</error></info>

</grid>

Why xml ?Why xml ? Self-describe (easy to understand) Language independent easy to assemble from multiple source Interchange of data among application Focus upon the data structure (can easily

convert to different presentation) Many free or commercial tools available to

process xml data

http://lqcd.jlab.org/pbs/

Jlab Web Based Grid ServiceJlab Web Based Grid Service

The grid service includes: - provides node information and status

- lists the contents of the grid node - moving files within the node

- performs file operations (delete, pin/unpin, mkdir, mkdirs etc.)

The URL of the service: http://lqcd.jlab.org/servlet/grid.data.GridService

Use the Service - directlyUse the Service - directly

Enter the url in a web browser:http://lqcd.jlab.org/servlet/grid.data.GridService

?service=<list+name='$CACHE/ychen'/>

or

https://lqcd.jlab.org/servlet/grid.data.GridService?service=<file-operation+type=‘delete’+ name=‘$DATA/foo'/>

Use the Service - Grid APIUse the Service - Grid API

A grid interface API is designed for using this Data Grid web service.

A java implementation of this API is being developed to encapsulate this xml service.

Using this API to obtain the service, the xml input and output are hidden by the implementation.

Grid API - GridFactoryGrid API - GridFactorypublic class GridFactory– provides an

interface to access the GridNode object using static methods.

public static GridNode getGridNode (String url)

public static GridNode getGridNode ()public static GridNode getReplica ()

interface GridNode – Is a grid node or replica catalog object that can be obtained from the GridFactory class.

Grid API - GridNodeGrid API - GridNodepublic interface GridNode {

public String getURL ();

public String getName();

public GridDirectory getDirectory (String path);

public GridFile[] list(String path);

public String[] listNames (String path);

public boolean isExist (String path);

public boolean isDirectory (String path);

public boolean isFile (String path);

}

Use the Grid APIUse the Grid APIpublic class GridAPIDemo { public static void main (String[] args) { try { GridNode jlab = GridFactory.getGridNode(); NodeDirectory dir = (NodeDirectory)

jlab.getDirectory(args[0]); String[] names = dir.listNames();

System.out.println(args[0]+“ directory contains:”);for (int i=0; i<names.length; i++) System.out.println(names[i]);

} catch (Exception e) { System.out.println(e.getMessage()); } }}

Future DevelopmentsFuture Developments

Standardize the xml request and response. Add additional web technology (SOUP,

WSDL, and UDDI) to build dynamic web-based system

Extend the batch web interface/service. Add third party file transfer capability using

remote web services (web server to web server)

Questions ?Questions ?