Post on 02-Apr-2015
JLab Lattice Portal – JLab Lattice Portal – Data Grid Web ServiceData Grid Web Service
Ying Chen, Chip WatsonThomas Jefferson National Accelerator Facility
Jlab Lattice PortalJlab Lattice Portal
Jefferson Lab (JLab) and its collaborators at MIT are in the process of developing a web portal for the Lattice Hadron Physics Collaboration.
This portal will eventually provide access to Linux clusters, disk caches, and tertiary storage located at JLab, MIT, and other universities.
Jlab Lattice PortalJlab Lattice Portal
Jlab Lattice Web Portal infrastructure capabilities:
- Publish/locate a data set.
- Submit/monitor/control a batch job.
- Move a data set to/from the compute resource.
- Authenticate/authorize and track the use of the resources.
- ...
Why Grid Web Service ?Why Grid Web Service ?
Disk Mgr
Tape Mgrusers
Managed diskUnmanaged disk
tertiary storageusers
users
login
login
login
Data Grid Node
Fire wall
Data Grid Web ServiceData Grid Web Service
Hides the complexity of the underlying systemusers don’t have to deal with the disk manager or tape manager etc.
Provides the service on the webusers don’t have to login to do the work
System Architecture OverviewSystem Architecture Overview
Grid Service
Disk Mgr
Tape Mgr
users
Managed disk
Unmanaged disk
tertiary storage
users
users
Web browser
web browser
Fire wall
Web Based Grid ServiceWeb Based Grid ServiceJlab Data Grid Web Service - a java
servlet
- uses apache with model-ssl as secure web server and tomcat as servlet engine
- service request is an xml format string
- the result/response is also in xml format
- a certificate is required for the privileged services
Service Request/ResponseService Request/ResponseRequest:
<info name="version“ />
Response:<grid name="LQCD at Jlab“ url="lqcd.jlab.org/servlet/grid.data.Service">
<info name="version"> <info>jlab data grid version '0.1'</info>
</info></grid> ---------------------- or -------------------<grid name=“LQCD at Jlab” url=“lqcd.jlab.org/servlet/grid.data.Service”>
<info nam=“version”> <error>Attribute “name” missing</error></info>
</grid>
Why xml ?Why xml ? Self-describe (easy to understand) Language independent easy to assemble from multiple source Interchange of data among application Focus upon the data structure (can easily
convert to different presentation) Many free or commercial tools available to
process xml data
http://lqcd.jlab.org/pbs/
Jlab Web Based Grid ServiceJlab Web Based Grid Service
The grid service includes: - provides node information and status
- lists the contents of the grid node - moving files within the node
- performs file operations (delete, pin/unpin, mkdir, mkdirs etc.)
The URL of the service: http://lqcd.jlab.org/servlet/grid.data.GridService
Use the Service - directlyUse the Service - directly
Enter the url in a web browser:http://lqcd.jlab.org/servlet/grid.data.GridService
?service=<list+name='$CACHE/ychen'/>
or
https://lqcd.jlab.org/servlet/grid.data.GridService?service=<file-operation+type=‘delete’+ name=‘$DATA/foo'/>
Use the Service - Grid APIUse the Service - Grid API
A grid interface API is designed for using this Data Grid web service.
A java implementation of this API is being developed to encapsulate this xml service.
Using this API to obtain the service, the xml input and output are hidden by the implementation.
Grid API - GridFactoryGrid API - GridFactorypublic class GridFactory– provides an
interface to access the GridNode object using static methods.
public static GridNode getGridNode (String url)
public static GridNode getGridNode ()public static GridNode getReplica ()
interface GridNode – Is a grid node or replica catalog object that can be obtained from the GridFactory class.
Grid API - GridNodeGrid API - GridNodepublic interface GridNode {
public String getURL ();
public String getName();
public GridDirectory getDirectory (String path);
public GridFile[] list(String path);
public String[] listNames (String path);
public boolean isExist (String path);
public boolean isDirectory (String path);
public boolean isFile (String path);
}
Use the Grid APIUse the Grid APIpublic class GridAPIDemo { public static void main (String[] args) { try { GridNode jlab = GridFactory.getGridNode(); NodeDirectory dir = (NodeDirectory)
jlab.getDirectory(args[0]); String[] names = dir.listNames();
System.out.println(args[0]+“ directory contains:”);for (int i=0; i<names.length; i++) System.out.println(names[i]);
} catch (Exception e) { System.out.println(e.getMessage()); } }}
Future DevelopmentsFuture Developments
Standardize the xml request and response. Add additional web technology (SOUP,
WSDL, and UDDI) to build dynamic web-based system
Extend the batch web interface/service. Add third party file transfer capability using
remote web services (web server to web server)
Questions ?Questions ?