Rob Allan

CCLRC

2nd Generation HPCPortal v3 for the NGS

Rob Allan and Dharmesh Chohane-Science Centre

CCLRC, Daresbury Laboratory

Presenter Name

Facility Name

1st Generation Portal Interfaces

HPCPortal

DataPortal

Presenter Name

Facility Name

InfoPortal, MDS Browser and Active Map

http://www.grids.ac.uk/InfoPortal

Operational monitoring: resource-centric and site-centric views of Grid resource and network status

Presenter Name

Facility Name

What are the Restrictions?

1st generation Grid portals have been focused on providing basic task-oriented services, such as user authentication, job submission, monitoring, data transfer. However, they are typically tightly coupled with Grid middleware tools such as Globus. The main limitations of first generation portals can be summarized as follows.

• Lack of Customization: Portal developers instead of portal users normally build portals because the knowledge and expertise required to use the portal toolkits. When end users access the Grid via a portal, it is almost impossible for them to customize the portal to meet their specific needs, e.g., to add or remove some portal services.

• Restricted Grid Services: First generation Grid portals are tightly coupled with specific Grid middleware technologies such as Globus, which results in restricted portal services. It is hard to integrate Grid services provided by different Grid middleware technologies via a portal of this generation.

• Static Grid Services : A Grid environment is dynamic in nature with more and more Grid services are being developed. However, first generation portals can only provide static Grid services in that they lack a facility to easily expose newly created Grid services to users.

Presenter Name

Facility Name

2nd Generation OGCE Portal

Presenter Name

Facility Name

NMI Portal and OGCE

With funding from the US National Science Foundation Middleware Initiative (NMI), the Open Grid Computing Environments (OGCE) project was established in autumn 2003 to foster collaborations and sharable components with portal developers worldwide. Tasks include the establishment of a Grid Portal Collaboratory, a repository of portlets and portal service components, an online forum for developers of Grid Portals, and the building of reusable portal components that can be integrated in a common portal container system.

The development of OGCE is based on the following projects:• Java CoG Kit sponsored by SciDAC and NSF Alliance;• The CHEF Project of the University of Michigan;• The Grid Portals Information Repository (GPIR) and GridPort of

the Texas Advanced Computing Center;• The Alliance Portal Expedition project, including NCSA, Indiana

University's Extreme Labs, and the Community Grids Lab at Indiana University.

Presenter Name

Facility Name

Alliance Grid Portal Services

The Alliance Portal is an ongoing project that focuses on building second-generation Grid portals. It is based on Jetspeed and targeted at the construction of Grid portals using Grid Portlets. Currently the Alliance Portal can provide the following Grid Portlets that are leveraged from existing Grid services.

• A Proxy Manager: portlet utility that allows users to load GSI proxy credentials into their account via MyProxy.

• An LDAP Browser: portlet interface to access the contents of the LDAP servers.

• A GridFTP Client: portlet provides the basic client functions of Grid FTP with a user-friendly interface.

Presenter Name

Facility Name

More…

• Gram Job Launcher: portlet allows a user to submit jobs to a Grid environment using the Globus GRAM protocol. For this the user must have a valid GSI Proxy Certificate which can be loaded through the Proxy Manager portlet.

• Grid Utilities: include the "Grid Ping" utility and the "Grid Job Submission" utility. The GridPing portlet can ping a resource in a Grid environment and determine if a user has the access to it or not. The Grid Job Submission portlet is similar to the Gram Job Launcher portlet except that this portlet will not return till the job has completed. The output and the error of the job is displayed by the portlet.

• OGSA Browser: portlet alows users to query a Grid service for its Service Data Elements (SDE). Users can query a SDE by using the name of the SDE as the query string. Users can also obtain a list of SDEs by using "serviceDataName" as the query string. Once users get a list of all the SDEs, they can click on each SDE to query it.

Presenter Name

Facility Name

Collaboration Tools

Collaboration and content management tools come as default:• Schedule - calendar for work site

– If you are a member of multiple work sites your personal calendar will be updated with all appointments

• Resources - resource list and uploaded documents and links• News - news service using RSS feed

– We have created a prototype news feed and web service• Discussion - threaded discussion tool• Chat - online chat room for group• Web – link to ordinary Web site

Uses CHEF content-management capabilities ported to PostgreSQL

We previously tried using PHPNuke to provide similar functionality in HPCPortal and InfoPortal v2. This is still being used by the e-Minerals project.

Aim is to enhance community participation by providing suitable tools.

Presenter Name

Facility Name

Content Resource Tool

Example of Sakai tool from ReDRESS project.

Presenter Name

Facility Name

Other tools from SDSC/ UTexas/ Indiana

• Anabas - join and participate in Anabas Impromptu collaborative session via applets (requires a license)

• Newsgroup (Read/ post) - topic-based threaded newsgroup tool using Indiana XML schema– Newsgroup (Request) - request a new topic– Newsgroup (Admin) - for admins only!

• Bibtex (Read/ edit) - add/ edit entries in Bibtex topics for shared authorship of reference lists. Uses Indiana XML schema– Bibtex (Request) - request new Bibtex topic– Bibtex (Admin) - for admins only!– Bibtex (Super Admin) - for super admins only!

• GridPing - test a remote Grid service• GridContext - browse and manage DNs in Xdirectory server - provides context

services for managing metadata• GPIR Browser - browse Grid resources via Grid Port Information Repository

giving information on HPC status, jobs, etc.– Historical GPIR - use GPIR to view stored information

• CSF Job Submission - submit job to a Community Scheduling Framework service

• Sequencer - ditto via GridPort sequencing service• OGRELauncher - Open Grid Computing Runtime Engine: launch simulation

event via OGRE from NCSA, workflow based on ANT or BPEL– OGREEventView - monitor OGRE events

• COG-Workflow - use workflow enactment via the Java CoG kit• App Manager - register, de-register and edit information about a Grid

application• Condor - submit jobs via Condor or DAGMan and monitor them

Presenter Name

Facility Name

Work sites for groups/ projects

User and group management• Login/ Logout

– as it says on the tin• Set up new work site

– Only if enabled by admin• Membership

– join/ unjoin from work site (admin can control permissions)

• Customize– chose which tools you

want in the work site from set provided

Portal

Work sitesMy workspace

Tools

Portlet views

Some projects could have their own work site to customise and use the collaboration tools.

Need to decide how we manage this capability.

Presenter Name

Facility Name

Why not simply deploy OGCE?

We have started with CHEF and will bring in the tools we need

• OGCE is a large Java stack, some tools not appropriate• Need to import UK-specific tools from existing projects• Learn to write portlets and do customisation• Link into Web services• Work with new projects, e.g. VRE developers• In future: adopt JSR-168 and WSRP standards

Presenter Name

Facility Name

Some Project Requirements

• e-HTPX – Customising OGCE tools and will move into NGS framework

• Integrative Biology– Have a VRE project funded to use OGCE. Will require their

own portal instance as they support “experiments” as work sites

• MyGrid – Currently developing portlet interfaces to their tools

• RealityGrid and GeneGrid– Previously used GridSphere – will move portlets into new

framework?• e-Minerals

– Continue to use bespoke solution for time being

What will others do?What other tools do users want?

InfoPortal tools?VRE tools will be coming…

Presenter Name

Facility Name

Hosting the Portals

• NGS, e-HTPX and e-Minerals portals hosted on IBM HS20 BladeCenter at Daresbury

• NGS portal aliases:– http://ganges.dl.ac.uk:8080– http://ngsportal.ac.uk – http://www.grids.ac.uk/ngsportal

• Development system for testing on 3rd blade• CVS into dedicated server, RAID and Tivoli tape store at

Daresbury• Backup facilites to ATLAS tape store at RAL• Mirroring PostgreSQL DB is planned

Presenter Name

Facility Name

We are currently exploring the “best of breed” 2nd generation portal –Sakai http://www.sakaiproject.org

• Originated from U. Michigan (the CHEF project) with partners at U. Indiana, MIT, Stanford.

• Mellon Foundation Funding with OKI standard interfaces• Rich set of Collaboration and Educational tools with templates

for tool creation• We have proposed using this as a VRE “front end” by extending

the framework to a Service Oriented Architecture• Evaluation exercise funded by JISC

http://tardis.dl.ac.k/ReDRESS/sakai_doc.pdf• Using in the ReDRESS project http://redress.lancs.ac.uk We will be:• Using technology: JSR-168 to create plug-in portlets• Using technology: WSRP to link to remote portlets and content• Using Grid and Web services for SOA including UDDI registry

Future Technology

Presenter Name

Facility Name

Virtual Research Environments

JISC are funding a number of portal-based VRE projects – most based on Sakai.

• Need to engage this community – Has links into education and digital information

• Will be developing additional tools which can be used for NGS – e.g. a Wiki portlet– Can we prioritise other tools?

• We have proposed a “Sakai Alliance” as a means of keeping track of developments, helping projects communicate with each other, and feed back to US developers

• First meeting next week in Oxford

Presenter Name

Facility Name

See it in action…

Demos will be given over the tea break this afternoon. Some of the people involved are:

Responsible for NGS portal:• Dharmesh Chohan – NGS Portal and Grid Information Services

Other development work by:• Dave Meredith – e-HTPX Portal and Hub• RikTyer – e-Minerals Mini-Grid and Portal• Xiao Dong Wang – OGSA and OGSA-DAI Grid Services and

portlets• Xiaobo Yang – recently started work at DL• Rob Allan – ReDRESS portal for Awareness and Training, Sakai

VRE

We will try to answer your questions!

Presenter Name

Facility Name

Proxy Login

Presenter Name

Facility Name

Proxy List

Presenter Name

Facility Name

LDAP Browser

Presenter Name

Facility Name

Job Launcher

Presenter Name

Facility Name

GridFTP

Presenter Name

Facility Name

Web Content in MyWorkspace