Global Access to Resources Using...
Transcript of Global Access to Resources Using...
12/18/2014
1
1
Global Access to Resources Using Distributed Architecture
Asvija BCentre for Development of Advanced Computing (C-DAC),
Bangalore, INDIA
2
GARUDA ‐Motivation
Sharing of high‐end computational resources with the larger scientific and engineering community across the country.
Creating a Collaborative Framework for solving applications which are interdisciplinary requiring experts from multiple domains and distributed locations
Providing Universal access to resources
12/18/2014
2
3
Scenario
GG-BLR GG-CHE GG-HYD TF BLR TF PUNEIITD YUVAPRL
Climate Modeling
Disaster Manageme
nt
Bio Informatics
CFDCrypt analysis
How to Make Super Computers available to these users
3
4
GARUDA
75+ Partners
1700+ Certificates
6000 CPUs 550 TFs
220TB Storage
EGI, CHAIN‐REDS, caBig
NKN
India’s National Grid Computing Initiative
12/18/2014
3
5
GARUDA Backbone
InternetAccessGeneric
User
Local UserLAN
CC--DAC, PuneDAC, Pune
Partner Partner without resourceswithout resources
Access Terminals
PartnerPartnerwith resourceswith resources
LAN
Telescope
Local UserAccess Terminal
AccessTerminal
Access Terminal
CC--DAC, DAC, BangaloreBangalore
LAN Local User
Access Terminal
HeadNode
DatabaseDatabase
HeadNodeHeadNode
DatabaseDatabaseDatabaseDatabase
HeadNode
HeadNode
HeadNode
HeadNode
DatabaseDatabase
HeadNodeHeadNode
DatabaseDatabaseDatabaseDatabase HeadNode
HeadNode
Storage
Compute Nodes
Port
Switch
HeadNode
DatabaseDatabase
HeadNodeHeadNode
DatabaseDatabaseDatabaseDatabase
HeadNodeHeadNode
HeadNodeGrid
Headnode
Grid Headnode
Port
Switch
Port Switch
Submit Node/
Grid Head node
PortSwitch
M P L S AccessM P L S Access
Submit Node/
Grid Head node
InternetAccess
InternetAccessGeneric
User
Local UserLAN
CC--DAC, PuneDAC, Pune
Partner Partner without resourceswithout resources
Access Terminals
PartnerPartnerwith resourceswith resources
LAN
TelescopeTelescope
Local UserAccess Terminal
AccessTerminal
Access Terminal
CC--DAC, DAC, BangaloreBangalore
LAN Local User
Access Terminal
HeadNode
DatabaseDatabase
HeadNodeHeadNode
DatabaseDatabaseDatabaseDatabase
HeadNodeHeadNodeHeadNode
HeadNodeHeadNodeHeadNode
HeadNodeHeadNode
HeadNode
DatabaseDatabase
HeadNodeHeadNode
DatabaseDatabaseDatabaseDatabase HeadNodeHeadNodeHeadNode
HeadNodeHeadNodeHeadNode
StorageStorage
Compute Nodes
Port
Switch
HeadNode
DatabaseDatabase
HeadNodeHeadNode
DatabaseDatabaseDatabaseDatabase
HeadNodeHeadNode
HeadNodeHeadNodeHeadNodeGrid
Headnode
Grid Headnode
Port
Switch
Port Switch
Submit Node/
Grid Head node
Submit Node/
Grid Head node
PortSwitch
M P L S AccessM P L S AccessM P L S AccessM P L S Access
Submit Node/
Grid Head node
NKN
6
High Level Architecture
Computing Resources and Virtual Organizations
Research Organizations
Educational institutions Computing Centers
WSRF+GT4 + other Services +Cloud S/w]
NKN
Grid-Enabled Applications
Grid PSE
Virtualization support
Workflow tool
Job Scheduler
Grid Security and High-Performance Grid Networking
Non‐Research Organizations
Data G
rid
Resou
rce Enab
ler & M
onitorin
g
CDAC Resource centers
Access PortalCLI
Visualization
Federated
Information Server
Grid ProgrammingEnvironment
Grid ApplicationsSecurity
Resource Management
User Environments Middleware
Grid Programming & Development Environment -MpichG2 -Compiler Service
12/18/2014
4
7
Grid Middleware
Middleware is a layer between grid applications and low level functionality of grid.
Grid Middleware is a layer of S/W to enforce:
Scalability, Transparency, Heterogeneity, Fault Tolerance and Security of the Grid below the application layers.
It provides an uniform interface to the Grid users and handle all the complexity generated due to heterogeneous systems.
8
Components of Grid Middleware
12/18/2014
5
9
Popular Middleware
Globus – Globus Alliance
GridBus – University of Melbourne
UNICORE ‐ Uniform Interface to Computing Resource
gLite – CERN / EGEE
Legion – (Avaki ‐ Corporate Distributor)
Alchemi – (.NET Grid Computing Framework)
10
12/18/2014
6
11
– Indian Grid Certification Authority located at C-DAC, Knowledge Park, Bangalore, India.
– IGCA is the accredited member of APGridPMA.
– Issues X.509 Certificates to support the secure environment in Grid. (for GARUDA, institutes that do research in grid from India and foreign institutes that collaborate with GARUDA).
IGCA
12
IGTF
The International Grid Trust Federation (IGTF) is a body to establish common policies and guidelines between its Policy Management Authorities (PMAs) members.
http://www.igtf.net/
12/18/2014
7
13
Virtual User Community in GarudaGroup Name Description
Bioinformatics application of statistics and computer science to the molecular biology
ClimateModelling Deals with the dynamics of the climate system.
OSDD Community dedicated to develop drugs for tropical infectious diseases like malaria, tuberculosis
GeoPhysics Study related to physics of the Earth and its environment in space
CAE usage of computer software to solve engineering problems
IndianHeritage Focused on technology products for preserving & processing Heritage texts
HealthInformatics
Focused on utilizing compute power for health informatics
MaterialScience interdisciplinary field applying the properties of matter to science and engineering
Euindia The vision of a worldwide Grid for Research by both Europe and India
ToolsDeveloper Forum to communicate and collaborate on developing Garuda Tools
14
GARUDA Tools & Services
12/18/2014
8
15GBC-2013- Pandharpur
Access Methods
Two ways of Accessing Garuda Resources
Command Line Interface
Web Interface
16GBC-2013- Pandharpur
Command Line Interface
Provides Command Line Access to Garuda Resources
Requirements Valid IGCA Certificate User is required to have Garuda User Account
Facilities Certificate and Proxy Management Single Sign‐On to all Garuda Resources Job Submission, Monitoring and Management Data transfer and Management Querying Resource Information Application Compilation Facility Accomplishing Resource Reservation
Useful for Advanced Users
12/18/2014
9
17GBC-2013- Pandharpur
Web Interface Provides Garuda Access using Web Browser Requirements
Valid IGCA Certificate Garuda User Account
Provides easy and intuitive interfaces for Certificate and Proxy Management Job Submission, Monitoring and Management Data transfer and Management Querying Resource Information Carrying out Resource Reservation
For Novice users Web Interfaces
GARUDA Access Portal Problem Solving Environment
18GBC-2013- Pandharpur
GARUDA Access Portal
12/18/2014
10
19GBC-2013- Pandharpur
• Job Submission
• Resubmission of jobs and checking the job status using Mobile phone
• Browse Available resources
• Download any files from Grid Head Node
• Provision to View Status, Output, Error Files
• Portal.garudaindia.in
GARUDA Access Portal ‐ Features
20GBC-2013- Pandharpur
GARUDA Job Submission
Supports sequential and parallel Jobs
The user can specify the required environment variables
Uses the scheduler for submission
The files can be taken from Data Grid, the submit node, or the local machine
12/18/2014
11
21
• The status of submitted jobs can be viewed at any point of time
• The output and error files can be viewed or downloaded
• Cancel Jobs
GARUDA Access Portal ‐ Job Information
22GBC-2013- Pandharpur
Problem Solving Environment
12/18/2014
12
23GBC-2013- Pandharpur
Protein Structure Prediction
Protein Structure Prediction (PSP) based on evolutionary computing (Genetic Algorithms) method.
Functional modules of the PSP application implemented as grid services on Garuda
Released PSE‐PSP v1.0
Accessible from Garuda Portal
24GBC-2013- Pandharpur 24
Utility Services
12/18/2014
13
25GBC-2013- Pandharpur 25
Login Service
The Login service provides an abstract mechanism for an application to securely verify the user credentials.
Login service can be integrated to any application portals in GARUDA.
26GBC-2013- Pandharpur
Login Service
Login Service
12/18/2014
14
27GBC-2013- Pandharpur 27
Grid Support
To log your support request
Go to http://gridsupport.garudaindia.in
Login with your GARUDA Account.
Raise ticket in appropriate queue.
To raise a problem ticket
Select the queue
Click new ticket
Support request can also be sent via e‐mail
28GBC-2013- Pandharpur 28
Garuda RT (Request Tracker) Login page
Raising a ticket
12/18/2014
15
29GBC-2013- Pandharpur 29
List of support queues in Garuda
Sl. No. Project Name Support queue E-mail
1 Any grid related problem GDeployment [email protected]
2 Grid Portal Portal [email protected]
3 Service Oriented Architecture
4 GARUDAVOMS Voms [email protected]
5 Grid IDE Tool GridIDE [email protected]
6 OSDD OSDD [email protected]
7 Garuda Network Related Netops [email protected]
8 GARUDA SRM GDS [email protected]
9 Not sure about the queue Other [email protected]
30
Applications on Garuda
12/18/2014
16
31
Grid Enabled Applications
• Objectives– Enable applications of
national importance – Deliverables
– Grid enablement of illustrative applications • Disaster
Management• Bioinformatics
– Other Applications• Geomatics• Earthquake
Research • Weather and Climate
Modeling• Evolutionary
Computing
32
Bioinformatics : Open Source Drug Discovery
• OSDD users given access to Garuda through OSDD VO
• HPC clusters to run drug discovery problems
• Users connected through both NKN and Internet
• Galaxy Workflow for genomics proteomics applications
• Distributed job execution through Gridway
Project Team : OSSD community
12/18/2014
17
33
Winglet Application
Designing of the winglet to reduce the drag and hence improve the fuel efficiency.
Winglet application simulation has been grid enabled to achieve remarkable improvements in the simulation time
In collaboration with Zeus Numerix
34
Disaster Management - SAC DMSAR System
DMSAR scans the terrain and capture the data in 10,5,1m resolution modes The min amount of raw data captured in a single run is 9GB
12/18/2014
18
35
SAC DMSAR System Cont ..
One flight schedule will have ‘n’ no of runs Flight duration – 3Hrs Speed 450km/hr Data collection time – 1 to 1 ½ hrsAmount of data collected per flight = 50 – 80 GB
36
Grid Enabled DMSAR System
Raw Data Transfer
Disaster
Slide courtesy : Mr. Manavalan
12/18/2014
19
37
Raw data Transfer
Collecting Data & Recording
Raw data Transmission to the Grid Head Node
38
12/18/2014
20
39
40
NLP on Garuda Centre for Indian Language Technology ‐ IIT Bombay
Services Available on Megha Cloud resources
Wordnets for Indian Languages
Hindi
http://megha.garudaindia.in/iitb‐nlp/hindiwn/
Marathi
http://megha.garudaindia.in/iitb‐nlp/marathiwn/
Sanskrit
http://megha.garudaindia.in/iitb‐nlp/sanskritwn/
12/18/2014
21
41
NLP ‐ Other services Sata‐Anuvadak
Broad coverage statistical machine translation system for Indian languages.
Translation among 10 major Indian languages and English ‐ a total of 110 language pairs.
Currently adapted for tourism and health domains.
Sata‐Lipyaantaran
Statistical machine transliteration system for Indian languages.
4217/11/2010 Indo-Russian Workshop
Collaborations Space Application Centre, Ahmedabad
Disaster Management Grid Middleware for Satellite Grids
Indian Institute of Science, Bangalore Climate Modeling
Madras Institute of Technology, Chennai Grid Middleware for Semantic Grid Services
Indian Institute of Technology, Mumbai Computational Fluid Dynamics Package on PARAM Padma
University of Pune Quantum Chemistry, Materials Modeling, Bioinformatics INDMOL Molecular Package developed
EU-India Grid BARC, C-DAC, ERNET, Saha Institute, TIFR, Univ. Pune Consortium GARR, ICTP, INFN (Italy) Cambridge University (UK)
12/18/2014
22
43
EU‐Indiagrid Project Goals
To support the interconnection and interoperability of the prominent European Grid infrastructure (EGEE) with the Indian Grid infrastructure (Garuda National Grid Initiative, DAE‐Grid) for the benefit of eScience applications;
To identify and aggregate research, scientific and industrial communities which may benefit from the use of Grid technology resulting in an eScienceNetwork Community
44
PARTICIPANTSEUROPE Italian National Institute of Nuclear Physics, INFN (project
coordinator), Metaware SpA, Italian Academic and Research Network (GARR) Cambridge University
INTERNATIONAL Abdus Salam International Centre for Theoretical Physics (ICTP) INDIA Indian Education and Research Network (ERNET), University of Pune, SAHA Institute of Nuclear Physics, Kolkata, Centre for Development of Advanced Computing (C‐DAC), Bhabha Atomic Research Centre, Mumbai TATA Institute for Fundamental Research (Mumbai)
National Centre for Biological Sciences (Bangalore)
12/18/2014
23
45
caBIG‐GARUDA
Collaboration between cancer research grid from NCI (National Cancer Institute, USA) and GARUDA.
Achieve interoperability among technologies of both the grids.
To provision tools available on caGrid to Indian oncologists.
As POC, caTissue tool has been setup.
46
CHAIN‐REDS
12/18/2014
24
47
CHAIN‐REDS
CHAIN‐REDS is funded by the European Commission aiming at promoting and supporting technological and scientific collaboration across different e‐Infrastructures established and operated in various continents.
http://www.chain‐project.eu/
4817/11/2010 Indo-Russian Workshop
International Visibility
12/18/2014
25
49
50
Thrust application areas
Addressing National Challenges in
Agriculture
Energy
Biodiversity
12/18/2014
26
51
Agriculture Challenges
Increase Crop productivity,
Water management & Drought prediction
Less use of pesticide
Developing new seeds with inbuilt immune for diseases
Technical Solutions
Building Mathematical models for Drought prediction
Creating ontological databases
Analytics for crop tolerance
towards drought, water, pesticides etc.,
52
Biodiversity
Challenges
Understanding the ecological relationship among species
Modeling of Migration and extinctions
Habitat changes due to climate change and natural calamities
Conservation and sustainability of Bio‐resources
Technical Solution
Tools for efficient access and analyses of data
Image courtesy : NBA India
12/18/2014
27
53
Energy Sector Challenges
Demand & supply mismatch
Quality of generated power
Continuous power supply to critical unit
Avoid power loss
Transmission & theft
Technical Solutions
Modeling of weather & integrate with Power system
Load flow analysis using Analytics system to predict
Forecasting of trends and demands
Preempting catastrophe
54
Thank You !!