Post on 12-Jun-2020
Introduction to the course
Week #1Basics of Grid and Cloud computing
Hardi Tederhardi@eenet.ee
Tartu ÜlikoolFebruary 13 2013
Satish Sriramasrirama@ut.ee
Basics of Grid and Cloud computing 2/33
Overview
● BGCC administrative information
● Introduction to Grid Computing
● Authentication procedures
Basics of Grid and Cloud computing 3/33
BGCC course● 2005 – 2009 Basics of Grid computing
– by prof. Eero Vainikko
– gLite middleware
● 2010 – Basics of Grid and Cloud computing– Dr Satish Narayana Srirama joined with Cloud part
– gLite and ARC middlewares
● 2013 – same but– Eero Vainikko is at the University of Bath, UK
– almost no gLite
● 2014 – Course will not be delivered in the current form– Cloud Computing becomes an independent course (6 ECTS)
Basics of Grid and Cloud computing 4/33
Web and discussion
● Course web page – http://courses.cs.ut.ee/2013/cloud
– Contacts
– Times and rooms
– Lectures slides
– Lab exercises and deadlines
● Mailing list– ati.gtla@lists.ut.ee
Basics of Grid and Cloud computing 5/33
Course Structure
● Basics of Grid Computing● 7 Lectures ● 7 Labs ● Structure will be introduced in the coming slides
● Basics of Cloud Computing● 7 Lectures● 7 Labs● No course projects for this year :-)
● Structure will be discussed on 3rd April 2013● http://courses.cs.ut.ee/2013/cloud
Basics of Grid and Cloud computing 6/33
Grades and points
● Grid part 50% and Cloud part 50%
● You can earn 50 Grid points:– 25p Grid part of the exam
– 20p Grid Labs
– 5p Active participation at lectures● Devising questions for on-line study-questionary in 24h after each
lecture
● Two different courses – you should pass both– You have to score at least 50% in both courses
Basics of Grid and Cloud computing 7/33
Grid part given by
● Hardi Teder– Lectures and Labs on Wednesday
● Pelle Jakovits– PhD student at Mobile Cloud Lab
– Labs on Tuesday and Thursday
● Ilja Kromonov– Student at Mobile Cloud Lab
– Labs on Thursday
Basics of Grid and Cloud computing 8/33
About me*
● Related with Grids and HPC since 2003
● at EENet since 2003● Grid projects:
– BalticGrid I-II, EGEE/EGI, Nordic Grid Neighbourhood
● Computing and storage resources
● X509 certificates and PKI● AAI and Federations
● Head of Distributed computing department at EENet
● EGI council member● EGI security policy group
member● Baltic Grid CA manager
– EUGridPMA, IGTF, OGF CAOPS-WG
● TERENA – TCS and REFEDS
* “me” is Hardi
Basics of Grid and Cloud computing 9/33
7 Grid lectures
● Introduction to Grid computing
● Grid middlewares and infrastructures
● Grid information services
● Data management on Grid
● Grid security
● Grid job management
● About Grid experiments
Basics of Grid and Cloud computing 10/33
7 Grid labs● Requesting the Grid certificate and VO registration and Python
warm-up. (2p)
● Hello Grid! Hello ARC! (2p)
● Grid information services (2p)
● Data Management on Grid (3p)
● Grid security and breaking RSA (4p)
● Grid experiment – Job management (4p)
● Grid experiment - GPUs on Grid (3p)
● Usually the deadline of exercises is 2 weeks but not always ● 50% of points after deadline in Grid exercises
Basics of Grid and Cloud computing 11/33
Introduction to Grid ● Who needs it?
● What is Grid?
● What is Grid useful for?
Basics of Grid and Cloud computing 12/33
Driving forces of computational science
● Environment simulation● Climate changes● Prediction of amount of fish in
Norwegian fjords● Ice glacier flow simulation
● Solving fluid dynamic problems
● Weather predictions● Design of hypersonic airplanes● Design of more efficient cars● Extremely quiet submarines● Design of efficient and safe
nuclear power stations
● Simulation of nuclear explosions
● Satellite data analysis
● Data analysis of DNA-sequences
● Simulation of 3D proteine molecules
● Simulation of global economical processes
● etc. in more and more fields
Basics of Grid and Cloud computing 13/33
Driving forces of computational science
● Common to all examples - need for larger than usual set of resources
● CPU cycles● Data volumes● Special devices producing
data● Parallel processing
● Common problems● how to store data?● how to move data?● which algorithms can be
used?
Basics of Grid and Cloud computing 14/33
Why the Grid?
● Science is becoming increasingly digital and needs to deal with increasing amounts of data.– Large Hadron Collider (LHC), Radio
telescopes, gene research
● More complicated simulations grew bigger than HPC centres could provide resource
● Collaboration– Grid provides infrastructure for sharing
resources.
Concorde(15 Km)
Balloon(30 Km)
CD stack with1 year LHC data!(~ 20 Km)
Mt. Blanc(4.8 Km)
Basics of Grid and Cloud computing 15/33
Grid
Grid is securely share distributed resources (computation, storage, etc) so that users can collaborate within Virtual Organisations (VO):
Basics of Grid and Cloud computing 16/33
GRID
MIDDLEWARE
Visualization
Supercomputer, PC-Cluster
Data-storage, Sensors, Experiments
Internet, networks
Desktop
Mobile Access
Basics of Grid and Cloud computing 17/33
Middleware
● Tools and packages for building Grid:– Globus toolkit
– Nordugrid ARC
– gLite
– Unicore
● EMI
European
Middleware
Initiative
Basics of Grid and Cloud computing 18/33
Grid foundations
Basics of Grid and Cloud computing 19/33
Resource management
● Computing resources
● Storage resources
● Other specific resources
Basics of Grid and Cloud computing 20/33
Information services
● Maintains information about hardware, software, services and people participating in a Virtual Organization– Should scale with the Grid´s growth
– Sharing jobs
– Logging and accounting
– Monitoring
Basics of Grid and Cloud computing 21/33
Data management
● Data access and transfer– Simple, automatic multi-protocol file transfer tools: Integrated
with Resource Management service● Move data from local machine to remote machine, where the job is
executed (input file staging)● Move the output files from the remote computer to the local
machines (output file staging)● Pull executable from a remote location
– To have a secure, high-performance, reliable file transfer over modern WANs: GridFTP
● Data replication and management
Basics of Grid and Cloud computing 22/33
Grid security
● Basic security:– Authentication: Who we are on the Grid?
– Authorization: Do we have access to a resource/service?
– Protection: Data integrity and confidentiality
● Grid Security Infrastructure (GSI):– Grid credentials: digital certificate and private key
● International trust– IGTF
– EUGridPMA
Basics of Grid and Cloud computing 23/33
AuthN and AuthZ
Grid resources (A)
Grid resources (B)
Certification Authority (CA)BobCert request
User Interface (UI)
Bob´s Grid certificate
VO Database
VO ServiceVO
Manager
VO membership request
VO
VO Account
Pool
VO Account
Pool
Automatic mappingfor Bob
Automatic mappingfor Bob
voms-proxy-init
Basics of Grid and Cloud computing 24/33
Virtual Organizations
● Distributed people and resources
R
R
RR
R
R
R
R
R
RR
R
Basics of Grid and Cloud computing 25/33
Virtual Organizations
● People and resources
● Network connections
● Sharing resources
● Dynamic
● Fault tolerant
RR
R
R
R
R
R
RR
R
R
R
VO-BVO-A
Basics of Grid and Cloud computing 26/33
Grid User Interfaces
● Linux command line UI
● GUIs
● Web portals
Basics of Grid and Cloud computing 27/33
Estonian Grid
● 2004 started with Grid developments in Estonia
● Started with NorduGrid middleware
● BalticGrid and BalticGrid-II projects for developing Grid in Baltic States 2005-2010
● Estonian NGI at European Grid Infrastructure (EGI)
Basics of Grid and Cloud computing 28/33
ETAIS● Estonian Scientific
Computing infrastructure– To provide computing and
storage resources for science and education
– 2011-2012
● Partners– Tartu Ülikool
– Tallinna Tehinkaülikool
– Keemilise ja Bioloogilise Füüsika Instituut
– EENet
Basics of Grid and Cloud computing 29/33
European Grid Infrastructure
● Central body for coordinating international Grid collaboration and standardization and interoperability
● 34 members
● EENet represents Estonian NGI
Basics of Grid and Cloud computing 30/33
Cloud computing
● Gartner: “Cloud computing is a style of computing where massively scalable IT-related capabilities are provided ‘as a service’ across the Internet to multiple external customers”
● Why it is getting cloudy? – Development
– Business model
– Management model
– Virtualization level
Basics of Grid and Cloud computing 31/33
Grid vs Cloud
● Rather – Science
– Real machines
– Similar resources (Linux clusters, mostly SLC5)
– Many resource providers in the same group
– Collaboration in big Vos
– PKI
● Rather– Business
– Virtual machines
– Different resources (Linux, Windows, etc)
– One (few) resource providers per cloud
– Services for special groups
– AAI
Basics of Grid and Cloud computing 32/33
What is Grid useful for?
● Resource sharing● simplified access to remote resources: computing, databases,
software● multiple geographically apart resource aggregation● flexibility: in case of sudden need for large amount of
computing resources● reliability: network cuts, resource downtimes ● collaboration: remote working groups and developing teams
Basics of Grid and Cloud computing 33/33
Thank you● More information from:
● Hardi Teder hardi@eenet.ee● Pelle Jakovits jakovits@smail.ee● Ilja Kromonov kromon@ut.ee
● http://courses.cs.ut.ee/2013/cloud
● ati.gtla@lists.ut.ee