Weaving the World-Wide Grid [Marketplace]: Rajkumar Buyya Melbourne, Australia WW Grid “ Economic...
-
date post
22-Dec-2015 -
Category
Documents
-
view
231 -
download
1
Transcript of Weaving the World-Wide Grid [Marketplace]: Rajkumar Buyya Melbourne, Australia WW Grid “ Economic...
Weaving the World-Wide Grid [Marketplace]:
Rajkumar Buyya
Melbourne, Australiawww.buyya.com/ecogrid
WW Grid
“Economic Paradigm for Distributed Resource Management and Scheduling for Service-Oriented Computing”
2
3
Vision: Grid for Service Oriented Computing?
WW Grid
World Wide Grid!
Nimrod-G
4
Overview
A quick glance at Grid computing Resource Management challenges for next
generation Grid computing A Glance at Approaches to Grid computing. Grid Architecture for Computational
Economy Nimrod-G -- Grid Resource Broker Scheduling Experiments on
the World Wide Grid: both Real and Simulation
ConclusionsScheduling Economics
Grid
EconomyGrid
5
2100
2100 2100 2100 2100
2100 2100 2100 2100
Desktop SMPs or SuperComputers
LocalCluster
GlobalCluster/Grid
PERFORMANCE
Inter PlanetaryGrid!
•Individual•Group•Department•Campus•State•National•Globe•Inter Planet•Galaxy
Administrative Barriers
EnterpriseCluster/Grid
?
Scalable HPC: Breaking Administrative Barriers & new challenges
6
Why SC? Large Scale Explorations need them—Killer Applications.
Solving grand challenge applications using modeling, simulation and analysis
Life Sciences
CAD/CAM
Aerospace
Military ApplicationsDigital Biology Military ApplicationsMilitary Applications
Internet & Ecommerce
7
What is Grid ?
A paradigm/infrastructure that allows sharing, selection, & aggregationof geographically distributed resources:
Computers – PCs, workstations, clusters, supercomputers, laptops, notebooks, mobile devices, PDA, etc;
Software – e.g., ASPs renting expensive special purpose applications on demand;
Catalogued data and databases – e.g. transparent access to human genome database;
Special devices/instruments – e.g., radio telescope – SETI@Home searching for life in galaxy.
People/collaborators.
[depending on their availability, capability, cost, and user QoS requirements]
for solving large-scale problems/applications. Thus enabling the creation of “virtual enterprises” (VEs)
Widearea
8
P2P/Grid Applications-Drivers
Distributed HPC (Supercomputing): Computational science.
High-Capacity/Throughput Computing: Large scale simulation/chip design & parameter studies.
Content Sharing (free or paid) Sharing digital contents among peers (e.g., Napster)
Remote software access/renting services: Application service provides (ASPs) & Web services.
Data-intensive computing: Drug Design, Particle Physics, Stock Prediction...
On-demand, realtime computing: Medical instrumentation & Mission Critical.
Collaborative Computing: Collaborative design, Data exploration, education.
Service Oriented Computing (SOC): Computing as Competitive Utility: New paradigm, new
industries, and new business.
9
Building and Using Grids require
Services that enable the execution of a job on a resource in different admistrative domain.
Security mechanisms that permit resources to be accessed only by authorized users.
App/Data Security (?) – A must for commercial users (protecting from GSPs/other users).
(New) programming tools that make our applications Grid Ready!.
Tools that can translate the requirements of an application/user into the requirements of computers, networks, and storage.
Tools that perform resource discovery, trading, selection/allocation, scheduling and distribution of jobs and collects results.
Globus
Nimrod-G
Resource Management Challenges in
Grid Computing Environments
11
A Typical Grid Computing Environment
Grid Resource Broker
Resource Broker
Application
Grid Information Service
Grid Resource Broker
databaseR2R3
RN
R1
R4
R5
R6
Grid Information Service
12
What users want ?Users in Grid Economy &
Strategy Grid Consumers
Execute jobs for solving varying problem size and complexity
Benefit by selecting and aggregating resources wisely Tradeoff timeframe and cost
Strategy: minimise expenses Grid Providers
Contribute (“idle”) resource for executing consumer jobs Benefit by maximizing resource utilisation Tradeoff local requirements & market opportunity
Strategy: maximise return on investment
13
Sources of Complexity in Grid for Resource Management and Scheduling
Size (large number of nodes, providers, consumers) Heterogeneity of resources (PCs, Workstations, clusters, and
supercomputers, instruments, databases, software) Heterogeneity of fabric management systems (single system image OS,
queuing systems, etc.) Heterogeneity of fabric management polices Heterogeneity of application requirements (CPU, I/O, memory, and/or
network intensive) Heterogeneity in resource demand patterns (peak, off-peak, ...) Applications need different QoS at different times (time critical results). The
utility of experimental results varies from time to time. Geographical distribution of users & located different time zones Differing goals (producers and consumers have different objectives and
strategies) Unsecure and Unreliable environment
14
Need Grid tools for managing
Security
Resource Allocation & Scheduling
Data locality
Network Management
System Management
Resource Discovery
Uniform Access
Computational Economy
Application Development Tools
15
Traditional approaches to resource management & scheduling are NOT useful
for Grid ? They use centralised policy that need
complete state-information and common fabric management policy or decentralised consensus-
based policy. Due to too many heterogenous parameters in the Grid it is
impossible to define/get: system-wide performance matrix and common fabric management policy that is acceptable to all.
“Economic” paradigm proved as an effective institution in managing decentralization and heterogeneity that is present in human economies!
Hence, we propose/advocate the use of “computational economy” principles in the management of resources and scheduling computations on the Grid.
16
Benefits of Computational Economies
It provides a nice paradigm for managing self interested and self-regulating entities (resource owners and consumers)
Helps in regulating supply-and-demand for resources. Services can be priced in such a way that equilibrium is maintained.
User-centric / Utility driven: Value for money! Scalable:
No need of central coordinator (during negotiation) Resources(sellers) and also Users(buyers) can make their own decisions and try to
maximize utility and profit. Adaptable It helps in offering different QoS (quality of services) to different applications
depending the value users place on them. It improves the utilisation of resources It offers incentive for resource owners for being part of the grid! It offers incentive for resource consumers for being good citizens There is large body of proven Economic principles and techniques available, we can
easily leverage it.
17
New challenges of Computational Economy
Resource Owners How do I decide prices ? (economic models?) How do I specify them ? How do I enforce them ? How do I advertise & attract consumers ? How do I do accounting and handle payments? …..
Resource Consumers How do I decide expenses ? How do I express QoS requirements ? How I trade between timeframe & cost ? ….
Any tools, traders & brokers available to automate the process ?
18
mix-and-match
Object-oriented
Internet/partial-P2P
Network enabled Solvers
Market/Computational Economy
19
Building an Economy Grid(Next Generation Grid
Computing!)
To enable the creation and promotion of:Grid Marketplace (competitive)
ASPService Oriented Computing
. . .And let users focus on their own work (science, engineering, or commerce)!
20
Grid Node N
GRACE: A ReferenceGrid Architecture for Computational Economy
Grid Consumer
Pro
gra
mm
ing
En
viro
nm
ents
Grid Resource Broker
Grid Service Providers
Grid Explorer
Schedule Advisor
Trade Manager
Job ControlAgent
Deployment Agent
Trade Server
Resource Allocation
ResourceReservation
R1
Misc. services
Information Service
R2 Rm…
Pricing Algorithms
Accounting
Grid Node1
…
Grid Middleware Services
…
…
HealthMonitor
Grid Market Services
JobExec
Info ?
Secure
Trading
QoS
Storage
Sign-on
Grid Bank
Ap
pli
cati
on
s
21
Grid Node N
GRACE: A ReferenceGrid Architecture for Computational Economy
Grid User
Application
Grid Resource Broker
Grid Service Providers
Grid Explorer
Schedule Advisor
Trade Manager
Job ControlAgent
Deployment Agent
Trade Server
Resource Allocation
ResourceReservation
R1
Misc. services
Information Server(s)
R2 Rm…
Pricing Algorithms
Accounting
Grid Node1
…
Grid Middleware Services
…
…
HealthMonitor
Grid Market Services
JobExec
Info ?
Secure
Trading
QoS
Storage
Sign-on
Grid Bank
See PDPTA 2000 paper!
22
Economic Models
Price-based: Supply,demand,value, wealth of economic system
Commodity Market Model Posted Price Model Bargaining Model Tendering (Contract Net) Model Auction Model
English, first-price sealed-bid, second-price sealed-bid (Vickrey), and Dutch (consumer:low,high,rate; producer:high, low, rate)
Proportional Resource Sharing Model Monopoly (one provider) and Oligopoly (few players)
consumers may not have any influence on prices. Bartering
Shareholder Model Partnership Model
See SPIE ITCom 2001 paper!: with Heinz Stockinger, CERN!
23
Cost Model
Without cost model any shared system becomes un-managable
Charge users more for remote facilities than their own
Choose cheaper resources before more expensive ones
Cost units (G$) may be Dollars Shares in global facility Stored in bank
24
Cost Matrix @ Grid site X
Non-uniform costing Encourages use of
local resources first Real accounting
system can control machine usage
11 33
22 11User 5User 5
Mach
ine 1
Mach
ine 1
User 1User 1
Mach
ine 5
Mach
ine 5
Resource Cost = Function (cpu, memory, disk, network, software, QoS, current demand, etc.)
Simple: price based on peaktime, offpeak, discount when less demand, ..
Nimrod-G:The Grid Resource Broker
Soft Deadline and Budget-based Economy Grid Resource Broker
for Parameter (Task Farming Applications) Processing on
Grids
26
A resource broker for managing, steering, and executing task farming (parameter sweep/SPMD model) applications on Grid based on deadline and computational economy.
Based on users’ QoS requirements, our Broker dynamically leases services at runtime depending on their quality, cost, and availability.
Key Features A single window to manage & control experiment Persistent and Programmable Task Farming Engine Resource Discovery Resource Trading Scheduling & Predications Generic Dispatcher & Grid Agents Transportation of data & results Steering & data management Accounting
Nimrod/G : A Grid Resource Broker
27
Parametric Computing(What Users think of Nimrod
Power)
Multiple RunsSame ProgramMultiple Data Killer Application for the Grid!
ParametersAge Hair
23 CleanAge Hair
23 Clean23 Beard28 Goatee
Age Hair23 Clean23 Beard
Age Hair23 Clean23 Beard28 Goatee28 Clean
Age Hair23 Clean23 Beard28 Goatee28 Clean19 Moustache
Age Hair23 Clean23 Beard28 Goatee28 Clean19 Moustache10 Clean
Age Hair23 Clean23 Beard28 Goatee28 Clean19 Moustache10 Clean
-4000000 Too much
Courtesy: Anand Natrajan, University of Virginia
Magic Engine
28
Sample P-Sweep/Task Farming Applications
Sample P-Sweep/Task Farming Applications
Bioinformatics: Bioinformatics: Drug Design / Protein Drug Design / Protein
ModellingModelling
SensitivitySensitivityexperiments experiments
on smog formationon smog formation
Combinatorial Combinatorial Optimization:Optimization:
Meta-heuristic Meta-heuristic parameter estimationparameter estimation
Ecological Modelling: Ecological Modelling: Control Strategies Control Strategies
for Cattle Tickfor Cattle Tick
Electronic CAD: Electronic CAD: Field Programmable Field Programmable
Gate ArraysGate ArraysComputer Graphics: Computer Graphics: Ray TracingRay Tracing
High Energy High Energy Physics: Physics:
Searching for Searching for Rare EventsRare Events
Finance: Finance: Investment Risk AnalysisInvestment Risk Analysis
VLSI Design: VLSI Design: SPICE SimulationsSPICE Simulations
Aerospace: Aerospace: Wing DesignWing Design
Network SimulationNetwork SimulationAutomobile:Automobile:
Crash Simulation Crash Simulation
Data MiningData Mining
Civil Engineering:Civil Engineering:Building Design Building Design
astrophysics astrophysics
29
Distributed Drug Design: Data Intensive Computing on the
Grid
A Virtual Laboratory environment for “Molecular Docking for Drug Design” on the Grid.
It provides tools for screening millions of chemical compounds (molecules) in the Chemical DataBase (CDB) to identify those having potential use in drug design (acts as inhibitor).
In collaboration with: Kim Branson, Structural Biology,
Walter and Eliza Hall Institute (WEHI)
http://www.buyya.com/vlab
30
Docking Application Input data configuration file
score_ligand yesminimize_ligand yesmultiple_ligands norandom_seed 7anchor_search notorsion_drive yesclash_overlap 0.5conformation_cutoff_factor 3torsion_minimize yesmatch_receptor_sites norandom_search yes . . . . . . . . . . . .maximum_cycles 1ligand_atom_file S_1.mol2receptor_site_file ece.sphscore_grid_prefix ecevdw_definition_file parameter/vdw.defnchemical_definition_file parameter/chem.defnchemical_score_file parameter/chem_score.tblflex_definition_file parameter/flex.defnflex_drive_file parameter/flex_drive.tblligand_contact_file dock_cnt.mol2ligand_chemical_file dock_chm.mol2ligand_energy_file dock_nrg.mol2
Molecule to Molecule to be screenedbe screened
31
score_ligand $score_ligandminimize_ligand $minimize_ligandmultiple_ligands $multiple_ligandsrandom_seed $random_seedanchor_search $anchor_searchtorsion_drive $torsion_driveclash_overlap $clash_overlapconformation_cutoff_factor $conformation_cutoff_factortorsion_minimize $torsion_minimizematch_receptor_sites $match_receptor_sitesrandom_search $random_search . . . . . . . . . . . .maximum_cycles $maximum_cyclesligand_atom_file ${ligand_number}.mol2receptor_site_file $HOME/dock_inputs/${receptor_site_file}score_grid_prefix $HOME/dock_inputs/${score_grid_prefix}vdw_definition_file vdw.defnchemical_definition_file chem.defnchemical_score_file chem_score.tblflex_definition_file flex.defnflex_drive_file flex_drive.tblligand_contact_file dock_cnt.mol2ligand_chemical_file dock_chm.mol2ligand_energy_file dock_nrg.mol2
Parameterize Dock input file(use Nimrod Tools: GUI/language)
Molecule to be Molecule to be screenedscreened
32
parameter database_name label "database_name" text select oneof "aldrich" "maybridge" "maybridge_300" "asinex_egc" "asinex_epc" "asinex_pre" "available_chemicals_directory" "inter_bioscreen_s" "inter_bioscreen_n" "inter_bioscreen_n_300" "inter_bioscreen_n_500" "biomolecular_research_institute" "molecular_science" "molecular_diversity_preservation" "national_cancer_institute" "IGF_HITS" "aldrich_300" "molecular_science_500" "APP" "ECE" default "aldrich_300";
parameter CDB_SERVER text default "bezek.dstc.monash.edu.au";parameter CDB_PORT_NO text default "5001";parameter score_ligand text default "yes";parameter minimize_ligand text default "yes";parameter multiple_ligands text default "no";parameter random_seed integer default 7;parameter anchor_search text default "no";parameter torsion_drive text default "yes";parameter clash_overlap float default 0.5;parameter conformation_cutoff_factor integer default 5;parameter torsion_minimize text default "yes";parameter match_receptor_sites text default "no"; . . . . . . . . . . . .parameter maximum_cycles integer default 1;parameter receptor_site_file text default "ece.sph";parameter score_grid_prefix text default "ece";parameter ligand_number integer range from 1 to 2000 step 1;
Create Dock PlanFile1. Define parameters and their value
Molecules to be Molecules to be screenedscreened
33
task nodestart copy ./parameter/vdw.defn node:. copy ./parameter/chem.defn node:. copy ./parameter/chem_score.tbl node:. copy ./parameter/flex.defn node:. copy ./parameter/flex_drive.tbl node:. copy ./dock_inputs/get_molecule node:. copy ./dock_inputs/dock_base node:.endtasktask main node:substitute dock_base dock_run node:substitute get_molecule get_molecule_fetch node:execute sh ./get_molecule_fetch node:execute $HOME/bin/dock.$OS -i dock_run -o dock_out copy node:dock_out ./results/dock_out.$jobname copy node:dock_cnt.mol2 ./results/dock_cnt.mol2.$jobname copy node:dock_chm.mol2 ./results/dock_chm.mol2.$jobname copy node:dock_nrg.mol2 ./results/dock_nrg.mol2.$jobnameendtask
Create Dock PlanFile2. Define the task that each job needs to
do
34
Nimrod-G Broker Automating Distributed Processing
0
10
20
30
40
50
60
70
80
90
1st Qtr 2nd Qtr 3rd Qtr 4th Qtr
East
West
North
South
Compose, Submit, & Play!
35
Nimrod & Associated Family of Tools
P-sweep App. Composition: Nimrod/
EnfusionResource Management and Scheduling:
Nimrod-G BrokerDesign Optimisations:
Nimrod-OApp. Composition and Online Visualization:
Active SheetsGrid Simulation in Java:
GridSimDrug Design on Grid:
Virtual Lab
0
10
20
30
40
50
60
70
80
90
1st Qtr 2nd Qtr 3rd Qtr 4th Qtr
East
West
North
South
Remote Execution Server(on demand Nimrod Agent)
File Transfer Server
36
A Glance at Nimrod-G Broker
Grid Middleware
Nimrod/G Client Nimrod/G ClientNimrod/G Client
Grid Information Server(s)
Schedule Advisor
Trading Manager
Nimrod/G Engine
GridStore
Grid Explorer
GE GISTM TS
RM & TS
Grid Dispatcher
RM: Local Resource Manager, TS: Trade Server
Globus, Legion, Condor, etc.
G
G
CL
Globus enabled node.Legion enabled node.
GL
Condor enabled node.
RM & TSRM & TS
C LSee HPCAsia 2000 paper!
37
Globus Legion
Fabric
Nimrod-G Broker
Nimrod-G ClientsP-Tools (GUI/Scripting)
(parameter_modeling)
Legacy Applications
P2P GTS
Farming Engine
Dispatcher & Actuators
Schedule Advisor
Trading Manager
Grid Explorer
Customised Apps(Active Sheet)
Monitoring and Steering Portals
Algorithm1
AlgorithmN
Middleware
. . .
Computers Storage Networks InstrumentsLocal Schedulers
G-Bank. . .
Agents
Resources
Programmable Entities Management
Jobs Tasks
. . .
AgentScheduler JobServer
PC/WS/Clusters Radio TelescopeCondor/LL/NQS . . .Database
Meta-Scheduler
Nimrod/G Grid Broker Architecture
Channels
. . .
Database
Condor GMD
IP hourglass!
Condor-AGlobus-A Legion-A P2P-A
38
A Nimrod/G Monitor
A Nimrod/G Monitor
CostCostDeadlineDeadline
Legion hosts
Globus Hosts
Bezek is in both Globus and Legion Domains
Arlington
Alexandria
Richmond
HamptonNorfolk
Virginia BeachChesapeakePortsmouth
Newport News
Roanoke
Ap p om a toxRive r
Ja m esRive r
Shena nd oa hRive r
Ra p p a ha nnoc kRive r
Potom a cRive r
VIRGINIA77
81
64
64
66
85
39
User Requirements: Deadline/Budget User Requirements: Deadline/Budget
40
Another User Interface:Active Sheet for Spreadsheet Processing on
Grid
NimrodNimrodProxyProxy
Nimrod/GNimrod/G
41
42
Nimrod/G Interactions
Grid InfoServer
ProcessServer
UserProcess
File accessFileServer
Grid Node
NimrodAgent
Compute NodeUser Node
GridDispatcher
Grid Trade Server
GridScheduler
Local Resource Manager
Nimrod-G Grid Broker
TaskFarmingEngine
Grid ToolsAnd
Applications
Do this in 30 min. for $10?
43
Discover Discover ResourcesResources
Distribute JobsDistribute Jobs
Establish Establish RatesRates
Meet requirements ? Remaining Meet requirements ? Remaining Jobs, Deadline, & Budget ?Jobs, Deadline, & Budget ?
Evaluate & Evaluate & RescheduleReschedule
Discover Discover More More
ResourcesResources
Adaptive Scheduling Steps
Compose & Compose & ScheduleSchedule
44
Deadline and Budget Constrained Scheduling Algorithms
Algorithm/Strategy
Execution Time(Deadline, D)
Execution Cost(Budget, B)
Cost Opt Limited by D Minimize
Cost-Time Opt Minimize when possible
Minimize
Time Opt Minimize Limited by B
Conservative-Time Opt
Minimize Limited by B, but all unprocessed jobs have guaranteed minimum budget
Application Scheduling Experiments on the World-
Wide Grid
Task Farming Applications on
World Wide Grid
WW Grid
46
The World Wide Grid SitesWW Grid
EUROPE:ZIB/GermanyPC2/GermanyAEI/Germany Lecce/ItalyCNR/ItalyCalabria/ItalyPozman/PolandLund/SwedenCERN/SwissCUNI/Czech R.Vrije: Netherlands
EUROPE:ZIB/GermanyPC2/GermanyAEI/Germany Lecce/ItalyCNR/ItalyCalabria/ItalyPozman/PolandLund/SwedenCERN/SwissCUNI/Czech R.Vrije: Netherlands
ANL/ChicagoUSC-ISC/LA
UTK/TennesseeUVa/Virginia
Dartmouth/NHBU/Boston
UCSD/San Diego
ANL/ChicagoUSC-ISC/LA
UTK/TennesseeUVa/Virginia
Dartmouth/NHBU/Boston
UCSD/San Diego
Monash/MelbourneVPAC/Melbourne
Monash/MelbourneVPAC/MelbourneSantiago/Chile
Santiago/Chile
TI-Tech/TokyoETL/TsukubaAIST/Tsukuba
TI-Tech/TokyoETL/TsukubaAIST/Tsukuba
Cardiff/UKPortsmoth/UKManchester, UK
Cardiff/UKPortsmoth/UKManchester, UK
Kasetsart/BangkokKasetsart/Bangkok
SingaporeSingapore
47
World Wide Grid (WWG)WW Grid
Globus+LegionGRACE_TS
Australia
Monash U. : Cluster
VPAC: Alpha
Solaris WS
Nimrod/G
Globus +GRACE_TS
Europe
ZIB: T3E/OnyxAEI: Onyx Paderborn: HPCLineLecce: Compaq SCCNR: ClusterCalabria: Cluster CERN: ClusterCUNI/CZ: OnyxPozman: SGI/SP2Vrije U: ClusterCardiff: Sun E6500Portsmouth: Linux PCManchester: O3K
Globus +GRACE_TS
Asia
Tokyo I-Tech.: Ultra WSAIST, Japan: Solaris ClusterKasetsart, Thai: ClusterNUS, Singapore: O2K
Globus/LegionGRACE_TS
North America
ANL: SGI/Sun/SP2USC-ISI: SGIUVa: Linux ClusterUD: Linux clusterUTK: Linux clusterUCSD: Linux PCsBU: SGI IRIX
Internet
Globus +GRACE_TS South America
Chile: Cluster
WW Grid
48
Experiment-1: Peak and Off-peak
Workload: 165 jobs, each need 5 minute of cpu time
Deadline: 1 hrs. and budget: 800,000 units
Strategy: Minimize Cost and meet the deadline
Execution Cost with cost optimisation AU Peaktime:471205 (G$) AU Offpeak time: 427155 (G$)
49
Resources Selected & Price/CPU-sec.
Resource Type & Size
Owner and Location
Grid services
Peaktime Cost (G$)
Offpeak cost
Linux cluster (60 nodes)
Monash, Australia
Globus/Condor 20 5
IBM SP2 (80 nodes)
ANL, Chicago, US
Globus/LL 5 10
Sun (8 nodes) ANL, Chicago, US
Globus/Fork 5 10
SGI (96 nodes) ANL, Chicago, US
Globus/Condor-G
15 15
SGI (10 nodes) ISI, LA, US Globus/Fork 10 20
50
Execution @ AU Peak Time
0
2
4
6
8
10
12
Time (minutes)
Jo
bs
Linux clus ter - Monash (20) Sun - ANL (5) SP2 - ANL (5) SGI - ANL (15) SGI - ISI (10)
51
Execution @ AU Offpeak Time
0
2
4
6
8
10
12
Time (minutes)
Jo
bs
Linux clus ter - Monash (5) Sun - ANL (10) SP2 - ANL (10) SGI - ANL (15) SGI - ISI (20)
52
Experiment-2 Setup
Workload: 165 jobs, each need 5 minute of CPU time
Deadline: 2 hrs. and budget: 396000 G$ Strategies: 1. Minimise cost 2. Minimise time Execution:
Optimise Cost: 115200 (G$) (finished in 2hrs.) Optimise Time: 237000 (G$) (finished in 1.25 hr.) In this experiment: Time-optimised scheduling run
costs double that of Cost-optimised. Users can now trade-off between Time Vs. Cost.
53
Resources Selected & Price/CPU-sec.
Resource & Location
Grid services & Fabric
Cost/CPU sec.or unit
No. of Jobs Executed
Time_Opt Cost_Opt.
Linux Cluster-Monash, Melbourne, Australia
Globus, GTS, Condor
2 64 153
Linux-Prosecco-CNR, Pisa, Italy
Globus, GTS, Fork 3 7 1
Linux-Barbera-CNR, Pisa, Italy
Globus, GTS, Fork 4 6 1
Solaris/Ultas2
TITech, Tokyo, Japan
Globus, GTS, Fork 3 9 1
SGI-ISI, LA, US Globus, GTS, Fork 8 37 5
Sun-ANL, Chicago,US Globus, GTS, Fork 7 42 4Total Experiment Cost (G$) 237000 115200
Time to Complete Exp. (Min.) 70 119
54
Resource Scheduling for DBC Time Optimization
0
2
4
6
8
10
12
Time (in Minute)
No.
of
Tas
ks i
n E
xecu
tion
Condor-Monash Linux-Prosecco-CNR Linux-Barbera-CNR
Solaris /Ultas2-TITech SGI-ISI Sun-ANL
55
Resource Scheduling for DBC Cost Optimization
0
2
4
6
8
10
12
14
Time (in Minute)
No.
of
Tas
ks i
n E
xecu
tion
Condor-Monash Linux-Prosecco-CNR Linux-Barbera-CNR
Solaris /Ultas2-TITech SGI-ISI Sun-ANL
56
Experiment-3 Setup: Using GridSim
Workload Synthesis: 200 jobs, each job processing requirement = 10K
MI or SPEC with random variation from 0-10%. Exploration of many scenarios:
Deadline: 100 to 3600 simulation time, step = 500 Budget: 500 to 22000 G$, step = 1000
DBC Strategies: Cost Optimisation Time Optimisation
Resources: Simulated WWG resources
57
Simulated WWG Resources
Resource Name in
Simulation
Simulated Resource Characteristics
Vendor, Resource Type, Node OS, No of PEs
Equivalent Resource in
Worldwide Grid
(Hostname, Location)
A PE SPEC/ MIPS Rating
Resource Manager
Type
Price
(G$/PE time unit)
MIPS per G$
R0 Compaq, AlphaServer,
CPU, OSF1, 4
grendel.vpac.org,
VPAC, Melb, Australia
515 Time-shared 8 64.37
R1 Sun, Ultra, Solaris, 4 hpc420.hpcc.jp,
AIST, Tokyo, Japan 377 Time-shared 4 94.25
R2 Sun, Ultra, Solaris, 4 hpc420-1.hpcc.jp,
AIST, Tokyo, Japan 377 Time-shared 3 125.66
R3 Sun, Ultra, Solaris, 2 hpc420-2.hpcc.jp,
AIST, Tokyo, Japan 377 Time-shared 3 125.66
R4 Intel, Pentium/VC820,
Linux, 2 barbera.cnuce.cnr.it,
CNR, Pisa, Italy 380 Time-shared 2 190.0
R5 SGI, Origin 3200, IRIX, 6 onyx1.zib.de,
ZIB, Berlin, Germany 410 Time-shared 5 82.0
R6 SGI, Origin 3200, IRIX,
16 Onyx3.zib.de,
ZIB, Berlin, Germany 410 Time-shared 5 82.0
R7 SGI, Origin 3200, IRIX,
16
mat.ruk.cuni.cz, Charles U., Prague,
Czech Republic 410 Space-shared 4 102.5
R8 Intel, Pentium/VC820,
Linux, 2
marge.csm.port.ac.uk,
Portsmouth, UK 380 Time-shared 1 380.0
R9 SGI, Origin 3200, IRIX, 4
(accessible) green.cfs.ac.uk, Manchester, UK
410 Time-shared 6 68.33
R10 Sun, Ultra, Solaris, 8, pitcairn.mcs.anl.gov, ANL, Chicago, USA
377 Time-shared 3 125.66
58
DBC Cost Optimisation
5000
9000
13000
17000
21000
10
0
60
0
11
00
16
00
21
00
26
00
31
00
36
00
0
20
40
60
80
100
120
140
160
180
200
Gridlets
Budget
Deadline
100
600
1100
1600
2100
2600
3100
3600
5000
9000
13000
17000
21000
10
0
60
0
11
00
16
00
21
00
26
00
31
00
36
00
0
500
1000
1500
2000
2500
3000
3500
4000
Deadline Time Utilised
Budget
Deadline
100
600
1100
1600
2100
2600
3100
3600
50
00
70
00
90
00
11
00
0
13
00
0
15
00
0
17
00
0
19
00
0
21
00
0
10
0
11
00
21
00
31
00
0
5000
10000
15000
20000
25000
Budget Spent
Budget
Deadline
100
600
1100
1600
2100
2600
3100
3600
59
DBC Time Optimisation
5000
8000 11
000 1400
0
1700
0
2000
0
100
600
1100
1600
2100
2600
3100
3600
0
20
40
60
80
100
120
140
160
180
200
Gidlets Completed
Budget
Deadline
100
600
1100
1600
2100
2600
3100
3600
Time Optimise
5000
7000
9000
1100
0
1300
0
1500
0
1700
0
1900
0
2100
0
100
1100 21
00 3100
0
500
1000
1500
2000
2500
3000
3500
4000
Time Utilised
BudgetDeadline
100
600
1100
1600
2100
2600
3100
3600
Time-Optimise
5000
9000
13000
17000
21000
100
600
1100
1600
2100
2600
3100
3600
0
5000
10000
15000
20000
25000
Budget Spent
Budget
Deadline
100
600
1100
1600
2100
2600
3100
3600
Time Optimise
60
Comparison: D = 3100, B varied
0
500
1000
1500
2000
2500
3000
3500
5000
7000
9000
1100
0
1300
0
1500
0
1700
0
1900
0
2100
0
Budget
Dea
dli
ne
Sp
ent
Cost Optimisation
Time Optimisation
Deadline = 3100 Completion Time
0
5000
10000
15000
20000
25000
5000
7000
9000
1100
0
1300
0
1500
0
1700
0
1900
0
2100
0
Budget
Bu
dg
et S
pen
t
Cost Optimisation
Time Optimisation
Deadline = 3100 Processing Expenses
Time Opt
Execution Time vs. Budget
Execution Cost vs. Budget
Cost Opt
Conclude with a comparison to the Electrical Grid………..
Where we are ????
Courtesy: Domenico Laforenza
Alessandro Volta in Paris in 1801 inside French National Institute shows the battery
while in the presence of Napoleon I
Fresco by N. Cianfanelli (1841) (Zoological Section "La Specula" of National History Museum of Florence
University)
63
….and in the future, I imagine a WorldwidePower (Electrical) Grid …...
What ?!?!This is a mad man…
Oh, monDieu !
64
2002 - 1801 = 201 Years
65
Electric Grid Management and Delivery methodology is highly
advanced
Central Grid
Regional Grid
Regional Grid
Local Grid
Local Grid
Production Utility
Consumption
Whereas, our Computational Grid is in primitive/infancy state?
66
” I think there is a world market for about five computers.”
Thomas J. Watson Sr., IBM Founder, 1943
Can we Predict its Future ?
67
Summary and Conclusion
Grid Computing is emerging as a next generation computing platform for solving large scale problems through sharing of geographically distributed resources.
Resource management is a complex undertaking as systems need to be adaptive, scalable, competitive,…, and driven by QoS.
We proposed a framework based on “computational economies” for resource allocation and for regulating supply-and-demand for resources.
Scheduling experiments on the World Wide Grid demonstrate our Nimrod-G broker ability to dynamically lease services at runtime based on their quality, cost, and availability depending on consumers QoS requirements.
Easy to use tools for creating Grid applications are essential to attracting and getting application community on board.
The use of economic paradigm for resource management and scheduling is essential for pushing Grids into mainstream computing and weaving the World-Wide Grid Marketplace!
68
Download Software & Information
Nimrod & Parameteric Computing: http://www.csse.monash.edu.au/~davida/nimrod/
Economy Grid & Nimrod/G: http://www.buyya.com/ecogrid/
Virtual Laboratory Toolset for Drug Design: http://www.buyya.com/vlab/
Grid Simulation (GridSim) Toolkit (Java based): http://www.buyya.com/gridsim/
World Wide Grid (WWG) testbed: http://www.buyya.com/ecogrid/wwg/
Cluster and Grid Info Centres: www.buyya.com/cluster/ || www.gridcomputing.com
69
Selected GridSim Users!
70
Final Word?
Backup Slides
72
Further Information
Books: High Performance Cluster Computing, V1,
V2, R.Buyya (Ed), Prentice Hall, 1999. The GRID, I. Foster and C. Kesselman (Eds),
Morgan-Kaufmann, 1999. IEEE Task Force on Cluster Computing
http://www.ieeetfcc.org Global Grid Forum
www.gridforum.org
IEEE/ACM CCGrid’xy: www.ccgrid.org CCGrid 2002, Berlin: ccgrid2002.zib.de
Grid workshop - www.gridcomputing.org
73
Further Information
Cluster Computing Info Centre: http://www.buyya.com/cluster/
Grid Computing Info Centre: http://www.gridcomputing.com
IEEE DS Online - Grid Computing area:
http://computer.org/dsonline/gc
Compute Power Market Project http://www.ComputePower.com
74
75
Deadline and Budget-based Cost Minimization Scheduling
1. Sort resources by increasing cost.2. For each resource in order, assign as
many jobs as possible to the resource, without exceeding the deadline.
3. Repeat all steps until all jobs are processed.
76
Deadline and Budget Constraint (DBC) Time Minimization
Scheduling
1. For each resource, calculate the next completion time for an assigned job, taking into account previously assigned jobs.
2. Sort resources by next completion time.3. Assign one job to the first resource for
which the cost per job is less than the remaining budget per job.
4. Repeat all steps until all jobs are processed. (This is performed periodically or at each scheduling-event.)
77
DBC Conservative Time Min. Scheduling
1. Split resources by whether cost per job is less than budget per job.
2. For the cheaper resources, assign jobs in inverse proportion to the job completion time (e.g. a resource with completion time = 5 gets twice as many jobs as a resource with completion time = 10).
3. For the dearer resources, repeat all steps (with a recalculated budget per job) until all jobs are assigned.
4. [Schedule/Reschedule] Repeat all steps until all jobs are processed.
78
M - Resources, N - Jobs, D - deadline Note: Cost of any Ri is less than any of Ri+1 …. Or Rm
RL: Resource List need to be maintained in increasing order of cost Ct - Time when accessed (Time now) Ti - Job runtime (average) on Resource i (Ri) [updated periodically]
Ti is acts as a load profiling parameter. Ai - number of jobs assigned to Ri , where:
Ai = Min (No.Unassigned Jobs, No. Jobs Ri can complete by remaining deadline) No.UnAssignedJobsi = Diff( N, (A1+…+Ai-1)) JobsRi consume = RemainingTime (D- Ct) DIV Ti
ALG: Invoke Job Assignment() periodically until all jobs done. Job Assignment()/Reassignment():
Establish ( RL, Ct , Ti , Ai ) dynamically – Resource Discovery. For all resources (I = 1 to M) { Assign Ai Jobs to Ri , if required}
Deadline-based Cost-minimization Scheduling
79
What is Grid ?
An infrastructure that logically couples distributed resources:
Computers – PCs, workstations, clusters, supercomputers, laptops, notebooks, mobile devices, PDA, etc;
Software – e.g., ASPs renting expensive special purpose applications on demand;
Catalogued data and databases – e.g. transparent access to human genome database;
Special devices – e.g., radio telescope – SETI@Home searching for life in galaxy.
People/collaborators. and presents them as an integrated global resource
for solving large-scale problems. It enables the creation of virtual enterprise (VE) for
resource sharing and aggregation.
Widearea
data archives
80
Virtual Enterprise
A temporary alliance of enterprises or organizations that come together to share resources and skills, or competencies in order to better respond to business opportunities or challenges, and who cooperation is supported by computer
networks.
81
Many Testbeds ? & who pays ?,
who regulates supply and demand ?
GUSTO (decommissioned)
Legion Testbed
NASA IPG
World Wide Grid
WW Grid
82
Testbeds so far -- observations
Who contributed resources & why ? Volunteers: for fun, challenge, fame, charismatic apps, public
good like distributed.net & SETI@Home projects. Collaborators: sharing resources while developing new
technologies of common interest – Globus, Legion, Ninf, Ninf, MC Broker, Lecce GRB,... Unless you know lab. leaders, it is impossible to get access!
How long ? Short term: excitement is lost, too much of admin. Overhead
(Globus inst+), no incentive, policy change,… What we need ? Grid Marketplace!
Regulates supply-and-demand, offers incentive for being players, simple, scalable solution, quasi-deterministic – proven model in real-world.
83
Grid Open Trading Protocols
Get Connected
Call for Bid(DT)
Reply to Bid (DT)
Negotiate Deal(DT)
Confirm Deal(DT, Y/N)
….
Cancel Deal(DT)
Change Deal(DT)
Get Disconnected
Trade Manager
Trade Server
Pricing Rules
DT - Deal Template: - resource requirements (TM) - resource profile (TS) - price (any one can set) - status - change the above values - negotiation can continue - accept/decline - validity period
API
84
Layered Grid Architecture
Networked Resources across Organizations
Computers Networks Data Sources Scientific InstrumentsStorage Systems
Local Resource Managers
Operating Systems Queuing Systems Internet ProtocolsLibraries & App Kernels
Distributed Resources Coupling Services
Information QoSProcess
Development Environments and Tools
Languages/Compilers Libraries Debuggers Web tools
Resource Management, Selection, and Aggregation (BROKERS)
Applications and Portals
Prob. Solving Env.Scientific…CollaborationEngineering Web enabled Apps
Trading
…
…
…
…
FABRIC
APPLICATIONS
SECURITY LAYER
Security Data
CORE MIDDLEWARE
USER LEVEL MIDDLEWARE
Monitors
85
GridFabric
GridApps.
GridMiddleware
GridTools
Networked Resources across Organisations
Computers Clusters Data Sources Scientific InstrumentsStorage Systems
Local Resource Managers
Operating Systems Queuing Systems TCP/IP & UDP
…
Libraries & App Kernels …
Distributed Resources Coupling Services
Security Information … QoSProcess
Development Environments and Tools
Languages Libraries Debuggers … Web toolsResource BrokersMonitoring
Applications and Portals
Prob. Solving Env.Scientific …CollaborationEngineering Web enabled Apps
Resource Trading
Grid Components
Market Info
86
Economy Grid = Globus + GRACE
Applications
GRAM
Globus Security Interface (GSI)
Local Services
LSF
Condor GRD QBank
PBS
TCP
SolarisIrixLinux
UDP
High-level Services and Tools
Cactus MPI-G
Nimrod-G Broker
CC++
GASS GTSGARA
GridFabric
GridApps.
GridMiddleware
GridTools
GBankGMD
eCash
JVM
DUROC
Core Services
Science
Engineering Commerce Portals ActiveSheet……
……
MDS
Higher Level Resource Aggregators
Nimrod Parametric Language
87
Virtual Drug DesignA Virtual Lab for “Molecular Modeling for Drug Design” on P2P Grid
“Screen 2K molecules in 30min. for $10”
Grid Market Directory
ResourceBroker
Grid Info. Service
GTS
GTS
GTS
GTS
“Give me list PDBs sourcesOf type aldrich_300?”
“serv
ice co
st?”
(GTS - Grid Trade Server)
PDB2
“get mol.10 from pdb1 & screen it.”
Data Replica Catalogue
“service providers?”
GTS
PDB1
“mol.10 please?”
“mol.5 please?”
(RB maps suitable Grid nodes and Protein DataBank)
88
P-study Applications -- Characteristics
Code (Single Program: sequential or threaded)
High Resource Requirements Long-running Instances Numerous Instances (Multiple Data) High Computation-to-Communication
Ratio Embarrassingly/Pleasantly Parallel
89
Many Grid Projects & Initiatives
Australia Nimrod-G GridSim Virtual Lab Active Sheets DISCWorld ..new coming up
Europe UNICORE MOL UK eScience Poland MC Broker EU Data Grid EuroGrid MetaMPI Dutch DAS XW, JaWS and many more...
Japan Ninf DataFarm and many more...
USA Globus Legion OGSA Javelin AppLeS NASA IPG Condor-G Jxta NetSolve AccessGrid and many more...
Cycle Stealing & .com Initiatives Distributed.net SETI@Home, …. Entropia, UD, Parabon,….
Public Forums Global Grid Forum P2P Working Group IEEE TFCC Grid & CCGrid conferences
http://www.gridcomputing.com
90
Using Pure Globus/Legion commands
Do all yourself! (manually)
Total Cost:$???
91
Build Distributed Application & Scheduler
Build App case by case basisComplicated Construction
E.g., AppLeS/MPI based Total Cost:$???
92
Experiment-3 Setup
Workload: 200 jobs, each need 10 minute of CPU time
Deadline: 4 hrs. and budget: 250,000 G$ Strategies: 1. Minimise cost 2. Minimise time Execution:
Optimise Cost: 141,869 (G$) (finished in 150min./2.5hrs)
Optimise Time: 199,968 (G$) (finished in 250min.) In this experiment: Time-optimised scheduling run
costs double that of Cost-optimised. Users can now trade-off between Time Vs. Cost.
93
Number of Jobs Executed Organization &
Location
Vendor, Resource Type, # CPU, OS,
hostname
Grid Services,
Fabric, and Role
Price (G$ per
CPU sec.)
TimeOpt
CostOpt
Monash University, Melbourne, Australia
Sun: Ultra-1, 1 node, bezek.dstc.monash.edu.au
Globus, Nimrod-G, CDB Server, Fork (Master node)
-- -- --
VPAC, Melbourne, Australia
Compaq: Alpha, 4 CPU, OSF1, grendel.vpac.org
Globus, GTS, Fork (Worker node)
1 7 59
AIST, Tokyo, Japan Sun: Ultra-4, 4 nodes, Solaris, hpc420.hpcc.jp
Globus, GTS, Fork (Worker node)
2 14 2
AIST, Tokyo, Japan Sun: Ultra-4, 4 nodes, Solaris, hpc420-1.hpcc.jp
Globus, GTS, Fork (Worker node)
1 7 3
AIST, Tokyo, Japan Sun: Ultra-2, 2 nodes, Solaris, hpc420-2.hpcc.jp
Globus, GTS, Fork (Worker node)
1 8 50
University of Lecce, Italy
Compaq: Alpa cluster, OSF1, sierra0.unile.it
Globus, GTS, RMS (Worker node)
2 0 0
Institute of the Italian National Research Council, Pisa, Italy
Unknown: Dual CPU PC, Linux, barbera.cnuce.cnr.it
Globus, GTS, Fork (Worker node)
1 9 1
Institute of the Italian National Research Council, Pisa, Italy
Unknown: Dual CPU PC, Linux, novello.cnuce.cnr.it
Globus, GTS, Fork (Worker node)
1 0 0
Konrad-Zuse-Zentrum Berlin, Berlin, Germany
SGI: Onyx2K, IRIX, 6, onyx1.zib.de
Globus, GTS, Fork (Worker node)
2 38 5
Konrad-Zuse-Zentrum Berlin, Berlin, Germany
SGI: Onyx2K, IRIX, 16 onyx3.zib.de
Globus, GTS, Fork (Worker node)
3 32 7
Charles University, Prague, Czech Republic
SGI: Onyx2K, IRIX, mat.ruk.cuni.cz
Globus, GTS, Fork (Worker node)
2 20 11
University of Portsmouth, UK
Unknown: Dual CPU PC, Linux, marge.csm.port.ac.uk
Globus, GTS, Fork (Worker node)
1 1 25
University of Manchester, UK
SGI: Onyx3K, 512 node, IRIX, green.cfs.ac.uk
Globus, GTS, NQS, (Worker node)
2 15 12
Argonne National Lab, Chicago, USA
SGI: IRIX lemon.mcs.anl.gov
Globus, GTS, Fork (Worker node)
2 0 0
Argonne National Lab, Chicago, USA
Sun: Ultra –8, Solaris, 8, pitcairn.mcs.anl.gov
Globus, GTS, Fork (Worker node)
1 49 25
Total Experiment Cost (G$) 199968 141869
Time to Finish Expt. (Min.) 150 258
94
Jobs Completed for DBC Time Optimization
0
10
20
30
40
50
60
0.0
0
5.4
3
11
.75
17
.85
24
.06
29
.62
35
.00
40
.87
47
.12
53
.37
59
.75
65
.08
70
.48
76
.71
83
.87
90
.42
97
.25
10
3.7
9
11
0.4
1
11
6.4
1
12
2.5
9
12
8.6
9
13
4.4
4
14
0.2
2
14
5.6
7
Time (in min.)
No
. of
Jo
bs
Fin
ish
ed
grendel.vpac.org
hpc420.hpcc.jp
hpc420-1.hpcc.jp
hpc420-2.hpcc.jp
sierra0.unile.it
barbera.cnuce.cnr.it
novello.cnuce.cnr.it
onyx1.zib.de
onyx3.zib.de
mat.ruk.cuni.cz
marge.csm.port.ac.uk
green.cfs.ac.uk
lemon.mcs.anl.gov
pitcairn.mcs.anl.gov
95
Jobs Completed for DBC Cost Optimization
0
5000
10000
15000
20000
25000
30000
35000
400000
.00
15
.05
31
.40
49
.05
66
.22
82
.70
99
.38
11
5.8
2
13
3.0
9
14
9.7
3
16
6.5
1
18
2.4
6
19
9.8
7
21
5.8
4
23
0.9
5
24
6.0
8Time (in min.)
Bu
dg
et
Sp
en
t
grendel.vpac.org
hpc420.hpcc.jp
hpc420-1.hpcc.jp
hpc420-2.hpcc.jp
sierra0.unile.it
barbera.cnuce.cnr.it
novello.cnuce.cnr.it
onyx1.zib.de
onyx3.zib.de
mat.ruk.cuni.cz
marge.csm.port.ac.uk
green.cfs.ac.uk
lemon.mcs.anl.gov
pitcairn.mcs.anl.gov
96
Active Sheet:Microsoft Excel Spreadsheet Processing on Grid
NimrodNimrodProxyProxy
Nimrod-GNimrod-G
World-Wide Grid