Exploring the Design Tradeoffs for Exascale System Services through Simulation
description
Transcript of Exploring the Design Tradeoffs for Exascale System Services through Simulation
Exploring the Design Tradeoffs for Exascale System Services
through Simulation
Ke WangUSRC Group, Los Alamos National Laboratory
Summer Internship ResultsCollaborated with DataSys Lab, IIT
August 16th, 2012
Acknowledgements
• DataSys Laboratory• Dr. Ioan Raicu• Michael Lang, USRC leader• Abhishek Kulkarni, Ph.D student of Indiana University• Poster Submission:
– Ke Wang, Abhishek Kulkarni, Michael Lang, Ioan Raicu, Andrew Lumsdaine, “Exploring the Design Tradeoffs for Exascale System Services through Simulation”, under review at SC12
Exploring the Design Tradeoffs for Exascale System Services through Simulation 2
Outline
• Introduction & Motivation• Long-Term Aims and Contributions• System Services Taxonomy• Peer-to-Peer System Simulators• Simulating System Services• Related Work• Contributions• Future Work & Conclusion
Exploring the Design Tradeoffs for Exascale System Services through Simulation 3
Outline
• Introduction & Motivation• System Services Taxonomy• Peer-to-Peer System Simulators• Simulating System Services• Related Work• Contributions• Future Work & Conclusion
Exploring the Design Tradeoffs for Exascale System Services through Simulation 4
Distributed System Services
• Operating System: a service provider offers basic services, such as Program development, Access to I/O devices, Controlled access to files, System access and Program execution
• Generalized distributed system services involve many servers coordinated with each other to offer different services to a lot of clients
• Typical services: key-value store, job scheduler, file servers, application job launch
• Key Issues: Scalability, Dynamicity, Resiliency, Consistency, Fault Tolerance
Exploring the Design Tradeoffs for Exascale System Services through Simulation 5
Exascale Computing
Top500 Performance Development,
http://top500.org/static/lists/2011/11/TOP500_201111_Poster.pdf 6
• Today (June 18, 2012): 16 Petaflop– O(100K) nodes (100X in the last 10 years) – O(1M) cores (1000X in the last 10 years)
• Near future (~2018): Exaflop Computing– ~1M nodes (10X) – ~1B processor-cores/threads (1000X)
Major Challenges of Exascale Computing
• Energy and Power– 7.89MW (Top 1 Supercomputer)– 20MW limitation
• Memory and Storage– Retain data at high enough capacities – Access data at high enough rates– Support the desired computational rate – Fit within acceptable power envelope
• Concurrency and Locality– Accelerators, GPUs, MIC– Programmability– Minimizing data movement
• Resiliency– MTTF decreases, MPI suffers
7Exploring the Design Tradeoffs for Exascale System Services through Simulation
Current HPC System Service
• Lack of decomposition in detail• Centralized server with at most a single
fail-over, for example Slurm (slurmctld, slurmd)
• Not clear about the scalability of different server topologies (centralized, hierarchical, distributed), either the costs of different resiliency and consistency models.
Outline
• Introduction & Motivation• Long-Term Aims and Contributions• System Services Taxonomy• Peer-to-Peer System Simulators• Simulating System Services• Related Work• Contributions• Future Work & Conclusion
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 9
Long-Term Aims
• Develop a simulator capable of simulating generic system services in supporting up to 1M nodes
• Compare the scalability of different server topology with or without churn property
• The costs of different resiliency models (fail over, replication) to different server topology under different failure rate
• The costs of different consistency models (strong/weak consistency) to different server topology
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 10
This Work’s Contributions– Deconstruct services into their most basic components, and
provide a general taxonomy to classify existing system services in terms of server architecture
– Investigate and compare different existing peer-to-peer simulators
– Simulate these service architectures at scale with millions of clients served by thousands of servers
– Estimate basic parameters such as memory consumption analytically, and complex parameters such as client-perceived throughput, server throughput, and overall system efficiency
– Demonstrate how churn property affects the performance and efficiency of the system under different distributed service architectures
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 11
Outline
• Introduction & Motivation• Long-Term Aims and Contributions• System Services Taxonomy• Peer-to-Peer System Simulators• Simulating System Services• Related Work• Contributions• Future Work & Conclusion
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 12
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales
System Service Blocks• We deconstruct services into their basic blocks to understand the design
tradeoffs for exascale system services• The taxonomy proposed is still being defined and modified as we investigate
more HPC services • Components
– Service Model: define overall behavior and constraints describe high-level functionality, its architecture and roles of entities ACID property, CAP property
– Data Model: define the distribution of persistent data Centralized, Distributed with different levels of replication Replication: partitioned(no replication), mirrored(full replication), overlapped (partial replication)
– Network Model: dictates how the components are connected Structured overlay: rings, binomial, k-ary, radix-trees, complete/binomial graphs Unstructured overlay: random graph Completed membership list (fully connected) vs Partial membership list (binomial graphs)
– Failure Model: how the servers handle failures Complete mirroring, triple modular redundancy
– Consistency Model: depends on data model and level of replication Strong, weak or eventual consistency
13
Services Architecture
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 14
SimMatrix Architecture
Client
Submit tasks
Submit tasks
ClientArbitrary Node
Figure 1: Simulation architectures; the left part is the centralized one with a single dispatcher connecting all nodes, the right part is the homogeneous distributed topology with each node having the same number of cores and neighbors
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 15
Dispatcher
Simulations
• Continuous time simulations– Abandoned the idea of creating a separate thread
per simulated node: we found that on our 48-core system with 256GB of memory, we were limited to 32K threads
• Discrete event simulations– The only viable approach (today) to explore
scheduling techniques at exascales (millions of nodes and billions of cores)
– Created a unique object per simulated node, and converted any behavior to an event
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 16
Outline
• Introduction & Motivation• Long-Term Aims and Contributions• SimMatrix Architecture• Implementation• Evaluation• Related Work• Contributions• Future Work & Conclusion
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 17
At the Heart of SimMatrixGlobal Event Queue
Figure 2: Event State Transition DiagramSimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 18
• All events are inserted to the queue, sorted based on the occurrence time ascending
• Handle the first event, advance the simulation time and update the event queue
• Implemented as red-black tree based “TreeSet” in Java, which ensures Θ(log ) 𝑛time for insert & remove
Simulator Features
• Node load information– Nested hash maps provides extremely fast
performance at large scales• Dynamic Task Submission
– Aims to reduce the memory foot-print• Dynamic Poll interval
– Exponential backoff to reduce the number of messages and increase speed of simulation
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 19
Implementation
• SimMatrix is developed in JAVA– Sun 64-bit JDK version 1.6.0_22– 1500 lines of code– Code accessible at:
• http://datasys.cs.iit.edu/projects/SimMatrix/index.html • SimMatrix has no other dependencies
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 20
Outline
• Introduction & Motivation• Long-Term Aims and Contributions• SimMatrix Architecture• Implementation• Evaluation• Related Work• Contributions• Future Work & Conclusion
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 21
Experiment Environment
• Fusion system:– fusion.cs.iit.edu– 48 AMD Opteron cores at 1.93GHz– 256GB RAM– 64-bit Linux kernel 2.6.31.5– Sun 64-bit JDK version 1.6.0_22
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 22
Metrics• Throughput
– Number of tasks finished per second. Calculated as total-number-of-tasks/simulation-time.
• Efficiency– The ratio between the ideal simulation time of completing a given workload
and the real simulation time. The ideal simulation time is calculated by taking the average task execution time multiplied by the number of tasks per core.
• Load Balancing– We adopted the coefficient variance of the number of tasks finished by each
node as a measure the load balancing. The smaller the coefficient variance, the better the load balancing is. It is calculated as the standard-deviation/average in terms of number of tasks finished by each node.
• Scalability– Total number of tasks, number of nodes, and number of cores supported.
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 23
Workloads
• Synthetic workloads: – Uniform distributions with different average task
lengths, such as 10s (ave_10), 100s (ave_100), 1000s (ave_1000), 5000s (ave_5000), 10000s (ave_10000), and 100000s (ave_100000); also all tasks of 1 sec each (all_1)
• Realistic application workloads: – General MTC workload from 2008-2009 trace of
173M tasks; average task length 64±486s (mtc_64), using Gamma Distribution
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 24
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales
Validation
Validate SimMatrix against the state-of-the-art MTC systems (e.g. Falkon), to ensure that the simulator can accurately predict the performance of current petascale systems. 25
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales
Comparing Work Stealing to Falkon’s Naïve Distributed Scheduler
26
Fine grained workloads:• 2% 99.3%
efficiency increase
Coarse grained workloads:• 99%
99.999% efficiency increase
Scalability1M Nodes and 10B tasks
Memory consumption• <13 KB/task• <200 GB
CPU Time• <90 us/task• <260 hours
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 27
Scalability1M Nodes and 10B tasks
Efficiency• 90%+
Co-variance• <0.06• Load
imbalance of <600 tasks from 10K tasks per node
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 28
Work Stealing ParametersNumber of Tasks to Steal
29
Stealing half of neighbor’s work is best
strategy!0%10%20%30%40%50%60%70%80%90%100% No. of Tasks to Steal
steal_1steal_2steal_logsteal_sqrtsteal_half
No. of Nodes
Effici
ency
Work Stealing ParametersNumber of Neighbors (Static)
30
Requires linear number of neighbors for good
performance!
0%10%20%30%40%50%60%70%80%90%100% No. of Static Neighbors
nb_2nb_lognb_sqrtnb_eighthnb_quarnb_half
No. of Nodes
Effici
ency
Work Stealing ParametersNumber of Neighbors (Dynamic Random)
31
An increasing number of neighbors are needed for 90%+ efficiency, with the largest scales requiring square root neighbors (e.g. 1K
neighbors from 1M nodes!0%10%20%30%40%50%60%70%80%90%
100% No. of Dynamic Random Neighbors
nb_1nb_2nb_lognb_sqrt
No. of Nodes
Effici
ency
Work Stealing ParametersOptimal Parameters Generality
32
The same optimal parameters achieve 90%+ efficiency across many different
workloads!0%10%20%30%40%50%60%70%80%90%
100% Different Workloads
ave_1064+/-486ave_100ave_1000
No. of Nodes
Effici
ency
Work StealingThroughput
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 33
Centralized scheduling has severe bottleneck, especially for workload with fine granularity. Distributed scheduling has great scalability, for workload with coarse granularity, there is no obvious upper bound
0.1250.5
28
32128512
20488191.9999999999932767.9999999999
131072524287.9999999982097151.999999998388607.9999999833554431.9999998134217727.999999
Centralized(ave_5000)Distributed(ave_5000)Centralized(all_1)Distributed(all_1)
No. of Nodes
Thro
ughp
ut(t
asks
/sec
)
0
100000
200000
300000
400000
500000
600000
700000
Ave No. of Messages / tasks
No. of Nodes
Aver
age
No.
of M
essa
ges p
er T
ask
Load Balancing Visualization1024 Nodes and Ave_5000 Workload
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 34
Good Load Balancing
Square Root Dynamic Neighbors
Starvation
Square Root Static Neighbors
Good Load Balancing
Quarter Static Neighbors
Starvation
2 Static Neighbors
Summary Plot for Distributed Scheduling
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 35
Steady state utilization is ~100% at exascales
Outline
• Introduction & Motivation• Long-Term Aims and Contributions• SimMatrix Architecture• Implementation• Evaluation• Related Work• Contributions• Future Work & Conclusion
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 36
Related Work
• Real Job Scheduling Systems: – Condor (University of Wisconsin), Bradley et al, 2012 – PBS (NASA Ames) , Corbatto et al, 2012 – LSF Batch (Platform Computing of Toronto), 2011– Falkon (University of Chicago), Raicu et al, SC07
• Job Scheduling System Simulators:– simJava (University of Edinburgh), Wheeler et al, 2004 – GridSim (University of Melbourne, Australia), Buyya et al, 2010
• Load Balancing: – Neighborhood averaging scheme, Sinha et al, 1993 – Charm++ (UIUC), Zheng et al, 2011
• Scalable Work Stealing– Dinan et al, SC09– Blumofe et al, Scheduling multithreaded computations by work stealing, 1994
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 37
Outline
• Introduction & Motivation• Long-Term Aims and Contributions• SimMatrix Architecture• Implementation• Evaluation• Related Work• Contributions• Future Work & Conclusion
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 38
Contributions
• Designed, Analyzed, and Implemented a discrete-event simulator (SimMatrix) enabling the study of MTC workloads at exascales
• Identified work stealing as a viable technique to achieve load balance at exascales
• Provided evidence that work stealing is scalable by finding optimal parameters affecting the performance of work stealing– Number of tasks to steal is half– Dynamic random neighbors strategy is required– There must be a squared root number of neighbors
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 39
Outline
• Introduction & Motivation• Long-Term Aims and Contributions• SimMatrix Architecture• Implementation• Evaluation• Related Work• Contributions• Future Work & Conclusion
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 40
Future Work
• Explore work stealing for manycore processors with 1000 cores
• Enhancing the network topology model to allow complex networks
• Insight from SimMatrix will be used to develop MATRIX, a distributed task execution fabric– MATRIX will employ work stealing for distributed load
balancing– MATRIX will be integrated with other projects, such as Swift
(a data-flow parallel programming systems) and FusionFS(a distributed file systems)
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 41
Conclusion
• Exascale systems bring great opportunities in unraveling of significant scientific mysteries
• There are significant challenges to achieve exascales, such as concurrency, resilience, I/O and memory, heterogeneity, and energy
• MTC requires a highly scalable and distributed task/job management system at large scales– Distributed scheduling is likely an efficient way to achieve load
balancing, leading to high job throughput and system utilization• Work stealing is a scalable method to achieve load
balance at exascales given the optimal parameters
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 42
• More information:– http://datasys.cs.iit.edu/~kewang/ – http://datasys.cs.iit.edu/projects/SimMatrix/
• Contact:– [email protected]
• Questions?
More Information
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales 43