CCA Common Component Architecture Manoj Krishnan Pacific Northwest National Laboratory MCMD...
-
Upload
meredith-barnett -
Category
Documents
-
view
215 -
download
0
Transcript of CCA Common Component Architecture Manoj Krishnan Pacific Northwest National Laboratory MCMD...
CCACommon Component Architecture
Manoj Krishnan
Pacific Northwest National Laboratory
MCMD Programming and Implementation Issues
CCACommon Component Architecture
2
Motivation
• The challenges in developing large-scale applications are …– Addressing complexity
• Improve productivity
– Scaling to massive number of processors• How applications can exploit the massive amount of
parallelism available in teraflop and petaflop-scale systems
CCACommon Component Architecture
3
Multilevel Parallelism in Computational Chemistry: Our Approach
• Proposed solution to improve scalability– Increase granularity of computation => improve the overall
scalability.– Exploitation of multiple levels of parallelism (MLP)
• Instead of execution entire application on the full set of processors, assign parts of application to appropriately-sized subsets of processors
• Many apps qualify
– Challenge: Difficult to implement
• Use advanced tools to address programming complexity• Common Component Architecture (CCA)
• Global Arrays (GA) shared-memory programming model
• Objective: To demonstrate how CCA and GA can be used together to address requirements of real scientific applications
CCACommon Component Architecture
4
Technology
• Technologies for exploiting multiple level parallelism– Global Arrays (GA) shared-memory programming
model• High level parallel data management abstractions
– Common Component Architecture (CCA)• Component technology for HPC applications• Hiding complexity• Enables composition of software modules written in
different languages and programming styles
Driver
Gradient GradientGradient Gradient
Energy EnergyEnergy
EnergyEnergyEnergy
Energy
Energy
Energy Energy Energy Energy
CCA
QM
CCACommon Component Architecture
5
Multiple Component Multiple DataModel
• Introducing Multiple Component Multiple Data– i.e. multiple program multiple data (MPMD) model in
context of CCA– instantiating components on subgroups of processors – create a dynamic environment to partition
computational resources and manage them to execute the overall application effectively
• Facilitate dynamic behavior of the application itself for example – Resizing processor groups based on memory
requirements or scaling characteristics– swapping components based on numerical or
computational performance
CCACommon Component Architecture
6
Numerical Hessian Example
0.1
1
10
100
0 32 64 96 128 160 192 224 256 288
Processors
Tim
e (
ho
urs
)one-level (native)
• Numerical Hessian Algorithm– determination of energy
second derivatives through numerical differentiation of gradients, which may in turn be obtained from numerical differentiation of energies
• Multiple gradient calculations– Each gradient has multiple
energy calculations
• limited scalability• Not effectively utilizing variable
degrees of parallelism
Gradient
Energy
Hessian
CCACommon Component Architecture
7
Numerical Hessian Scalability - I
0
20
40
60
80
100
120
1 2 4 8 16 32 64 128
Processors
Para
llel E
ffic
ien
cy
Single Energy CalculationSingle Energy Calculation
0.1
1
10
100
0 32 64 96 128 160 192 224 256 288
Processors
Tim
e (
ho
urs
)
one-level (native)
two-level (groups)
• Single energy calculation does not scale beyond 4 processes*
• Two-level Parallelism– Native parallel code – Energy
level– group-based energy
calculations at gradient level• using GA processor groups
QM Gradient
Energy
Energy
Energy
0
10
20
30
40
50
60
70
16 32 64 128 256
Processors
% o
verh
ead native-groups
CCACommon Component Architecture
8
Multilevel Parallelism
• Combining SPMD and MPMD paradigms
• MCMD – Multi Component Multiple Data• MPMD + Component
• The MCMD Driver launches multiple instances of NWChem QM components on subsets of processors (CCA)
• Each NWChem QM (gradient) component does multiple energy computations on subgroups (GA)
MCMD Hessian DriverMCMD Hessian DriverGoGocProps
cPropsModelFactory
ModelFactoryNWChem_QM_1NWChem_QM_1
ModelFactory
ModelFactory
cPropscProps
Param PortParam Port
EnergyEnergy
EnergyEnergy
EnergyEnergy
EnergyEnergy
EnergyEnergy
EnergyEnergy
EnergyEnergy
EnergyEnergy
EnergyEnergy
EnergyEnergy
EnergyEnergy
EnergyEnergy
NWChem_QM_0NWChem_QM_0
ModelFactory
ModelFactory
cPropscProps
Param PortParam Port
NWChem_QM_2NWChem_QM_2
ModelFactory
ModelFactory
cPropscProps
Param PortParam Port
NWChem_QM_nNWChem_QM_n
ModelFactory
ModelFactory
cPropscProps
Param PortParam Port
Driver
Gradient GradientGradient Gradient
Energy EnergyEnergy
EnergyEnergyEnergy
Energy
Energy
Energy Energy Energy Energy
CCA
QMGradient
Energy
Energy
Energy
CCACommon Component Architecture
9
Multiple Component Multiple Data (CCA’s MCMD Model)
MCMD DriverMCMD DriverGoGocProps
cPropsModelFactory
ModelFactory
BuilderBuilder
Builder ServiceBuilder ServiceBuilder
BuildercPropscProps
QM_0QM_0ModelFact
ory
ModelFactory
cPropscProps
Parameter Parameter
QM_0QM_0ModelFact
ory
ModelFactory
cPropscProps
Parameter Parameter
QM_0QM_0ModelFact
ory
ModelFactory
cPropscProps
Parameter Parameter
QM_0QM_0ModelFact
ory
ModelFactory
cPropscProps
Parameter Parameter
MCMD Driver• Create new components • Create processor groups• Assign processor groups to
components• Connect components • Collect results
EnergyEnergy
EnergyEnergy
EnergyEnergy
EnergyEnergy
EnergyEnergy
EnergyEnergy
EnergyEnergy
EnergyEnergy
EnergyEnergy
EnergyEnergy
EnergyEnergy
EnergyEnergy
Collect Results
CCACommon Component Architecture
10
Numerical Hessian Scalability - II
0.1
1
10
100
0 32 64 96 128 160 192 224 256 288
Processors
Tim
e (
ho
urs
)
one-level (native)
two-level (groups)
three-level (groups + CCA)
Application efficiency improved 10x times on 256
CPUs
Driver
Gradient GradientGradient Gradient
Energy EnergyEnergy
EnergyEnergyEnergy
Energy
Energy
Energy Energy Energy Energy
CCA
QM
• Three-level ParallelismThree-level Parallelism• Energy-Level
– Native parallel code• Gradient-Level
– group-based single energy calculations using GA groups
• Hessian Level– Task-based gradient
calculations using CCA
CCACommon Component Architecture
11
Potential Applications Relevant To This Approach
• Molecular Dynamics• Monte Carlo
– Growth nucleation
• Numerical Hessians– Vibrational spectra
• Optimization techniques– Simulated annealing with local optimization
• Nudged Elastic Band methods– Determine reaction path for kinetic rates
• Trajectory simulations
CCACommon Component Architecture
12
MCMD Programming
• Multi-level parallelism– Nested parallel decomposition– Possibly multiple levels of parallelism– Multiple parallel simulations are run concurrently in
a coupled fashion, exchanging data at boundaries or perhaps even within volumes.
CCACommon Component Architecture
13
MCMD Services
• Develop MCMD services to support MLP– Creating and management of processor groups
• CCA Represenation for Groups id, membership
– Mapping of component to groups and their coordination• Coordination of concurrent and nested SCMD/MCMD tasks
– Communication between groups– Dynamic reconfiguration– Handling termination of processor groups, components
• MCMD as a service or a component ?
CCACommon Component Architecture
14
Activities
• Year 1:– Develop a model to express Multi-level parallelism through
processor groups– Requirements gathering and design of flexible dynamic
multi-level parallelism model– Coordinate & interact with other initiatives (ongoing)
• Year 2– Define a CCA Standard way of specifying and translating
processor group membership and mapping between components
• Year 3, 4, 5.– …
CCACommon Component Architecture
15
Implications of MCMD for CCA model
• Model for Applications with Multi-Level Parallelism – Important
• Process group abstraction – compatible with MPI, PVM, GA, GAS languages, HPCS languages (?)– MPI as default ? Group translators– How to address threaded components? OpenMP?
Pthreads? Processor group for a threaded component?
• Group-awareness to CCA and a CCA way of naming groups– i.e. multi-level parallelism at the CCA level/BuilderService
CCACommon Component Architecture
16
Implications of MCMD for CCA Implementations
• Processor group management• Run-time configuration
– At run-time, user should be able to blow-up connections, create components and assign groups
– Swapping components, ..
• Mapping communicators• Overlapping/Disjoint processor groups
CCACommon Component Architecture
17
Summary - Found MCMD Effective
• Implemented a flexible, multi-level software architecture for computational chemistry applications – Exploits variable levels of parallelism – A order of magnitude of performance improvement
• Hiding complexity and enabling better s/w composition
• MCMD model has potential for addressing scalability in future large scale systems
• More work is needed in CCA infrastructure and s/w to take advantage for larger class of apps
– Facilitate dynamic groups• Make MCMD easier to adopt for apps