HPC Technology Track:Foundations of Computational Science

Lecture 1

Dr. Greg Wettstein, Ph.D.

Research Support Group LeaderDivision of Information Technology

Adjunct ProfessorDepartment of Computer Science

North Dakota State University

What is Computational Science?

Application of computational methods to fundamental problems in science and engineering.

Concerned with application of computational methods to one of three objectives.

Modeling Simulation Prediction

Modeling

Involves development of a mathematical model capable of predicting physical phenomenon.

weather prediction molecular energy and force fields properties of materials

Typically involves solving systems of differential equations expressed in linear form.

Challenges in Modeling

Currently only order 0 systems can be solved directly.

Computational approaches are frequently implemented with methods involving truncated series.

Garbage In/Garbage out (GIGO) Limitations constantly demand attention to model

parameterization and viability. The field considers better differential equation

solvers to be essential.

Emerging Opportunities in Modeling

Model parameter evaluation. Considered important by the national laboratories Involves computation of first and second

derivatives of the model with respect to finalized model parameters

Important focus is to determine quality or stability of model.

** Model evaluation is critical.

Multi-Scale Studies

Defn: Calculation of system behavior or properties on one level using information from subordinate levels.

Continuum of levels (physical systems): quantum mechanical molecular dynamics meso or nanoscale levels level of continuum level of device

Simulation

Focuses on simulating the behavior of physical systems.

Usually involves Monte Carlo methods to solve stochastic systems.

Most commonly employed in computational physics.

Simulation – con't.

Central to the 'birth' of computational science. Metropolis, Rosenbluth, Rosenbluth, Teller and

Teller “Equation of State Calculations by Fast Computing

Machines” Goal is to develop 'ensembles' or collections of

parameters. Typically implemented as 'coarse grained'

parallelism.

Prediction

The analysis or 'mining' of large sets of data for the purpose of predicting future phenomenon.

Centrally important to marketing and e-commerce. Represents a type of computational problem referred

to as 'embarassingly' parallel. Most famous example is NetFlix competition.

Challenges in Prediction

Data locality More processors does not equal more speed. NetFlix competition demonstrated inadequacy of

improperly 'balanced' computational architectures. Primary concern of national labs involved in

security based computation. Current HPC architectures exacerbate data locality

problems.

'The Wettstein Rule of Computational Reality'

“If filling a cache line is too slow you willbe really unhappy doing a cross-nodelookup to a machine 200 racks away.”

RoadRunner Configuration

1.72 petaflops peak / 1.456 petaflops demonstrated 296 racks covering 6,000 square feet Massively parallel – hybrid architecture

6,480 Opteron (x86) processors 12,960 IBM PowerXCell processors 122,400 cores

103.6 terrabytes of memory

But when is it fast?

A Tradeoff

Compelling speeds when each node can work on a discrete element of the problem.

Strictly orthogonal decomposition Embarrassingly parallel problems.

MIMD Less efficient when.

Boundary condition dependent problem. Access to entire memory space is required.

LatencyThe Enemy of Prediction

Latency definition: The amount of time required to retrieve the next

relevant item of data required in a computational or predictive sequence.

'Achilles Heel' of modern massively parallel systems such as RoadRunner.

Common problem since the design of the Cray-1. Wiring optimized to place time critical connections

on the inner portion of the computer.

Reducing Latencies through PTree's

Current area of research interest. Addition of second order PTree's to optimize data

selection decisions. Minimizes:

cache line flushing cross-node data lookups

Exercise

Log into cluster1.chpc.ndsu.nodak.edu. Use sinfo command to locate an available node.

e.g. node64-49 Use ping command to measure message latency

over standard TCP/IP network. ping -c 5 node64-49

Use ping command to measure message latency over Myrinet:

ping -c 5 node64M-49

Exercise – con't.

Bottom of ping command details min, average and maximum communication latencies.

Compute expected performance change if a computation is constrained by the length of time required to pass a message from one node to another.

goto technology_lecture1_2;