Parallel Programming

Henri Bal

(bal@cs.vu.nl)

Vrije Universiteit

Faculty of Sciences

Amsterdam

Overview

• What is parallel programming?

• Why do we need parallel programming?

• Organization of this course

• Practicum Parallel Programming

ICWall

•A large display suitable for:– Education (classroom)– Modern scientific applications thatneed to visualize large data-sets

– 3D applications

ICWall: an "off-the-shelf" video wall

•High-end graphics is very expensive•Use tiled display with many projectors•Use standard (inexpensive) components

– 9 PCs, graphic and network cards, 16 projectors

•Similar idea as cluster computing, but applied to graphics

•Current ICWall is strongly simplified version–1 PC with 2 projectors

Parallel Programming

• Sequential programming

Single thread of control

• Parallel programming

Multiple threads of control

• Why parallel programming?

Eases programming? Not really.

Performance? Yes!

Famous quote

• “Parallel programming may do something to revive the pioneering spirit in programming,which seems to be degenerating intoa rather dull and routine occupation’’

• S. Gill, Computer Journal, 1958

Why do we need parallel processing?

• Many applications need much faster machines

• Sequential machines are reaching their speed limits

• Use multiple processors to solve large problems fast

• Microprocessors are getting cheaper and cheaper

• Cheap multiprocessors and multicore CPUsbring parallel processing to the desktop!

History

• 1950s: first ideas (see Gill’s quote)

• 1967 first parallel computer (ILLIAC IV)

• 1970s programming methods, experimental machines

• 1980s: parallel languages (SR, Linda, Orca), commercial supercomputers

• 1990s: software standardization (MPI), clusters, large-scale machines (Blue Gene)

• 2000s: grid computing: combining resources world-wide (Globus)

Large-scale parallel machines

• Many parallel machines exist

• See http://www.top500.org

• Current #1 is IBM BlueGene/L with 131072 CPUs

131072 131072

Challenging Applications• Modeling ozone layer, climate, ocean• Quantum chemistry• Protein folding• General: computational science• Aircraft modeling• Handling use volumes of data from scientific instruments

– Lofar (astronomy)– LHC (CERN, high-energy physics)

• Computer chess• Analyzing multimedia content• Generating movies

About this Course

Goal: Study how to write programs that run in parallel on a large number of machines.

Focus on programming methods, languages, applications

Prerequisites:

Some knowledge about sequential languages

Little knowledge about networking and operating systems

Aspects of Parallel Computing

Algorithms and applications

Programming methods, languages, and environments

Parallel machines and architectures

Course Outline• Introduction in algorithms and applications• Parallel machines and architectures

Overview of parallel machines, trends in top-500Cluster computers, BlueGene

• Programming methods, languages, and environmentsMessage passing (SR, MPI, Java)Higher-level language: HPF

• ApplicationsN-body problems, search algorithms, bioinformatics

• Grid computing Multimedia content analysis on Grids (guest lecture Frank Seinstra)

Course Information

ExaminationWritten exam based on:- Reader: available electronically from Blackboard- Lectures

More information (and slides):http://www.cs.vu.nl/~bal/college07.html

Practicum Parallel Programming

Separate practicum (6 ECTS)

• Implement ASP + SOR algorithms in C/MPI

• Implement IDA* search algorithm in Java/RMI

• Test and measure the programs on our DAS cluster

More information

Register by email to ndrost@cs.vu.nl (Niels Drost) now

Starts after MPI lecture (mid October)

See http://www.cs.vu.nl/pp-cursus/ for a complete description