Parallel Programming

Henri Bal(bal@cs.vu.nl)

Vrije UniversiteitFaculty of Sciences

Amsterdam

Overview

• What is this large display??• What is parallel programming?• Why do we need parallel programming?• Organization of this course• Parallel Programming & Visualization• Practicum Parallel Programming

ICWall•A large display suitable for:–Education (classroom)–Modern scientific applications thatneed to visualize large data-sets

–3D applications

ICWall: an "off-the-shelf" video wall

•High-end graphics is very expensive•Use tiled display with many projectors•Use standard (inexpensive) components–PCs, graphic and network cards–'PowerPoint' video projectors (~ 1 kg)

•Similar idea as cluster computing, but applied to graphics

Setup•4x2x2 projectors

–1024x768 pixels (4096x1536 total)

•Screen–Back-projection, ~5.5m x 2m

•PC cluster–8 nodes + 1 server –Myrinet network–Dual AMD Athlons–Nvidia GeForce 3

•Problems–Mechanical alignment ofprojectors is difficult

–Projectors have differentcolors, brightness, etc.

Solution–Project test image–Observe image with a camera–Analyse difference between original and observed image–Compensate for differences in software

Making a single image

Parallel Programming

• Sequential programmingSingle thread of control

• Parallel programmingMultiple threads of control

• Why parallel programming?

Parallel Programming

• Sequential programmingSingle thread of control

• Parallel programmingMultiple threads of control

• Why parallel programming?Eases programming? Not really.

Parallel Programming

• Sequential programmingSingle thread of control

• Parallel programmingMultiple threads of control

• Why parallel programming?Eases programming? Not really.Performance? Yes!

Parallel Processing

• Many applications need much faster machines

• Sequential machines are reaching their speed limits

• Use multiple processors to solve large problems fast

• Microprocessors are getting cheaper and cheaper

Grand Challenges

• Applications with high impact on industry or society

• Require extraordinary performance:1 Teraflop = 1.000.000.000.000 operations/sec

• Several Teraflop parallel machines existSee http://www.top500.org

Challenging Applications

• Analyzing video images• Aircraft modeling• Ozone layer modeling• Climate modeling• Ocean circulation• Quantum chemistry• General: computational science• Computer chess• Protein folding

About this Course

Goal: Study how to write programs that run in parallel on a large number of machines.Focus on programming methods, languages, applications

Prerequisites:Some knowledge about sequential languagesLittle knowledge about networking and operating systems

Aspects of Parallel Computing

Algorithms and applications

Programming methods, languages, and environments

Parallel machines and architectures

Course Outline• Introduction in algorithms and applications• Parallel machines and architectures

Overview of parallel machinesCluster computers (Myrinet)

• Programming methods, languages, and environmentsMessage passing (SR, MPI, Java)Higher-level languages: Linda, Orca, HPF

• ApplicationsN-body problems, search algorithms

• Grid computing

Course Information

ExaminationWritten exam based on:- Reader- Handouts- Lectures

More information (slides, recommended books):http://www.cs.vu.nl/~bal/college04.html

Related Courses

Parallel programming practicum

Computer graphics

Scientific visualization (together with Physics)

Practicum Parallel Programming

Separate practicum (6 ECTS)

• Implement ASP + SOR algorithms in C/MPI

• Implement IDA* search algorithm in Java/RMI

• Test and measure the programs on our DAS-2 cluster

More information

Register by email to rob@cs.vu.nl (Rob van Nieuwpoort) now

Starts after MPI lecture (25 October)

See http://www.cs.vu.nl/pp-cursus/ for a complete description