CIS4930/CDA5125 Parallel and Distributed Systems Florida State University

CIS4930/CDA5125: Parallel and Distributed Systems

Instructor: Xin Yuan, 168 Love, xyuan@cs.fsu.edu, 4-9133• Office hour: 10:30am-11:20am M, W, F, or by appointment

Course website: http://www.cs.fsu.edu/~xyuan/cda5125/index.html

Course objectives

• Systematically introduce parallel and distributed systems (PDS) and programming

• Expose contemporary PDS technologies Processors, networking, system software, and programming

paradigms

• Study the trends of technology advances in PDS. • At the end of the course, the students are expected to

Explain general concepts and important issues in parallel architectures, programming, and algorithms.

Understand the challenges in parallel and distributed computing Program with different programming paradigms for different

architectures.

CIS4930/CDA5125 Parallel and Distributed Systems Florida State University

Topics covered

• Architecture– Architecture classification– SIMD, MIMD, shared memory (SMP, multicore, SMT, heterogeneous multi-core,

many-core), distributed memory (cluster, massively parallel, grid and cloud).• Programming

– Vector programming (SSE), Thread, OpenMP, MPI, CUDA, and OpenCL• Algorithm

– PRAM, BSP• Advanced

– Power and others

CIS4930/CDA5125 Parallel and Distributed Systems Florida State University

Textbok

• Required: none

• Reference books:– D. E. Culler and J. P. Singh, "Parallel Computer Architecture, A

Hardware/Software Approach," Morgan Kaufmann, 1999.

– Maurice Herlihy and Nir Shavit, "The Art of Multiprocessor Programming," Morgan Kaufmann, 2008.

• A significant portion of the course materials come from online resources.

– MPI: http://www-unix.mcs.anl.gov/mpi/ – OpenMP: http://www.openmp.org – CUDA: NVIDIA CUDA Programming Guide – OpenCL: OpenCL Programming Guide for the CUDA Architecture

CIS4930/CDA5125 Parallel and Distributed Systems Florida State University

Course schedule (tentative)• Week 1: Introduction and architectures classification

• Week 2: Performance

• Week 3: SIMD/Vector architectures, SSE and SIMD intrinsics

• Week 4: Shared memory architectures

• Week 5: Shared memory programming, thread

• Week 6: Shared memory programming, OpenMP

• Week 7: MIMD, distributed memory architectures, MPI

• Week 8: communication hardware and Interconnection networks

• Week 9: Stream architectures, CUDA/OpenCL

• Week 10: Spring break

• Week 11: CUDA/OpenCL

• Week 12: Performance monitoring and Optimization

• Week 13: PRAM and BSP

• Week 14: Advanced topics

• Week 15: Project presentations

• Week 16: Final exam

CIS4930/CDA5125 Parallel and Distributed Systems Florida State University

Prerequisite

• COP 4610: Operating systems– No requirement for background knowledge of

parallel and distributed computing– The class is designed to minimized the

dependence on COP4610.

CIS4930/CDA5125 Parallel and Distributed Systems Florida State University

Grading

• 25 % - homework, programming assignment, class participation.– Homework includes paper critics– Programming assignments can be done by groups of two.

• 25% - course project– Research or development– Groups of two– End product: report, software, and presentation.

• 20% - Midterm• 30% - Final

• Final target letter grade: half A’s and half B’s.

CIS4930/CDA5125 Parallel and Distributed Systems Florida State University

Course project• Development projects

– Parallelizing a fairly significant application (preferably some well known efficient sequential code).

• Evaluation projects– E.g. Evaluating SSE performance on different processors, evaluate

different synchronization algorithms on one or more processors

• Research projects– Survey an emerging area (e.g. techniques to improve power efficiency in

multi-core machines, emerging programming models for multi-core systems).

– Develop new techniques in any area of PDS (e.g. new communication algorithms, new synchronization algorithms, new scheduling schemes, etc)

CIS4930/CDA5125 Parallel and Distributed Systems Florida State University

Course policies

• Attendance: required.• Late assignments: not accepted without a valid

excuse. • Missed exam: following the university rules.

– Let me know when you need to miss an exam ASAP.• Incomplete grade:

– Miss the final with an accepted excuse– Due to extraordinary circumstances with appropriate

documentation.

CIS4930/CDA5125 Parallel and Distributed Systems Florida State University

• Academic Integrity– No copying from anywhere– Don’t ask others for solutions and don’t give

solutions to others.

• Violation– The university requires all violations to be

reported.– First violation with level 1 agreement:

• 0 for the particular assignment/exam and the lowering of one letter (A->B) for course final grade.

– Second violation: resolved through the office of the Dean and the Faculties

CIS4930/CDA5125 Parallel and Distributed Systems Florida State University

• Accommodation for Disabilities

1.Register with and provide documentation to the Student Disability Resource Center (SDRC).

2.Bring a letter to me from the SDRC `indicating you need academic accomodations. This should be done within the first week of class

CIS4930/CDA5125 Parallel and Distributed Systems Florida State University