Bo Hong Electrical and Computer Engineering Georgia Institute of Technology
description
Transcript of Bo Hong Electrical and Computer Engineering Georgia Institute of Technology
![Page 1: Bo Hong Electrical and Computer Engineering Georgia Institute of Technology](https://reader035.fdocuments.in/reader035/viewer/2022062411/568166a8550346895dda9bef/html5/thumbnails/1.jpg)
Early AdopterIntroduction to Parallel Computing:
Research Intensive University: 4th Year Elective
Bo HongElectrical and Computer Engineering
Georgia Institute of Technology
![Page 2: Bo Hong Electrical and Computer Engineering Georgia Institute of Technology](https://reader035.fdocuments.in/reader035/viewer/2022062411/568166a8550346895dda9bef/html5/thumbnails/2.jpg)
2 NSF/IEEE-TCPP PDC Curriculum
Syllabus – Overview (1)
• Format– A 4th year elective course on parallel computing.
• Target Audience– students who have background in computer
organization and programming, and would like to learn parallel computing.
![Page 3: Bo Hong Electrical and Computer Engineering Georgia Institute of Technology](https://reader035.fdocuments.in/reader035/viewer/2022062411/568166a8550346895dda9bef/html5/thumbnails/3.jpg)
3 NSF/IEEE-TCPP PDC Curriculum
Syllabus – Overview (2)
• Topical Coverage:– Parallel computing systems– Parallel programming– Parallel algorithms
• Expected Outcomes:– Upon completion of the course, the students are
expected to (1) understand parallel computing systems, and (2) be able to design algorithms and implement programs for such systems.
![Page 4: Bo Hong Electrical and Computer Engineering Georgia Institute of Technology](https://reader035.fdocuments.in/reader035/viewer/2022062411/568166a8550346895dda9bef/html5/thumbnails/4.jpg)
4 NSF/IEEE-TCPP PDC Curriculum
Syllabus – Topical Coverage (1)
Topics BLOOM# Hours Source
Parallel ArchitecturesStacks and thread execution K 1 Addition to the proposed curriculum
Multi-threading on shared memory computers
C 1 Addition to the proposed curriculum
Memory Hierarchy and cache coherency
K 1 NSF-TCPP Curriculum Architecture (C)
Snoopy based cache coherency
C 2 NSF-TCPP Curriculum Architecture (N)
Directory based cache coherency
A 2 NSF-TCPP Curriculum Architecture (N)
Interconnect, topologies and routing
A 1 NSF-TCPP Curriculum Architecture (C/K)
Massive multi-threading and GPU
K 2 Addition to the proposed curriculum
![Page 5: Bo Hong Electrical and Computer Engineering Georgia Institute of Technology](https://reader035.fdocuments.in/reader035/viewer/2022062411/568166a8550346895dda9bef/html5/thumbnails/5.jpg)
5 NSF/IEEE-TCPP PDC Curriculum
Syllabus – Topical Coverage (2)
Topics BLOOM# Hours SourceParallel Programming
SIMD K 1 NSF-TCPP Curriculum Programming (K)
Shared Memory K 2 NSF-TCPP Curriculum Programming (A)
Message Passing K 2 NSF-TCPP Curriculum Programming (C)
Synchronization A 1 NSF-TCPP Curriculum Programming (A)
Pthread Library K 3 NSF-TCPP Curriculum Programming (C)
MPI K 3 Briefly mentioned in the proposed curriculum
CUDA K 3 Briefly mentioned in the proposed curriculum
![Page 6: Bo Hong Electrical and Computer Engineering Georgia Institute of Technology](https://reader035.fdocuments.in/reader035/viewer/2022062411/568166a8550346895dda9bef/html5/thumbnails/6.jpg)
6 NSF/IEEE-TCPP PDC Curriculum
Syllabus – Topical Coverage (3)
Topics BLOOM# Hours SourceParallel Algorithms
Asymptotics K 1 NSF-TCPP Curriculum Algorithm (C)Speedup, efficiency, cost, and scalability
K 2 NSF-TCPP Curriculum Algorithm (C/K/N)
Broadcast, reduction, scatter and gather
C 3 NSF-TCPP Curriculum Algorithm (C/A)
Matrix multiplication K 3 NSF-TCPP Curriculum Algorithm (optional)
Solving linear systems K 2 NSF-TCPP Curriculum Algorithm (optional)
Sorting (merge, bitonic, quick) K/C 3 NSF-TCPP Curriculum Algorithm (C)
Graphs (minimum spanning tree)
C 2 Addition to the proposed curriculum
FFT K/C 3 Addition to the proposed curriculum
![Page 7: Bo Hong Electrical and Computer Engineering Georgia Institute of Technology](https://reader035.fdocuments.in/reader035/viewer/2022062411/568166a8550346895dda9bef/html5/thumbnails/7.jpg)
7 NSF/IEEE-TCPP PDC Curriculum
Teaching Experience: Melding with Old Topics• Students knew uni-thread execution mechanisms
before coming to the course.• The lectures analyzed the role of stacks and address
spaces for uni-threads, and then extended the analysis to multiple threads.
• Students found it easy to understand the execution of multi-threaded programs, which further extends to other parallel computing models.
![Page 8: Bo Hong Electrical and Computer Engineering Georgia Institute of Technology](https://reader035.fdocuments.in/reader035/viewer/2022062411/568166a8550346895dda9bef/html5/thumbnails/8.jpg)
8 NSF/IEEE-TCPP PDC Curriculum
Evaluation (1)
• Pedagogy1. Gradual transition from sequential experience to
parallel thinking2. Explanation of abstract concepts via concrete
examples (e.g. cache coherency)• Student feedbacks
– Item 1 is important for accepting parallel thinking– Item 2 is helpful for understanding of the materials
![Page 9: Bo Hong Electrical and Computer Engineering Georgia Institute of Technology](https://reader035.fdocuments.in/reader035/viewer/2022062411/568166a8550346895dda9bef/html5/thumbnails/9.jpg)
9 NSF/IEEE-TCPP PDC Curriculum
Evaluation (2)
• Learning outcome– Evaluated through programming projects
(pthread/MPI/CUDA intro, finding prime numbers, solving linear systems, sorting) and a final survey on current topics
• With the course, the students– can approach a problem with parallel thinking– understand the mechanisms of different parallel
architectures– can design and analysis simple parallel algorithms– can implement and evaluate parallel algorithms using
pthread, MPI, and CUDA
![Page 10: Bo Hong Electrical and Computer Engineering Georgia Institute of Technology](https://reader035.fdocuments.in/reader035/viewer/2022062411/568166a8550346895dda9bef/html5/thumbnails/10.jpg)
10 NSF/IEEE-TCPP PDC Curriculum
Evaluation (3)
• Appropriateness of topical overages– Students find that
• The topics are comprehensive, and it is appreciated• Some topics are complicated (cache coherency, CUDA
programming, synchronization, etc)– Students suggest that
• Add some brief discussions on the current state-of-the-art of parallel computing
• Include more hands-on programming projects to help understand the concepts
![Page 11: Bo Hong Electrical and Computer Engineering Georgia Institute of Technology](https://reader035.fdocuments.in/reader035/viewer/2022062411/568166a8550346895dda9bef/html5/thumbnails/11.jpg)
11 NSF/IEEE-TCPP PDC Curriculum
Deviation from the Proposed Curriculum and the Impact• Additions to the curriculum
– Thread execution mechanisms• facilitates the sequential-to-parallel transition
– Massive multi-threading and GPU• helps cover the emerging field of GPU computing
– Minimum spanning tree and FFT• discusses two useful applications of parallel computing
• Expected level of understanding of the topics– Slight adjusted to fit the comprehensive introductory
course
![Page 12: Bo Hong Electrical and Computer Engineering Georgia Institute of Technology](https://reader035.fdocuments.in/reader035/viewer/2022062411/568166a8550346895dda9bef/html5/thumbnails/12.jpg)
12 NSF/IEEE-TCPP PDC Curriculum
Summary
• With the introductory course, the students achieved the expected learning outcome.
• The comprehensiveness and level of difficulty of the course are appreciated by the students.