Introduction to the practical use of multi-processor workstations and supercomputing clusters. Developing and using parallel programs for solving computationally intensive problems. The course builds on basic concepts of programming and problem solving.
Introduction to parallel programming and high performance distributed computing (HPDC) (1)
Motivation for HPDC
Review of parallel programs and platforms
Implicit parallelism and limitations of instruction level parallelism (ILP)
Survey of architecture of commonly used HPDC platforms
Concurrency and parallelism (1)
Introduction to concurrency & Parallelism
Levels of parallelism
Instruction level parallelism
SIMD versus MIMD
Review of C programming language and the Linux environment (1.5)
Review of basic programming constructs
Applying Java/C++ syntax and semantics to C language
Introduction to problem solving using the C language
Introduction to Linux
C programming using Linux
C structures
Exploring instruction level parallelism (1)
Review of instruction level parallelism and sources of hazards
Concepts of hazard elimination via code restructuring (dependency reduction, loop unrolling)
Timing and statistical comparison of performance of c programs
Introduction to parallel programming (2)
Principles of parallel algorithms
Effects of synchronization and communication latencies
Overview of physical and logical communication topologies
Using MPE for parallel graphical visualization (parallel libraries)
Introduction to message passing paradigm (.5)
Principles of message-passing programming
The building blocks of message passing
Programming in MPI (3)
Introduction to MPI: The Message Passing Interface
MPI Fundamentals
Partitioning data versus partitioning control
Blocking communications and parallelism
MPI communication models
Blocking vs. non-blocking communication and impacts of parallelism
Developing MPI programs that exchange derived data types
Create MPI programs that use structure derived data types
Review of portability and interoperability issue
Performance profiling (1)
Using software tools for performance profiling
Performance profiling of MPI programs
Speedup anomalies in parallel algorithms
Collective communications (2)
Introduction to collective communications
Distributed debugging
Introduction to MPI scatter/gather operations
Exploring the complete collective communication operations in MPI
Scalability and performance (1)
Understanding notions of scalability and performance
Metrics of scalability and performance
Asymptotic analysis of scalability and performance
Exams/Reviews (1)
Learning Outcomes:
Identify various forms of parallelism, their application, advantages, and drawbacks
Effectively utilize instruction level parallelism
Effectively utilize multi-core CPUs and multithreading
Analyze and implement parallel and distributed programs using the Message Passing Interface (MPI) that solves a stated problem in a clean, robust, efficient, and scalable manner
Describe and empirically demonstrate concepts of parallel efficiency and scalability
Apply concepts to solve computationally demanding problems
Analyze experimental results and draw inferences to verify hypotheses
Graduate students:
Students taking the course for graduate credit will be expected to apply course concepts to solve computationally demanding problems, analyze experimental results, and draw inferences to verify hypotheses.
Department of Computer Science and Software Engineering
