SPPU · Third Year Information Technology · 2024 Pattern
High Performance Computing
PEC-362B-IT · Elective III · Semester VI · 3 credits · CCE 30 + end-semester 70
One of the Elective III options.
Unit 1: Parallel Computing and HPC Architectures
9 hoursUses of HPC; Flynn's taxonomy; shared versus distributed memory; multicore, cache coherence (MESI) and NUMA; bus, crossbar, mesh, hypercube and fat-tree networks; speedup by Amdahl's and Gustafson's laws, efficiency, scalability, FLOPS and TOP500.
Marks: 14 in the end-semester exam · CCE: Unit test (12 marks, Units 1 & 2)
- Flynn's taxonomy
- Shared and distributed memory systems
- Cache coherence and NUMA
- Interconnection networks
- Speedup: Amdahl's and Gustafson's laws
Unit 2: Shared-memory Programming with OpenMP
9 hoursProcesses versus threads and Pthreads; the fork–join model and #pragma omp; parallel for, sections and single; schedule clauses; critical, atomic, barrier and ordered; shared, private, firstprivate and reduction; tasks and SIMD.
Marks: 14 in the end-semester exam · CCE: Unit test (12 marks, Units 1 & 2)
- OpenMP fork–join model and directives
- Work-sharing constructs and scheduling
- Synchronisation in OpenMP
- Data scoping and reduction
- OpenMP tasks
Unit 3: Distributed-memory Programming with MPI
9 hoursMessage passing and MPI program structure; MPI_Send and MPI_Recv, blocking and non-blocking; broadcast, scatter, gather, reduce and allreduce; communicators, groups, Cartesian topologies and derived datatypes.
Marks: 14 in the end-semester exam · CCE: Assignments or case study (12 marks, Units 3 & 4)
- MPI programming model
- Point-to-point communication
- Non-blocking communication
- Collective communication
- Communicators and virtual topologies
Unit 4: GPU Computing with CUDA and OpenCL
9 hoursGPU architecture: SMs, CUDA cores, warps and the memory hierarchy; kernels, threads, blocks and grids; cudaMalloc, cudaMemcpy, unified memory, coalescing and shared memory; occupancy, warp divergence, bank conflicts and streams; OpenCL's models.
Marks: 14 in the end-semester exam · CCE: Assignments or case study (12 marks, Units 3 & 4)
- GPU architecture and memory hierarchy
- CUDA kernels, threads, blocks and grids
- CUDA memory management
- CUDA performance optimisation
- OpenCL
Unit 5: HPC Applications
9 hoursParallel BFS and DFS; parallel bubble and merge sort; distributed document classification; Kubernetes; GPU and AI/ML workloads.
Marks: 14 in the end-semester exam · CCE: Seminar, open-book test or quiz (6 marks, Unit 5)
- Parallel search: BFS and DFS
- Parallel sorting
- Parallel computing for AI and ML
Unit contents are summarised from the official SPPU syllabus . AiCanCode.org is not affiliated with Savitribai Phule Pune University.