SPPU TE IT syllabus

SPPU · Third Year Information Technology · 2024 Pattern

High Performance Computing

PEC-362B-IT · Elective III · Semester VI · 3 credits · CCE 30 + end-semester 70

One of the Elective III options.

All notes for this subject
  1. Unit 1: Parallel Computing and HPC Architectures

    9 hours

    Uses of HPC; Flynn's taxonomy; shared versus distributed memory; multicore, cache coherence (MESI) and NUMA; bus, crossbar, mesh, hypercube and fat-tree networks; speedup by Amdahl's and Gustafson's laws, efficiency, scalability, FLOPS and TOP500.

    Marks: 14 in the end-semester exam · CCE: Unit test (12 marks, Units 1 & 2)

    • Flynn's taxonomy
    • Shared and distributed memory systems
    • Cache coherence and NUMA
    • Interconnection networks
    • Speedup: Amdahl's and Gustafson's laws
  2. Unit 2: Shared-memory Programming with OpenMP

    9 hours

    Processes versus threads and Pthreads; the fork–join model and #pragma omp; parallel for, sections and single; schedule clauses; critical, atomic, barrier and ordered; shared, private, firstprivate and reduction; tasks and SIMD.

    Marks: 14 in the end-semester exam · CCE: Unit test (12 marks, Units 1 & 2)

    • OpenMP fork–join model and directives
    • Work-sharing constructs and scheduling
    • Synchronisation in OpenMP
    • Data scoping and reduction
    • OpenMP tasks
  3. Unit 3: Distributed-memory Programming with MPI

    9 hours

    Message passing and MPI program structure; MPI_Send and MPI_Recv, blocking and non-blocking; broadcast, scatter, gather, reduce and allreduce; communicators, groups, Cartesian topologies and derived datatypes.

    Marks: 14 in the end-semester exam · CCE: Assignments or case study (12 marks, Units 3 & 4)

    • MPI programming model
    • Point-to-point communication
    • Non-blocking communication
    • Collective communication
    • Communicators and virtual topologies
  4. Unit 4: GPU Computing with CUDA and OpenCL

    9 hours

    GPU architecture: SMs, CUDA cores, warps and the memory hierarchy; kernels, threads, blocks and grids; cudaMalloc, cudaMemcpy, unified memory, coalescing and shared memory; occupancy, warp divergence, bank conflicts and streams; OpenCL's models.

    Marks: 14 in the end-semester exam · CCE: Assignments or case study (12 marks, Units 3 & 4)

    • GPU architecture and memory hierarchy
    • CUDA kernels, threads, blocks and grids
    • CUDA memory management
    • CUDA performance optimisation
    • OpenCL
  5. Unit 5: HPC Applications

    9 hours

    Parallel BFS and DFS; parallel bubble and merge sort; distributed document classification; Kubernetes; GPU and AI/ML workloads.

    Marks: 14 in the end-semester exam · CCE: Seminar, open-book test or quiz (6 marks, Unit 5)

    • Parallel search: BFS and DFS
    • Parallel sorting
    • Parallel computing for AI and ML

Unit contents are summarised from the official SPPU syllabus . AiCanCode.org is not affiliated with Savitribai Phule Pune University.