SPPU TE AI&DS syllabus

SPPU · Third Year Artificial Intelligence and Data Science · 2024 Pattern

High Performance Computing

PEC-361C-AID · Elective II · Semester VI · 3 credits · CCE 30 + end-semester 70

One of the Elective II options.

All notes for this subject
  1. Unit 1: Fundamentals of HPC

    9 hours

    Sequential to parallel computing; parallel versus distributed, shared versus distributed memory; concurrency and parallelism, data and task parallelism; Flynn's taxonomy; speedup, efficiency, throughput and scaling; Amdahl's and Gustafson's laws.

    Marks: 14 in the end-semester exam · CCE: Unit test (12 marks, Units 1 & 2)

    • Parallel and distributed computing
    • Flynn's taxonomy
    • Performance metrics and scaling
    • Amdahl's and Gustafson's laws
  2. Unit 2: Parallel Programming Models

    9 hours

    Thinking parallel; OpenMP parallel for, reduction and tasks; MPI send, receive and broadcast; OpenMP versus MPI; hybrid programming; barriers and race conditions.

    Marks: 14 in the end-semester exam · CCE: Unit test (12 marks, Units 1 & 2)

    • OpenMP
    • MPI
    • Hybrid programming
    • Synchronisation and race conditions
  3. Unit 3: Accelerators and GPU Computing

    9 hours

    GPUs, FPGAs and TPUs; streaming multiprocessors, threads, blocks and grids; SIMT; GPU memory; CUDA kernels and host–device transfers; GPUs in deep learning; cuBLAS and cuDNN.

    Marks: 14 in the end-semester exam · CCE: Assignments or case study (12 marks, Units 3 & 4)

    • Hardware accelerators
    • GPU architecture
    • CUDA programming model
    • GPUs for deep learning
  4. Unit 4: HPC for AI and Data-intensive Work

    9 hours

    Why AI needs HPC; training versus inference; data and model parallelism; distributed deep learning; big data architectures, distributed file systems and data locality; MapReduce and Spark; communication overhead.

    Marks: 14 in the end-semester exam · CCE: Assignments or case study (12 marks, Units 3 & 4)

    • Data and model parallelism
    • Distributed deep learning
    • Data-intensive computing
  5. Unit 5: Cloud and Modern HPC

    9 hours

    Cloud HPC and virtual clusters; IaaS for HPC; Docker and Kubernetes; elastic scaling and spot instances; energy-efficient HPC; exascale computing.

    Marks: 14 in the end-semester exam · CCE: Seminar, open-book test or quiz (6 marks, Unit 5)

    • Cloud HPC
    • Containers for HPC
    • Exascale and green HPC

Unit contents are summarised from the official SPPU syllabus . AiCanCode.org is not affiliated with Savitribai Phule Pune University.