SPPU · Third Year Artificial Intelligence and Data Science · 2024 Pattern
High Performance Computing
PEC-361C-AID · Elective II · Semester VI · 3 credits · CCE 30 + end-semester 70
One of the Elective II options.
Unit 1: Fundamentals of HPC
9 hoursSequential to parallel computing; parallel versus distributed, shared versus distributed memory; concurrency and parallelism, data and task parallelism; Flynn's taxonomy; speedup, efficiency, throughput and scaling; Amdahl's and Gustafson's laws.
Marks: 14 in the end-semester exam · CCE: Unit test (12 marks, Units 1 & 2)
- Parallel and distributed computing
- Flynn's taxonomy
- Performance metrics and scaling
- Amdahl's and Gustafson's laws
Unit 2: Parallel Programming Models
9 hoursThinking parallel; OpenMP parallel for, reduction and tasks; MPI send, receive and broadcast; OpenMP versus MPI; hybrid programming; barriers and race conditions.
Marks: 14 in the end-semester exam · CCE: Unit test (12 marks, Units 1 & 2)
- OpenMP
- MPI
- Hybrid programming
- Synchronisation and race conditions
Unit 3: Accelerators and GPU Computing
9 hoursGPUs, FPGAs and TPUs; streaming multiprocessors, threads, blocks and grids; SIMT; GPU memory; CUDA kernels and host–device transfers; GPUs in deep learning; cuBLAS and cuDNN.
Marks: 14 in the end-semester exam · CCE: Assignments or case study (12 marks, Units 3 & 4)
- Hardware accelerators
- GPU architecture
- CUDA programming model
- GPUs for deep learning
Unit 4: HPC for AI and Data-intensive Work
9 hoursWhy AI needs HPC; training versus inference; data and model parallelism; distributed deep learning; big data architectures, distributed file systems and data locality; MapReduce and Spark; communication overhead.
Marks: 14 in the end-semester exam · CCE: Assignments or case study (12 marks, Units 3 & 4)
- Data and model parallelism
- Distributed deep learning
- Data-intensive computing
Unit 5: Cloud and Modern HPC
9 hoursCloud HPC and virtual clusters; IaaS for HPC; Docker and Kubernetes; elastic scaling and spot instances; energy-efficient HPC; exascale computing.
Marks: 14 in the end-semester exam · CCE: Seminar, open-book test or quiz (6 marks, Unit 5)
- Cloud HPC
- Containers for HPC
- Exascale and green HPC
Unit contents are summarised from the official SPPU syllabus . AiCanCode.org is not affiliated with Savitribai Phule Pune University.