Artificial Intelligence & Data Science is coming soon
We're still writing and reviewing the notes for every subject, and we'll open it when every subject is ready. Meanwhile you can practise aptitude, ask Aria about any subject, or see the branches that are ready.
Artificial Intelligence & Data Science
High Performance Computing
Parallel computing and scaling laws, OpenMP and MPI, GPUs and CUDA, HPC for AI, cloud HPC.
Notes for High Performance Computing are being written and reviewed. Until they're up, Aria teaches every topic — ask for any part of the syllabus, with worked numericals.
Coming next in High Performance Computing
- Parallel and distributed computing· notes coming
- Flynn's taxonomy· notes coming
- Performance metrics and scaling· notes coming
- Amdahl's and Gustafson's laws· notes coming
- OpenMP· notes coming
- MPI· notes coming
- Hybrid programming· notes coming
- Synchronisation and race conditions· notes coming
- Hardware accelerators· notes coming
- GPU architecture· notes coming
- CUDA programming model· notes coming
- GPUs for deep learning· notes coming
- Data and model parallelism· notes coming
- Distributed deep learning· notes coming
- Data-intensive computing· notes coming
- Cloud HPC· notes coming
- Containers for HPC· notes coming
- Exascale and green HPC· notes coming
- Parallel algorithm models· notes coming
- Conditions of parallelism: data and resource dependences· notes coming
- Control-flow versus data-flow computers· notes coming
- Reservation tables and arithmetic pipelines· notes coming
- Vector processing, VLIW and array processors· notes coming
- The PRAM model and shared-memory algorithms· notes coming
- Sorting and routing on a 2D mesh· notes coming
- Data routing operations and greedy routing· notes coming
- Meshes, meshes of trees and low-diameter architectures· notes coming
- Hypercube algorithms: sorting, routing and broadcasting· notes coming
- Bitonic sorting networks· notes coming
- Butterfly, cube-connected cycles and shuffle-exchange networks· notes coming
- Emulation and scheduling among parallel architectures· notes coming
- Multithreading and latency hiding· notes coming
- Parallel I/O and fault tolerance· notes coming
- Parallel operating systems and file systems· notes coming
- Stored-program and cache-based microprocessor architecture· notes coming
- Moore's law, pipelining and superscalarity· notes coming
- Memory hierarchies, cache mapping and prefetch· notes coming
- Profiling: function-level, line-based and hardware counters· notes coming
- Common-sense serial code optimisations· notes coming
- Role of compilers in optimisation· notes coming
- Data access optimisation· notes coming
- Interconnection networks: buses, switched, fat-tree and mesh· notes coming
- MPI collective and non-blocking communication· notes coming
- MPI virtual topologies and the Jacobi solver· notes coming
- OpenMP loop scheduling and tasking· notes coming
- OpenMP performance pitfalls· notes coming