Artificial Intelligence & Data Science is coming soon

We're still writing and reviewing the notes for every subject, and we'll open it when every subject is ready. Meanwhile you can practise aptitude, ask Aria about any subject, or see the branches that are ready.

Artificial Intelligence & Data Science

High Performance Computing

Parallel computing and scaling laws, OpenMP and MPI, GPUs and CUDA, HPC for AI, cloud HPC.

Interview questions
Notes for High Performance Computing are being written and reviewed. Until they're up, Aria teaches every topic — ask for any part of the syllabus, with worked numericals.

Coming next in High Performance Computing

  1. Parallel and distributed computing· notes coming
  2. Flynn's taxonomy· notes coming
  3. Performance metrics and scaling· notes coming
  4. Amdahl's and Gustafson's laws· notes coming
  5. OpenMP· notes coming
  6. MPI· notes coming
  7. Hybrid programming· notes coming
  8. Synchronisation and race conditions· notes coming
  9. Hardware accelerators· notes coming
  10. GPU architecture· notes coming
  11. CUDA programming model· notes coming
  12. GPUs for deep learning· notes coming
  13. Data and model parallelism· notes coming
  14. Distributed deep learning· notes coming
  15. Data-intensive computing· notes coming
  16. Cloud HPC· notes coming
  17. Containers for HPC· notes coming
  18. Exascale and green HPC· notes coming
  19. Parallel algorithm models· notes coming
  20. Conditions of parallelism: data and resource dependences· notes coming
  21. Control-flow versus data-flow computers· notes coming
  22. Reservation tables and arithmetic pipelines· notes coming
  23. Vector processing, VLIW and array processors· notes coming
  24. The PRAM model and shared-memory algorithms· notes coming
  25. Sorting and routing on a 2D mesh· notes coming
  26. Data routing operations and greedy routing· notes coming
  27. Meshes, meshes of trees and low-diameter architectures· notes coming
  28. Hypercube algorithms: sorting, routing and broadcasting· notes coming
  29. Bitonic sorting networks· notes coming
  30. Butterfly, cube-connected cycles and shuffle-exchange networks· notes coming
  31. Emulation and scheduling among parallel architectures· notes coming
  32. Multithreading and latency hiding· notes coming
  33. Parallel I/O and fault tolerance· notes coming
  34. Parallel operating systems and file systems· notes coming
  35. Stored-program and cache-based microprocessor architecture· notes coming
  36. Moore's law, pipelining and superscalarity· notes coming
  37. Memory hierarchies, cache mapping and prefetch· notes coming
  38. Profiling: function-level, line-based and hardware counters· notes coming
  39. Common-sense serial code optimisations· notes coming
  40. Role of compilers in optimisation· notes coming
  41. Data access optimisation· notes coming
  42. Interconnection networks: buses, switched, fat-tree and mesh· notes coming
  43. MPI collective and non-blocking communication· notes coming
  44. MPI virtual topologies and the Jacobi solver· notes coming
  45. OpenMP loop scheduling and tasking· notes coming
  46. OpenMP performance pitfalls· notes coming