Information Technology is coming soon

We're still writing and reviewing the notes for every subject, and we'll open it when every subject is ready. Meanwhile you can practise aptitude, ask Aria about any subject, or see the branches that are ready.

Information Technology

High Performance Computing

Parallel architectures and speedup, OpenMP, MPI, CUDA and OpenCL, parallel algorithms.

Interview questions
Notes for High Performance Computing are being written and reviewed. Until they're up, Aria teaches every topic — ask for any part of the syllabus, with worked numericals.

Coming next in High Performance Computing

  1. Flynn's taxonomy· notes coming
  2. Shared and distributed memory systems· notes coming
  3. Cache coherence and NUMA· notes coming
  4. Interconnection networks· notes coming
  5. Speedup: Amdahl's and Gustafson's laws· notes coming
  6. OpenMP fork–join model and directives· notes coming
  7. Work-sharing constructs and scheduling· notes coming
  8. Synchronisation in OpenMP· notes coming
  9. Data scoping and reduction· notes coming
  10. OpenMP tasks· notes coming
  11. MPI programming model· notes coming
  12. Point-to-point communication· notes coming
  13. Non-blocking communication· notes coming
  14. Collective communication· notes coming
  15. Communicators and virtual topologies· notes coming
  16. GPU architecture and memory hierarchy· notes coming
  17. CUDA kernels, threads, blocks and grids· notes coming
  18. CUDA memory management· notes coming
  19. CUDA performance optimisation· notes coming
  20. OpenCL· notes coming
  21. Parallel search: BFS and DFS· notes coming
  22. Parallel sorting· notes coming
  23. Parallel computing for AI and ML· notes coming
  24. Process-processor mapping· notes coming
  25. Decomposition techniques· notes coming
  26. Mapping for load balancing· notes coming
  27. Parallel algorithm models· notes coming
  28. OpenACC· notes coming
  29. Sources of overhead in parallel programs· notes coming
  30. Scalability and cost-optimality· notes coming
  31. Parallel matrix multiplication and Cannon's algorithm· notes coming
  32. Parallel all-pairs shortest paths· notes coming
  33. Levels of parallelism· notes coming
  34. Parallel computing models: SIMD, MIMD, SIMT and SPMD· notes coming
  35. Physical organisation of parallel platforms· notes coming
  36. Communication costs in parallel machines· notes coming
  37. Broadcast and reduction operations· notes coming
  38. Parallel matrix-vector multiplication· notes coming
  39. Parallel graph algorithms and sample sort· notes coming
  40. OpenCL device architectures· notes coming
  41. Conditions of parallelism: data and resource dependences· notes coming
  42. Control-flow versus data-flow computers· notes coming
  43. Linear pipelines and reservation tables· notes coming
  44. Vector processing, VLIW and array processors· notes coming
  45. PRAM model and basic parallel algorithms· notes coming
  46. Parallel prefix computation· notes coming
  47. Data routing operations and greedy routing· notes coming
  48. Mesh and hypercube algorithms· notes coming
  49. Butterfly, shuffle-exchange and cube-connected cycles networks· notes coming
  50. Multithreading and latency hiding· notes coming
  51. Parallel I/O and parallel file systems· notes coming
  52. Mapping threads to multidimensional data· notes coming
  53. Thread scheduling and latency tolerance· notes coming
  54. Warps, memory bandwidth and resource partitioning· notes coming
  55. GPU acceleration for neural networks· notes coming