Computer Engineering is coming soon

We're still writing and reviewing the notes for every subject, and we'll open it when every subject is ready. Meanwhile you can practise aptitude, ask Aria about any subject, or see the branches that are ready.

SMP, UMA and NUMA

SMP, UMA and NUMA are memory-architecture designs for multiprocessor systems that decide whether all processors see equal memory access cost (UMA) or unequal costs (NUMA); understanding them is essential for designing and optimising parallel programs and hardware.

Drafted with Aria, reviewed by the AiCanCode.org team · Updated . How we write and check our notes. Spotted an error? Use Give Feedback at the bottom of the page.

Why it matters

Multiprocessor systems are common in servers, desktops and embedded devices. An engineer uses knowledge of SMP, UMA and NUMA when selecting or designing system architecture, writing parallel programs, and tuning OS or runtime so that programs get best latency and bandwidth to memory.

Key ideas

Start with simple definitions (plain language):

  • Processor (core): a hardware unit that executes instructions. Measured by clock rate (Hz) but here treated as a unit that issues memory requests.
  • Latency: the time between issuing a memory request and the first byte being available (SI unit: seconds). Typical sub-claim numbers are not used here—verify local numbers for hardware.
  • Bandwidth: the sustained data rate a memory system can deliver (SI unit: bytes per second).

Now the architectures:

  • Symmetric Multiprocessing (SMP): a system where two or more processors share a single coherent memory address space and are peers (no single master processor). All processors can run the OS and user code. Assumption: there is a single logical shared memory that obeys cache coherence.

  • Uniform Memory Access (UMA): a memory organisation where all processors have equal access time to any memory location. Exact statement: for any processor Pi and any memory location Mj, the memory access latency Latency(Pi,Mj) is constant and independent of i and j. Assumptions: shared bus or interconnect that provides equal-distance access, and caches are kept coherent.

  • Non-Uniform Memory Access (NUMA): a memory organisation where memory is physically distributed, each processor (or socket) has local memory and can also access remote memory; access times differ. Exact statement: for a given processor Pi, there exist two memory locations M_local in its local node and M_remote in a remote node such that Latency(Pi, M_local) < Latency(Pi, M_remote); typically local accesses are faster than remote ones. Assumptions: processors are grouped in nodes/sockets, each node has local DRAM; nodes are connected by an interconnect (e.g., point-to-point links or switches).

Important related terms:

  • Local memory: DRAM attached to the same node/socket as the processor.
  • Remote memory: DRAM attached to a different node/socket.
  • Cache coherence: a protocol that ensures multiple caches in different processors do not have conflicting views of the same memory location. In SMP systems with caches, coherence is typically implemented; UMA describes uniform access and may or may not involve caches. Many NUMA systems are cache-coherent (ccNUMA), but some are non-coherent; remote coherence traffic adds overhead.
  • Memory consistency model: defines the order in which memory operations appear to execute across processors (e.g., sequential consistency). This affects programming and correctness.
  • ccNUMA: cache-coherent NUMA, where a single coherent shared address space spans all nodes; non-coherent NUMA lacks hardware coherence and requires explicit data movement or message passing.
  • Directory-based coherence: a scalable coherence mechanism common in NUMA systems, where a directory tracks sharers/owners to route invalidations; contrasts with bus snooping used in small UMA systems.

Diagrams to draw (exam):

  • UMA diagram: draw N processors (P0..PN-1) at top, a single shared memory block below, and a common interconnect or bus between them. Label: "single shared memory" and annotate latency as equal for all processors. Show caches on each processor.
  • NUMA diagram: draw nodes (Node0..NodeK-1). Each node contains a processor block and a local memory block connected by a local interconnect. Connect nodes with a network fabric (label: interconnect). Annotate local latency (short) and remote latency (longer).

Derivations universities commonly ask for:

  1. Average memory access time in a NUMA system when fraction p of accesses are local. Start from definition: average = weighted sum.
    • Let T_local be local access latency, T_remote be remote access latency, p_local fraction = p, p_remote = 1 - p. Then T_avg = p*T_local + (1-p)*T_remote. This is an exact linear expectation and assumes independent accesses.
  2. Bandwidth per processor in a shared-bandwidth UMA: if total memory bandwidth is B_shared and processors are symmetric and fully utilise bandwidth, then per-processor bandwidth = B_shared / N. This assumes perfect-sharing fairness and no other bottlenecks.
  3. Show that NUMA can improve scalability because each node adds local bandwidth. If B_local is per-node local bandwidth and nodes do not share the same bottleneck, aggregate bandwidth ≈ N_nodes * B_local for local-dominated workloads. Assumption: minimal remote access and low interconnect contention.

Trade-offs (exact):

  • UMA advantages: simple shared address space, easier programming model, predictable equal latency. Note: SMP (symmetric multiprocessor) refers to processor/OS symmetry and can be built on UMA or NUMA; do not conflate SMP with UMA. Assumptions: small to moderate N, coherence and bus/arbitration scale.
  • NUMA advantages: scales to many sockets because memory bandwidth/DRAM banks grow with nodes; better aggregate bandwidth and capacity. Cost: programming must be NUMA-aware to place memory, coherence/remote access overheads.

Practical engineering rule: for small numbers of processors (e.g., within one socket), UMA within a socket is simplest; for large-scale multi-socket systems, NUMA is common. SMP refers to processor symmetry and applies to both UMA and NUMA systems. Verify platform-specific latency and bandwidth numbers when tuning.

Formulas

T_avg = p_local·T_local + (1 - p_local)·T_remote where T_avg is average memory access latency (seconds), p_local is fraction of accesses that go to local memory (dimensionless), T_local is local access latency (s), T_remote is remote access latency (s). Applies when accesses are independent and fractions are stable. Effective_AMAT_with_cache = h·T_cache + (1 - h)·(p_local·T_local + (1 - p_local)·T_remote) where h is cache hit ratio (dimensionless) and T_cache is cache hit latency (s). Applies for a single dominant cache level; extends the NUMA average to include cache hits.

Bandwidth_per_processor_UMA = B_shared / N where B_shared is total memory bandwidth (bytes/s), N is number of processors. Applies when all processors contend equally for a single memory resource.

Aggregate_local_bandwidth_NUMA ≈ N_nodes · B_local where N_nodes is number of NUMA nodes, B_local is local bandwidth per node (bytes/s). Applies when most traffic is local and interconnect is not the bottleneck.

Speedup_due_to_locality ≈ T_UMA / T_NUMA where T_UMA is average access time when all accesses see UMA latency, T_NUMA is average when p_local portion are local; use the T_avg formula to compute T_NUMA. This is a practical comparison metric.

Worked examples

Example 1 — Average access latency in NUMA (numerical): Given: T_local = 80 ns, T_remote = 200 ns, fraction local p_local = 0.75. Compute T_avg.

  1. Formula: T_avg = p_local·T_local + (1 - p_local)·T_remote.
  2. Numbers: T_avg = 0.75·80 ns + 0.25·200 ns.
  3. Calculation: = 60 ns + 50 ns = 110 ns.
  4. Final: T_avg = 110 ns. (Units: nanoseconds. This assumes steady-state proportions and does not include cache hit effects.)

Example 2 — Bandwidth per processor: UMA vs NUMA (numerical comparison): Given: UMA system with shared memory bandwidth B_shared = 50 GB/s and N = 10 processors. NUMA system: 5 nodes, each node has local bandwidth B_local = 20 GB/s; assume one processor per node for simplicity.

  1. UMA per-processor bandwidth = B_shared / N = 50 GB/s / 10 = 5 GB/s.
  2. NUMA aggregate local bandwidth = N_nodes · B_local = 5 · 20 GB/s = 100 GB/s. If processors keep traffic local, per-processor effective bandwidth ≈ 20 GB/s.
  3. Conclusion: UMA: 5 GB/s per processor; NUMA (local-dominated): 20 GB/s per processor. But remote accesses or interconnect bottleneck reduce NUMA advantage.

Example 3 — Small model of speedup from locality (theory/short answer): If a program on UMA sees memory latency T_UMA = 120 ns for every access, while on NUMA with p_local=0.8, T_local=80 ns, T_remote=200 ns, then T_NUMA = 0.8·80 + 0.2·200 = 64 + 40 = 104 ns. Speedup in memory-limited work ~ T_UMA / T_NUMA = 120/104 ≈ 1.154 (≈15.4% faster). Show steps and assumptions (memory-bound work, other latencies ignored).

Common mistakes

  • Forgetting cache coherence cost: students forget that coherence messages travel over the interconnect in NUMA and add overhead.
  • Using per-processor bandwidth = B_shared / N without checking contention, bus arbitration, or access patterns.
  • Drawing diagrams that omit caches or do not label local vs remote latency; exam diagrams must show caches, memory blocks, and interconnect with labels.

For your exam

What question types appear and what to include:

  • Definitions: clearly define SMP, UMA, NUMA, local/remote memory, latency and bandwidth; give the exact statement of UMA (equal access time) and NUMA (unequal access times).
  • Short notes: compare advantages/disadvantages and state assumptions (e.g., coherence present, number of processors small or large).
  • Derivations: derive the average access time formula for NUMA and show how per-processor bandwidth changes with N for shared vs distributed memory.
  • Numericals: compute T_avg, per-processor bandwidth, or percentage speedup using given numbers; carry units and show steps.
  • Diagrams: draw and label UMA and NUMA architectures including processors, caches, local memory, and interconnect. Annotate local and remote latencies.
  • Programming/OS questions: explain NUMA-aware allocation policies (bind memory to node, first-touch policy) and how they affect p_local. Also discuss CPU affinity (pinning threads to nodes) to keep computation near its memory.

A full-marks answer includes: precise definitions, assumptions stated, stepwise derivations with formulas, labelled diagrams, units in numericals, and an explanation of how coherence and consistency affect performance.

Quick check

  1. Define UMA in one sentence.
  2. Write the formula for average access time in NUMA when p of accesses are local.
  3. In an UMA system with 8 processors and shared bandwidth 64 GB/s, what is the fair per-processor bandwidth in GB/s (show the calculation)? Answers: 1) UMA: a memory organisation where every processor has equal access time to any memory location. 2) T_avg = p·T_local + (1 - p)·T_remote. 3) 64 GB/s ÷ 8 = 8 GB/s.

Finished this topic? Mark it so your progress, study plan and readiness keep up.

Stuck on something here?