Computer Engineering is coming soon

We're still writing and reviewing the notes for every subject, and we'll open it when every subject is ready. Meanwhile you can practise aptitude, ask Aria about any subject, or see the branches that are ready.

Designing for performance

Designing for performance explains how to measure processor and system speed, the math behind optimizations (CPI, clock, Amdahl's law, AMAT), and practical ways to evaluate trade-offs when improving parts of a computer system.

Drafted with Aria, reviewed by the AiCanCode.org team · Updated . How we write and check our notes. Spotted an error? Use Give Feedback at the bottom of the page.

Why it matters

Designing for performance helps an engineer decide where to invest effort (hardware or software) to make a computer system faster and more cost‑effective. It gives measurable metrics and formulas so you can predict the overall benefit of improving one part (CPU, pipeline, cache, etc.) before building it.

Key ideas

  • Throughput — the number of tasks completed per unit time (e.g., jobs/s). For CPUs it can be instructions per second.

  • Clock rate and cycle time — clock rate is the number of clock ticks per second (Hz); cycle time is the duration of one clock tick (s). They are inverses: clock rate = 1 / cycle time.

  • Instruction count (IC) — total number of instructions executed for a program; dimensionless.

  • Cycles per instruction (CPI) — average number of clock cycles each instruction takes. If there are instruction classes, average CPI = sum (CPI_i × fraction_i).

  • Instructions per cycle (IPC) — number of instructions retired per clock cycle; by definition IPC = 1 / CPI when both are averaged over the same execution. For a single-issue machine, IPC ≤ 1; for superscalar designs, IPC is bounded by the issue width and microarchitectural limits.

  • CPU performance (the fundamental equation). On first principles: total CPU time equals number of clock cycles times cycle time; cycles equals instruction count times average CPI. Universities state this exactly and ask derivations: start from time = cycles × cycle time, cycles = IC × CPI, so combine.

  • Speedup — ratio of old execution time to new execution time: speedup = T_old / T_new. If >1 the new system is faster.

  • Amdahl's law (exact statement used in exams): If a fraction p of execution time can be improved by a factor s (s >= 1), and the remaining fraction (1 − p) is unchanged, the overall speedup S is S = 1 / ((1 − p) + p / s). Assumptions: p is measured on the original baseline execution time; improvement applies only to that fraction; serial portion remains serial. Derivation (stepwise): baseline time = 1 (normalize). Improved time = (1 − p) + p / s. Speedup = 1 / improved_time.

  • Maximum possible speedup: as s → infinity, S_max = 1 / (1 − p). This shows diminishing returns when p is small.

  • Average memory access time (AMAT) — used when memory effects matter (link to the memory hierarchy topic): AMAT = hit_time + miss_rate × miss_penalty. Define terms: hit_time is time to access when data is in the cache, miss_rate fraction of accesses missing, miss_penalty additional time on a miss to service from lower level.

  • Pipeline — dividing instruction execution into stages (examples: IF, ID, EX, MEM, WB). Ideal pipeline throughput is one instruction per cycle after fill, but hazards reduce this. Draw in exams: five horizontal boxes labeled IF, ID, EX, MEM, WB left to right; below them draw a grid with cycles on horizontal axis and instructions vertical, show movement diagonally right-down to indicate each stage per cycle.

  • Hazard — a condition that prevents the next instruction in the pipeline from executing in its designated clock cycle. Types: structural, data, and control hazards. Explain typical mitigation: forwarding, stall (bubble), branch prediction.

  • Parallel speedup and scalability — for multiple processors, speedup is T_serial / T_parallel. Scaling can be strong (fixed problem size) or weak (problem size grows). For scaled speedup with problem size growing (Gustafson's law), S(N) = N − α (N − 1), where α is the serial fraction on one processor.

Notes on measurement: use wall-clock time for whole-program latency; use profilers to measure time fractions p. Always state the baseline and workload when quoting speedup.

Formulas

clock_rate = 1 / cycle_time — clock_rate in Hz (s⁻¹), cycle_time in seconds (s). Applies always for synchronous digital systems.

CPU_time = instruction_count × CPI × cycle_time — instruction_count (dimensionless), CPI (cycles/instruction), cycle_time (s). Applies for CPU-bound program time on a single processor.

CPI_average = Σ (CPI_i × fraction_i) — CPI_i is cycles for instruction class i, fraction_i is fraction of instructions in class i (dimensionless). Use when instruction mix is known. IPC = 1 / CPI — averaged over the same execution; for single-issue, IPC ≤ 1; for width W superscalar, IPC ≤ W. instructions_per_second = IPC × clock_rate and equivalently CPU_time = instruction_count / (IPC × clock_rate) — alternative performance forms.

speedup = T_old / T_new — T_old and T_new in seconds. Applies whenever comparing two implementations on same workload.

Amdahl: speedup = 1 / ((1 − p) + p / s) — p is fraction of original execution time improved (0 ≤ p ≤ 1), s is speedup of that part.

AMAT = hit_time + miss_rate × miss_penalty — times in seconds, miss_rate dimensionless. Applies for single-level cache; extend to multi-level by adding further miss terms. For two levels explicitly: AMAT = hit_time_L1 + miss_rate_L1 × (hit_time_L2 + miss_rate_L2 × miss_penalty_L2).

theoretical_pipeline_speedup ≤ number_of_stages — number_of_stages dimensionless. This is an upper bound assuming no stalls and equal stage times.

Worked examples

Example 1 — Basic CPU-time and speedup by higher clock rate Problem data: a program executes IC = 1.2×10^9 instructions, average CPI = 2.0 on baseline, baseline clock_rate = 2.5 GHz. New design doubles the clock_rate to 5.0 GHz without changing CPI. Compute baseline CPU time, new CPU time, and speedup. Steps:

  1. cycle_time_baseline = 1 / 2.5×10^9 = 0.4×10^−9 s.
  2. CPU_time_baseline = IC × CPI × cycle_time = 1.2×10^9 × 2.0 × 0.4×10^−9 = 0.96 s. (showing units: instructions × cycles/instruction × seconds/cycle = s)
  3. cycle_time_new = 1 / 5.0×10^9 = 0.2×10^−9 s.
  4. CPU_time_new = 1.2×10^9 × 2.0 × 0.2×10^−9 = 0.48 s.
  5. speedup = 0.96 / 0.48 = 2.0. Answer: baseline 0.96 s, new 0.48 s, speedup 2.0.

Example 2 — Amdahl's law: improving floating-point unit Problem data: on a program, floating-point operations account for p = 0.30 (30%) of execution time. You make the FP unit 5× faster (s = 5). Compute overall speedup and maximum possible if s → ∞. Steps:

  1. Apply Amdahl: speedup = 1 / ((1 − 0.30) + 0.30 / 5) = 1 / (0.70 + 0.06) = 1 / 0.76 = 1.315789. Answer: overall speedup ≈ 1.316.
  2. Maximum as s → ∞: S_max = 1 / (1 − p) = 1 / 0.70 ≈ 1.429.

Example 3 — Cache AMAT effect on CPU time Problem data: program executes IC = 500×10^6 instructions. Each instruction does 1 memory access on average. Baseline: cache hit_time = 1 ns, miss_rate = 0.02, miss_penalty = 50 ns. CPU CPI excluding memory = 1.0, cycle_time = 1 ns. Compute CPU time including memory, then effect of reducing miss_rate to 0.01. Steps:

  1. AMAT_baseline = 1 ns + 0.02 × 50 ns = 1 + 1 = 2 ns per access.
  2. Memory stall cycles per instruction = accesses_per_instruction × miss_rate × (miss_penalty / cycle_time) = 1 × 0.02 × (50 ns / 1 ns) = 1.0 cycles.
  3. Total CPI = CPI_base + memory_stall_cycles = 1.0 + 1.0 = 2.0.
  4. CPU_time = IC × CPI × cycle_time = 500×10^6 × 2.0 × 1 ns = 500×10^6 × 2.0 × 1×10^−9 s = 1.0 s.
  5. With miss_rate = 0.01: memory_stall_cycles = 1 × 0.01 × (50/1) = 0.5 → total CPI = 1.5 → CPU_time = 500×10^6 × 1.5 × 1 ns = 0.75 s. Answer: baseline CPU_time 1.0 s, improved 0.75 s, speedup ≈ 1.333.

Common mistakes

  • Forgetting to change instruction count or instruction mix after an optimization that changes the program (e.g., compiler changes).
  • Not carrying units (seconds, ns, Hz) through calculations.
  • Assuming pipeline speedup equals number of stages without accounting for hazards and stage imbalance.
  • Using miss_rate and hit_rate interchangeably without noting hit_rate = 1 − miss_rate.

For your exam

Types of questions:

  • Definitions and short notes: define performance, CPI, Amdahl's law, AMAT, pipeline and hazards.
  • Derivations: show the CPU performance equation from first principles; derive Amdahl's formula and maximum speedup limit.
  • Numericals: compute CPU times, speedups, AMAT, CPI average with given instruction mix.
  • Diagrams: draw a pipeline stage diagram (boxes IF, ID, EX, MEM, WB) and a timing diagram (instructions vs cycles) showing a stall or forwarding example.
  • Design discussion: given cost or area, argue which part to optimize using Amdahl reasoning; include measurements and justify using p.

What a full-marks answer includes:

  • Clear statement of baseline, units and assumptions.
  • Correct formula early (e.g., CPU_time = IC × CPI × cycle_time) and substitution with units.
  • Step-by-step arithmetic with units and rounding only at the end.
  • For Amdahl: state p measured on original execution time and show derivation or substitution.
  • If a diagram is required, label everything (stages, cycles, hazards) and explain what the diagram shows.

Quick check

  1. Write the CPU performance equation linking instruction count, CPI and cycle time.
  2. State Amdahl's law formula and explain meaning of p and s in one sentence.
  3. Give the AMAT formula for a single-level cache. Answers: CPU_time = instruction_count × CPI × cycle_time; speedup = 1 / ((1 − p) + p / s) where p is fraction of original time improved and s is speedup of that fraction; AMAT = hit_time + miss_rate × miss_penalty.

Finished this topic? Mark it so your progress, study plan and readiness keep up.

Stuck on something here?