Probability & Statistics

What is Probability?

Flip a fair coin. What is the chance of heads? 1/2. Probability measures how likely an event is, from 0 (impossible) to 1 (certain).

Sample space Ω = set of all possible outcomes. Event A = a subset of outcomes we care about.

Roll a die: Ω = {1, 2, 3, 4, 5, 6}
Event A = "roll an even number" = {2, 4, 6}
P(A) = 3/6 = 1/2

Basic Rules

P(Aᶜ) = 1 − P(A)                        [complement]
P(A ∪ B) = P(A) + P(B) − P(A ∩ B)       [inclusion-exclusion]
P(A ∪ B) = P(A) + P(B)  only if A, B are mutually exclusive

---

Conditional Probability — Updating Beliefs

"Given that it's cloudy, what's the probability of rain?"

P(rain | cloudy) = P(rain AND cloudy) / P(cloudy)

General formula:

P(A | B) = P(A ∩ B) / P(B)     [requires P(B) > 0]

Independence vs Mutually Exclusive

These are completely different concepts that confuse many students:

Concept	Definition	Real-world analogy
Independent	P(A∩B) = P(A)·P(B)	Two separate coin flips
Mutually exclusive	P(A∩B) = 0	"heads" and "tails" on same flip

Mutually exclusive events (if P(A)>0 and P(B)>0) are actually dependent — if A happened, B definitely didn't!

---

Bayes' Theorem — Reversing Conditional Probability

You know P(B|A), but want P(A|B). Bayes flips it:

P(A|B) = P(B|A) · P(A) / P(B)

The medical test example (classic GATE scenario):

• Disease prevalence: P(D) = 0.01 (1% of population)
• Test sensitivity: P(+|D) = 0.95 (if you have disease, 95% chance test is positive)
• False positive rate: P(+|no D) = 0.10

What is P(D|+)? (Probability you actually have disease given positive test?)

P(+) = P(+|D)·P(D) + P(+|no D)·P(no D)
     = 0.95×0.01 + 0.10×0.99
     = 0.0095 + 0.099 = 0.1085

P(D|+) = P(+|D)·P(D) / P(+) = 0.0095 / 0.1085 ≈ 0.088

Surprising result: Even with 95% sensitivity, a positive test only means ~9% chance of disease! This is because the disease is rare.

---

Discrete Random Variables and Distributions

Binomial Distribution B(n, p)

Scenario: Flip a biased coin (P(heads)=p) exactly n times. X = number of heads.

P(X = k) = C(n,k) · pᵏ · (1−p)ⁿ⁻ᵏ
E[X] = np
Var(X) = np(1−p)

Example: 10 questions, each answered randomly (p=0.25). Expected correct = 10 × 0.25 = 2.5.

Geometric Distribution G(p)

Scenario: Keep flipping until first head. X = number of flips needed.

P(X = k) = (1−p)ᵏ⁻¹ · p     [k = 1, 2, 3, ...]
E[X] = 1/p
Memoryless: P(X > s+t | X > s) = P(X > t)

Poisson Distribution Poisson(λ)

Scenario: Rare events in a fixed interval. Customers arriving, typos per page, cosmic rays hitting a detector.

P(X = k) = e⁻λ · λᵏ / k!
E[X] = Var(X) = λ    ← mean equals variance — unique to Poisson!

When to use Poisson: n is very large, p is very small, np = λ is moderate.

---

Continuous Distributions

Normal Distribution N(μ, σ²)

The famous bell curve. Symmetric around mean μ.

68-95-99.7 Rule:
P(μ−σ < X < μ+σ)   ≈ 68%
P(μ−2σ < X < μ+2σ) ≈ 95%
P(μ−3σ < X < μ+3σ) ≈ 99.7%

Standardise: Z = (X − μ)/σ converts any normal to standard normal N(0,1).

Exponential Distribution Exp(λ)

Models time between events in a Poisson process (time until next customer, equipment lifetime).

E[X] = 1/λ,  Var(X) = 1/λ²
Memoryless: P(X > s+t | X > s) = P(X > t)

---

Expected Value and Variance

Expected value = weighted average of all outcomes:

E[X] = Σ x · P(X = x)   [discrete]

Example: Fair die: E[X] = 1·(1/6) + 2·(1/6) + ... + 6·(1/6) = 21/6 = 3.5

Variance = expected squared deviation from mean:

Var(X) = E[X²] − (E[X])²    ← shortcut formula

Key linearity rules:

E[aX + b] = a·E[X] + b
Var(aX + b) = a²·Var(X)
E[X + Y] = E[X] + E[Y]           [always true]
Var(X + Y) = Var(X) + Var(Y)     [only if X, Y are independent]

---

Quick Check

Q1. P(A) = 0.4, P(B) = 0.3, P(A∩B) = 0.1. Find P(A|B).

Answer: P(A|B) = P(A∩B)/P(B) = 0.1/0.3 = 1/3 ≈ 0.33.

Q2. X ~ Poisson(4). What is E[X] and Var(X)?

Answer: Both equal 4 (Poisson mean = variance = λ).

Q3. X ~ Binomial(20, 0.3). Find E[X] and Var(X).

Answer: E[X] = 20×0.3 = 6. Var(X) = 20×0.3×0.7 = 4.2.