Exponents, Logarithms & Summation Notation

Overview

Three pieces of notation appear constantly in AI and scare people needlessly. Exponentials (e^x) show up in the sigmoid, softmax, and anywhere probabilities are formed. Logarithms (log x) are their inverse and appear in every loss function that involves probabilities (log-likelihood, cross-entropy) because they turn tiny multiplied probabilities into manageable added numbers and punish confident-but-wrong predictions hard. Summation notation, the big Σ, is nothing more than "add these up in a loop" — once you read Σ as `for` and `+=`, dense-looking formulas become code you already know how to write. Nail these three and most ML formulas stop looking like hieroglyphics.

e^x and log: inverses that undo each other

import numpy as np

p = np.array([0.9, 0.2, 0.01])   # three probabilities
print(np.log(p))                 # [-0.105 -1.609 -4.605]  (more negative = more surprising)

# multiplying probabilities vs adding their logs (same ranking, safe math):
print(np.prod(p))                # 0.0018
print(np.exp(np.sum(np.log(p)))) # 0.0018  -> log-sum then exp recovers it

Σ (sigma) is a for-loop

import numpy as np

x = np.array([4.0, 8.0, 6.0, 2.0])

# The formula (1/n) * Σ x_i  is literally:
total = 0.0
for xi in x:            # the Σ
    total += xi
mean = total / len(x)

print(mean)             # 5.0
print(np.mean(x))       # 5.0  -> NumPy writes the loop for you

Key Points to Remember

• 1e^x grows fast and stays positive; log is its inverse and compresses scale
• 2ML uses log-probabilities to avoid underflow and to shape loss functions
• 3Σ (sigma) means "iterate and add" — a for-loop in disguise
• 4log turns products into sums, which is why log-likelihood is everywhere