Linear Transformations & Geometric Intuition

Overview

The deepest way to understand linear algebra is geometric: multiplying a vector by a matrix transforms space. The matrix's columns tell you where the basis vectors (the arrows along each axis) land, and every point moves accordingly — rotations, stretches, flips, and shears are all matrix multiplications. This reframes a neural network layer: XW does not just "mix numbers", it moves your data to a new coordinate system where the next layer can separate it more easily. Concepts like eigenvectors (directions the transformation leaves pointing the same way) and SVD (any transformation = rotate, scale, rotate) only make sense once you hold this picture. When you hear "the model learns a good representation", it means it learned transformations that reshape messy data into a space where classes pull apart.

A matrix moves space; its columns are where axes land

import numpy as np

# 90-degree rotation matrix
theta = np.pi / 2
R = np.array([[np.cos(theta), -np.sin(theta)],
              [np.sin(theta),  np.cos(theta)]])

e1 = np.array([1.0, 0.0])     # x-axis unit vector
e2 = np.array([0.0, 1.0])     # y-axis unit vector
print(np.round(R @ e1, 3))    # [0. 1.]  -> x-axis lands on y-axis
print(np.round(R @ e2, 3))    # [-1. 0.] -> y-axis lands on -x-axis

A layer transforms data into a friendlier space

import numpy as np

# Scale x by 3, y by 0.5 -> stretches a cloud of points
S = np.array([[3.0, 0.0],
              [0.0, 0.5]])
points = np.array([[1.0, 2.0],
                   [2.0, 1.0]])
print(points @ S.T)   # each point re-scaled into the new space

Key Points to Remember

• 1A matrix is a function that transforms space (rotate, scale, shear, flip)
• 2The columns of a matrix show where the basis (axis) vectors land
• 3A neural layer XW relocates data into a new coordinate system
• 4"Representation learning" = learning transformations that make data separable