Matrix Multiplication — the Engine of Neural Nets

Overview

A neural network layer takes a batch of input vectors and produces a batch of output vectors, and it does this with one matrix multiplication. Each output entry is a dot product between an input row and a weight column, so multiplying an (m×k) matrix by a (k×n) matrix gives an (m×n) result — the inner dimensions must match, which is the rule behind every shape error you will ever hit. The reason GPUs matter for AI is precisely that they multiply enormous matrices in parallel. Understand matrix multiply as "apply this transformation to every vector in the batch simultaneously" and you understand what a layer physically does. Order matters (AB ≠ BA), and the shape bookkeeping (m×k · k×n → m×n) is worth memorising cold.

The shape rule: inner dimensions must match

import numpy as np

X = np.random.randn(32, 784)    # 32 examples, 784 features (m x k)
W = np.random.randn(784, 128)   # weight matrix (k x n)

H = X @ W                       # (32, 784) @ (784, 128)
print(H.shape)                  # (32, 128)  -> 32 examples, 128 hidden units

# Mismatch -> error:
# X @ np.random.randn(100, 128)  # ValueError: 784 != 100

A linear layer is one matmul plus a bias

import numpy as np

def linear_layer(X, W, b):
    return X @ W + b            # the whole forward pass of a Dense layer

X = np.random.randn(4, 3)       # 4 examples, 3 features
W = np.random.randn(3, 2)       # 3 -> 2 units
b = np.zeros(2)

out = linear_layer(X, W, b)
print(out.shape)                # (4, 2)

Key Points to Remember

• 1Matrix multiply = a grid of dot products; result[i,j] = row i · column j
• 2Shape rule: (m×k)·(k×n) → (m×n); inner dimensions must match
• 3A dense layer is exactly one matmul plus a bias: H = XW + b
• 4GPUs accelerate AI because they parallelise large matrix multiplications