Orthogonality generalizes perpendicularity to any vector space. Orthonormal bases simplify calculations enormously — and Gram-Schmidt is the algorithm that builds them.
Key Concepts
Inner Products
An inner product ⟨u,v⟩ generalizes the dot product. Must satisfy: linearity, symmetry, positive-definiteness. In ℝⁿ: ⟨u,v⟩ = u·v = Σuᵢvᵢ. Induces a norm ‖v‖ = √⟨v,v⟩.
Orthogonal & Orthonormal Sets
Vectors are orthogonal if ⟨uᵢ,uⱼ⟩ = 0 for i≠j. Orthonormal: also each ‖uᵢ‖ = 1. An orthonormal basis makes projections trivial: cᵢ = ⟨v,eᵢ⟩.
Gram-Schmidt Process
Given linearly independent vectors v₁,...,vₙ, build orthonormal set: u₁ = v₁/‖v₁‖. For each k: subtract projections onto previous uᵢ, then normalize.
Orthogonal Projections
proj_v(u) = (u·v/v·v)v. In ℝⁿ with orthonormal basis {e₁,...,eₙ}: proj_W(v) = Σ⟨v,eᵢ⟩eᵢ. Used in least-squares regression, PCA, and signal processing.
Live Python Practice
import math
def dot(u, v):
return sum(a*b for a,b in zip(u,v))
def norm(v):
return math.sqrt(dot(v, v))
def proj(u, v):
"""Project u onto v"""
scale = dot(u, v) / dot(v, v)
return [scale * vi for vi in v]
def gram_schmidt(vectors):
"""Build orthonormal basis from linearly independent vectors"""
basis = []
for v in vectors:
w = list(v)
for e in basis:
p = proj(w, e)
w = [wi - pi for wi, pi in zip(w, p)]
n = norm(w)
if n > 1e-10:
basis.append([wi/n for wi in w])
return basis
# Example: 2D
vectors = [[2, 1], [1, 3]]
basis = gram_schmidt(vectors)
print("Input vectors:", vectors)
print("Orthonormal basis:")
for i, e in enumerate(basis):
print(f" e{i+1} = {[round(x,4) for x in e]}, |e| = {norm(e):.4f}")
print("Dot product of basis vectors:", round(dot(*basis), 10))
# 3D example
vectors3 = [[1,1,0],[1,0,1],[0,1,1]]
basis3 = gram_schmidt(vectors3)
print("\n3D orthonormal basis:")
for i, e in enumerate(basis3):
print(f" e{i+1} = {[round(x,4) for x in e]}")
Interactive Lab
v₁:v₂:
Gray = original. Blue = e₁ (normalized v₁). Orange = projection of v₂ onto e₁. Red = e₂ (orthogonalized v₂). Result: orthonormal basis.
Check Your Understanding
Two vectors are orthogonal when their dot product is: