Source: MATH 416 Abstract Linear Algebra, UIUC Practice Exam
Tags: inner product, inner product space, Gram-Schmidt, orthogonal, orthonormal, orthogonal complement, Cauchy-Schwarz, Frobenius inner product, projection, linear algebra
Difficulty: Intermediate Prerequisites: Vector spaces, linear independence, basis and dimension, matrix operations.
Inner product spaces give abstract vector spaces a notion of geometry: lengths, angles, and perpendicularity. Everything from measuring how "close" two vectors are to decomposing a space into orthogonal pieces depends on having a well-defined inner product. This topic is foundational for the rest of MATH 416, since adjoints, the Spectral Theorem, and SVD all rely on inner product structure. If you are coming in cold, make sure you are comfortable with vector spaces and bases first.
An inner product is a function on pairs of vectors that behaves like a generalised dot product. It must be linear in its first argument, conjugate symmetric, and positive definite. From an inner product you get orthogonality, the Gram-Schmidt process for building orthogonal bases, and orthogonal complements that let you split any finite-dimensional space into two perpendicular pieces.
Inner product
A function ⟨ · , · ⟩ : V × V → F satisfying three axioms: linearity in the first component, conjugate symmetry (⟨x, y⟩ = conjugate of ⟨y, x⟩), and positive definiteness (⟨x, x⟩ > 0 for all x ≠ 0). Think of it as the abstract version of the dot product that lets you talk about lengths and angles in any vector space.
Linearity in the first component
The inner product satisfies ⟨x + z, y⟩ = ⟨x, y⟩ + ⟨z, y⟩ and ⟨cx, y⟩ = c⟨x, y⟩ for any scalar c. In simple terms, the inner product distributes over addition and pulls out scalars in its first slot.
Conjugate symmetry
⟨x, y⟩ = the complex conjugate of ⟨y, x⟩. Over the reals this just means ⟨x, y⟩ = ⟨y, x⟩, i.e. order does not matter.
Positive definiteness
⟨x, x⟩ > 0 for every nonzero vector x, and ⟨0, 0⟩ = 0. In simple terms, the "length squared" of any nonzero vector is always strictly positive.
Frobenius inner product
An inner product on the space of n × n matrices M_{n×n}(F), defined by ⟨A, B⟩ = tr(BA), where B is the conjugate transpose of B and tr is the trace. Think of it as "flatten both matrices into long vectors and take the dot product."
Cauchy-Schwarz inequality
For any vectors x, y in an inner product space: |⟨x, y⟩| ≤ ‖x‖ · ‖y‖. Equality holds if and only if one vector is a scalar multiple of the other (i.e. they are linearly dependent).
Orthogonal vectors
Two vectors x and y are orthogonal when ⟨x, y⟩ = 0. In simple terms, they are "perpendicular" in the geometry defined by the inner product.
Orthogonal complement
Given a subspace W of an inner product space V, the orthogonal complement W⊥ is the set of all vectors in V that are orthogonal to every vector in W. Formally: W⊥ = {v ∈ V : ⟨v, w⟩ = 0 for all w ∈ W}.
Direct sum decomposition (orthogonal)
In a finite-dimensional inner product space, every subspace W gives V = W ⊕ W⊥. Every vector in V can be written uniquely as the sum of a piece in W and a piece in W⊥.
Gram-Schmidt process
An algorithm that takes any basis {w₁, w₂, ..., wₙ} and produces an orthogonal (or orthonormal) basis {v₁, v₂, ..., vₙ} for the same space.
Orthonormal basis
An orthogonal basis where every vector also has unit length (‖vᵢ‖ = 1). Think of it as the neatest possible coordinate system for a space.
A function ⟨ · , · ⟩ on V must satisfy all three of the following to qualify as an inner product:
Linearity in the first component: ⟨x + z, y⟩ = ⟨x, y⟩ + ⟨z, y⟩ and ⟨cx, y⟩ = c⟨x, y⟩.
Conjugate symmetry: ⟨x, y⟩ = conjugate of ⟨y, x⟩.
Positive definiteness: ⟨x, x⟩ > 0 for all x ≠ 0.
Over the reals, conjugate symmetry reduces to plain symmetry (⟨x, y⟩ = ⟨y, x⟩), and the inner product is also linear in the second component. Over the complex numbers, the second component is conjugate-linear, not linear.
For matrices A, B in M_{n×n}(F):
⟨A, B⟩ = tr(B*A)
where B* is the conjugate transpose and tr is the trace (sum of diagonal entries). This is the standard way to put an inner product on a space of matrices.
Note the order: it is tr(BA), not tr(AB) or tr(A + B).
The inequality |⟨x, y⟩| ≤ ‖x‖ · ‖y‖ holds for all vectors x, y.
Equality (|⟨x, y⟩| = ‖x‖ · ‖y‖) occurs precisely when one vector is a scalar multiple of the other, or when one of them is zero.
This is the abstract generalisation of the fact that cos θ is bounded between -1 and 1.
Given a basis {w₁, w₂, ..., wₙ}, produce orthogonal vectors {v₁, v₂, ..., vₙ}:
Set v₁ = w₁.
For each subsequent vector, subtract off the projections onto all previously constructed orthogonal vectors:
vₖ₊₁ = wₖ₊₁ − Σⱼ₌₁ᵏ [ ⟨wₖ₊₁, vⱼ⟩ / ‖vⱼ‖² ] · vⱼ
To get an orthonormal basis, normalise each vᵢ by dividing by its norm.
The key formula is the subtraction of projections. Option B (adding projections) and option C (missing the ‖vⱼ‖² denominator) are common exam distractors.
For any subspace W of a finite-dimensional inner product space V:
V = W ⊕ W⊥ (the direct sum decomposition).
Every vector v ∈ V can be written uniquely as v = u + z where u ∈ W and z ∈ W⊥.
dim(W) + dim(W⊥) = dim(V).
To find W⊥ when W = span({w₁, ..., wₖ}), solve the system ⟨x, wᵢ⟩ = 0 for each spanning vector wᵢ.
Gram-Schmidt formula: vₖ₊₁ = wₖ₊₁ − Σⱼ₌₁ᵏ [ ⟨wₖ₊₁, vⱼ⟩ / ‖vⱼ‖² ] · vⱼ
Frobenius inner product: ⟨A, B⟩ = tr(B*A)
Cauchy-Schwarz: |⟨x, y⟩| ≤ ‖x‖ · ‖y‖
Orthogonal complement dimension: dim(W⊥) = dim(V) − dim(W)
Gram-Schmidt is the backbone of QR factorisation, which is how numerical software (MATLAB, NumPy) solves least-squares problems and fits models to data. Orthogonal complements show up whenever you split a signal into a "useful" component and noise, as in signal processing and statistics (regression residuals live in the orthogonal complement of the column space).
Students often think the inner product must be linear in both components simultaneously. Over the complex numbers, the second component is conjugate-linear, not linear.
Students frequently forget the ‖vⱼ‖² denominator in the Gram-Schmidt formula and just use ⟨wₖ₊₁, vⱼ⟩ as the coefficient. The denominator is essential unless the basis is already orthonormal.
Students sometimes confuse W ∩ W⊥ = {0} with W ∩ W⊥ = V. The intersection is always just the zero vector; it is the direct sum W ⊕ W⊥ that equals V.
The Frobenius inner product is tr(BA), not tr(AB) and not tr(A + B). The placement of the conjugate transpose matters.
⚠️ The exam tests whether you know that linearity in the first component is the defining property. Do not confuse this with conjugate linearity (which applies to the second component over ℂ).
⚠️ The Frobenius inner product definition ⟨A, B⟩ = tr(B*A) is a common multiple-choice target. Watch the order of the arguments.
⚠️ Cauchy-Schwarz equality condition (one vector is a scalar multiple of the other) is tested directly. Know when equality holds, not just the inequality.
⚠️ Gram-Schmidt computation is a standard 5-mark short answer. You must show each step: setting v₁, computing the inner product, computing the norm squared, and subtracting the projection.
⚠️ Finding orthogonal complements by solving ⟨x, w⟩ = 0 is tested directly. Be ready to write out the general solution and give a basis.
True or false: Over the complex numbers, the inner product is linear in both its first and second components.
Fill in the blank: The Frobenius inner product is defined as ⟨A, B⟩ = tr(______).
True or false: If |⟨x, y⟩| = ‖x‖ · ‖y‖, then x and y must both be zero.
Fill in the blank: In a finite-dimensional inner product space, V = W ⊕ ______.
True or false: In the Gram-Schmidt process, you add the projections onto previously constructed vectors.
Answers: 1. False (conjugate-linear in the second). 2. B*A. 3. False (one is a scalar multiple of the other). 4. W⊥. 5. False (you subtract them).
Q: Let V be an inner product space over F. Which property is strictly required for the function to be defined as an inner product regarding its first component?
A: Linearity. Conditions (a) and (b) in the definition require ⟨x + z, y⟩ = ⟨x, y⟩ + ⟨z, y⟩ and ⟨cx, y⟩ = c⟨x, y⟩.
Q: For the Frobenius inner product on M_{n×n}(F), how is ⟨A, B⟩ defined?
A: ⟨A, B⟩ = tr(B*A).
Q: If |⟨x, y⟩| = ‖x‖ · ‖y‖, what can be concluded about x and y?
A: One is a scalar multiple of the other (or at least one is zero). This is the equality condition of the Cauchy-Schwarz inequality.
Q: Using the Gram-Schmidt process, find an orthogonal basis for W = span({(1,1,0), (1,0,1)}) in ℝ³.
A: Set v₁ = (1, 1, 0). Compute ⟨w₂, v₁⟩ = 1·1 + 0·1 + 1·0 = 1 and ‖v₁‖² = 2. Then v₂ = (1, 0, 1) − (1/2)(1, 1, 0) = (1/2, −1/2, 1). The orthogonal basis is {(1, 1, 0), (1/2, −1/2, 1)}.
Q: Find the orthogonal complement W⊥ of W = span({(1, 2, 1)}) in ℝ³.
A: A vector (x, y, z) is in W⊥ when ⟨(x, y, z), (1, 2, 1)⟩ = 0, giving x + 2y + z = 0. This is a plane in ℝ³ with basis {(−2, 1, 0), (−1, 0, 1)}.
Q: Which of the following describes the relationship between W and W⊥ in a finite-dimensional inner product space V? (a) W ∩ W⊥ = V (b) V = W ⊕ W⊥ (c) dim(W) = dim(W⊥) (d) W⊥ = {0}
A: (b) V = W ⊕ W⊥. Every vector in V decomposes uniquely into a component in W and a component in W⊥.
This material connects directly to adjoint operators and the Spectral Theorem, both of which require inner product structure. The orthogonal projection formula (next set of notes) is built on the direct sum decomposition V = W ⊕ W⊥. The Singular Value Decomposition also relies on orthonormal bases constructed via Gram-Schmidt.
inner product space, dot product, standard inner product, Frobenius inner product, trace inner product, Cauchy-Schwarz inequality, triangle inequality, orthogonal, perpendicular, orthonormal, Gram-Schmidt process, orthogonalisation, orthogonal complement, perp, direct sum decomposition, projection, norm, length, MATH 416, abstract linear algebra, UIUC