The four fundamental subspaces of a matrix are the ranges and kernels of the linear maps defined by the matrix and its transpose. They are linked to each other by several interesting relations.
In what follows, we denote by:
the space of all column vectors;
the space of all column vectors.
For the time being, we are not going to specify whether the entries of the vectors are real or complex.
Furthermore, we consider a matrix , which defines a linear map that associates to any vector the product :
Example Define the matrix The associated linear map transforms vectors into vectors. For instance,and
The first fundamental subspace is the range of the linear map defined by :which is called column space of .
Example Consider the matrix introduced in the previous example. We can write a product asTherefore, the column space of is the span of two column vectors:
More in general, the column space of is the span of its columns.
The second fundamental space is the kernel of the linear map defined by :which is called null space of .
Example We keep defining as in the previous example. There, we have shown thatfrom which it is clear that if and only if , in which case can take any value. Thus, all vectors of the null space have the formwhere the scalar can be chosen arbitrarily. In other words, the null space of is the linear span of one vector:
Let us consider a second linear map , defined by
The third fundamental subspace is the range of the linear map defined by :which is called row space of .
Example We continue with the same matrix as in the previous examples. Thus,andSo, the row space of is the span of two vectors:
The fourth fundamental space is the kernel of the linear map defined by :which is called left-hand null space of .
Example In the previous example, we have thatTherefore, if and only if . Thus, the left-hand null space of contains only the zero vector:
The dimensions of the four subspaces are all related to each other.
A product can be written as a linear combination of the columns of :where the coefficients of the linear combination are the entries of . Therefore, the dimension of is the column rank of .
Similarly, a product can be seen as a linear combination of the rows of with coefficients . Therefore, the dimension of is the row rank of .
We have previously proved that row and column rank coincide, that is,
By the rank-nullity theorem, we have and
By combining (1), (2) and (3), we can get many interesting relations among the dimensions of the four subspaces.
For example, both and are subspaces of and we have
Similarly, and are subspaces of and we have
Example In the previous examples, is a matrix. Thus we have and . We have shown thatSince the two vectors spanning are linearly independent, we haveMoreover, we have found thatwhich impliesThus,
When the matrix is real (i.e., its entries are real numbers), not only the dimensions of the four fundamental subspaces are related to each other, but the four spaces form two couples of orthogonal complements.
Proposition If is real, the two subspaces and are orthogonal complements.
By definition, the orthogonal complement of iswhere is the standard inner product for spaces of real vectors. But if and only if there exists such that . Thus,Moreover,As a consequence, we haveSince and for every , we havewhich, by the definiteness property of the inner product, implies Hence,
Proposition If is real, the two subspaces and are orthogonal complements.
By definition, the orthogonal complement of isBut if and only if there exists such that . Thus,Moreover,As a consequence, we haveSince and for every , we havewhich, by the definiteness property of the inner product, implies Hence,
As a consequence, the domain and codomain of the linear map can be written as two direct sums:
Example In the previous examples we haveandA generic vector can be written aswhere and are scalars. Any vector can be written asThe inner product between a vector of and a vector of iswhich shows that couples of vectors from the two subspaces are orthogonal, as expected.
Below you can find some exercises with explained solutions.
Suppose a matrix is full-rank. Find the dimensions of its four subspaces.
We infer from the dimension of the matrix thatandSince the matrix is full-rank, we haveMoreover, by the rank-nullity theorem, we have and
Please cite as:
Taboga, Marco (2021). "The four fundamental subspaces", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/four-fundamental-subspaces.
Most of the learning materials found on this website are now available in a traditional textbook format.