Search for probability and statistics terms on Statlect
StatLect
Index > Matrix algebra

The four fundamental subspaces

by , PhD

The four fundamental subspaces of a matrix are the ranges and kernels of the linear maps defined by the matrix and its transpose. They are linked to each other by several interesting relations.

Table of Contents

Notation

In what follows, we denote by:

For the time being, we are not going to specify whether the entries of the vectors are real or complex.

Furthermore, we consider a $K	imes L$ matrix A, which defines a linear map [eq1]that associates to any $L	imes 1$ vector $s$ the Kx1 product $As$:[eq2]

Example Define the [eq3] matrix [eq4]The associated linear map transforms $3	imes 1$ vectors into $2	imes 1$ vectors. For instance,[eq5]and[eq6]

Column space

The first fundamental subspace is the range of the linear map defined by A:[eq7]which is called column space of A.

Example Consider the matrix A introduced in the previous example. We can write a product $As$ as[eq8]Therefore, the column space of A is the span of two column vectors:[eq9]

More in general, the column space of A is the span of its columns.

Null space

The second fundamental space is the kernel of the linear map defined by A:[eq10]which is called null space of A.

Example We keep defining A as in the previous example. There, we have shown that[eq11]from which it is clear that $As=0$ if and only if $s_{1}=s_{3}=0$, in which case $s_{2}$ can take any value. Thus, all vectors of the null space have the form[eq12]where the scalar $lpha $ can be chosen arbitrarily. In other words, the null space of A is the linear span of one vector:[eq13]

Row space

Let us consider a second linear map $g:T
ightarrow S$, defined by [eq14]

The third fundamental subspace is the range of the linear map defined by $A^{	op }$:[eq15]which is called row space of A.

Example We continue with the same matrix as in the previous examples. Thus,[eq16]and[eq17]So, the row space of A is the span of two vectors:[eq18]

Left-hand null space

The fourth fundamental space is the kernel of the linear map defined by $A^{	op }$:[eq19]which is called left-hand null space of A.

Example In the previous example, we have that[eq20]Therefore, $A^{	op }t=0$ if and only if $t_{1}=t_{2}=0$. Thus, the left-hand null space of A contains only the zero vector:[eq21]

How the dimensions of the four subspaces are related

The dimensions of the four subspaces are all related to each other.

A product $As$ can be written as a linear combination of the columns of A:[eq22]where the coefficients of the linear combination [eq23] are the $L$ entries of $s$. Therefore, the dimension of [eq24] is the column rank of A.

Similarly, a product $A^{	op }t$ can be seen as a linear combination of the rows of A with coefficients [eq25]. Therefore, the dimension of [eq26] is the row rank of A.

We have previously proved that row and column rank coincide, that is,[eq27]

By the rank-nullity theorem, we have [eq28]and[eq29]

By combining (1), (2) and (3), we can get many interesting relations among the dimensions of the four subspaces.

For example, both [eq26] and [eq31] are subspaces of $S$ and we have[eq32]

Similarly, [eq33] and [eq34] are subspaces of $T$ and we have[eq35]

Example In the previous examples, A is a $2	imes 3$ matrix. Thus we have [eq36] and [eq37]. We have shown that[eq38]Since the two vectors spanning [eq33] are linearly independent, we have[eq40]Moreover, we have found that[eq41]which implies[eq42]Thus,[eq43]

Orthogonal complementarity

When the matrix A is real (i.e., its entries are real numbers), not only the dimensions of the four fundamental subspaces are related to each other, but the four spaces form two couples of orthogonal complements.

Proposition If A is real, the two subspaces [eq26] and [eq45] are orthogonal complements.

Proof

By definition, the orthogonal complement of [eq46] is[eq47]where [eq48] is the standard inner product for spaces of real vectors. But [eq49] if and only if there exists $tin T$ such that $A^{	op }t=r$. Thus,[eq50]Moreover,[eq51]As a consequence, we have[eq52]Since $Asin T$ and [eq53] for every $tin T$, we have[eq54]which, by the definiteness property of the inner product, implies [eq55]Hence,[eq56]

Proposition If A is real, the two subspaces [eq33] and [eq58] are orthogonal complements.

Proof

By definition, the orthogonal complement of [eq33] is[eq60]But [eq61] if and only if there exists $sin S$ such that $As=u$. Thus,[eq62]Moreover,[eq63]As a consequence, we have[eq64]Since $A^{	op }tin S$ and [eq65] for every $sin S$, we have[eq66]which, by the definiteness property of the inner product, implies [eq67]Hence,[eq68]

As a consequence, the domain $S$ and codomain $T$ of the linear map can be written as two direct sums:[eq69]

Example In the previous examples we have[eq70]and[eq71]A generic vector [eq72] can be written as[eq73]where $lpha $ and $eta $ are scalars. Any vector [eq74] can be written as[eq75]The inner product between a vector of [eq26] and a vector of [eq77] is[eq78]which shows that couples of vectors from the two subspaces are orthogonal, as expected.

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Suppose a $5	imes 3$ matrix A is full-rank. Find the dimensions of its four subspaces.

Solution

We infer from the dimension of the matrix that[eq79]and[eq80]Since the matrix is full-rank, we have[eq81]Moreover, by the rank-nullity theorem, we have [eq82]and[eq83]

How to cite

Please cite as:

Taboga, Marco (2017). "The four fundamental subspaces", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/four-fundamental-subspaces.

The book

Most of the learning materials found on this website are now available in a traditional textbook format.