This lecture discusses some facts about matrix products and their rank. In particular, we analyze under what conditions the rank of the matrices being multiplied is preserved.

The next proposition provides a bound on the rank of a product of two matrices.

Proposition Let be a matrix and an matrix. Then,

Proof

The space spanned by the columns of is the space of all vectors that can be written as linear combinations of the columns of :where is the vector of coefficients of the linear combination. We can also writewhere is an vector (being a product of an matrix and an vector). Thus, any vector can be written as a linear combination of the columns of , with coefficients taken from the vector . As a consequence, the space is no larger than the span of the columns of , whose dimension is . This implies that the dimension of is less than or equal to . Since the dimension of is the rank of , we haveNow, the space spanned by the rows of is the space of all vectors that can be written as linear combinations of the rows of :where is the vector of coefficients of the linear combination. We can also writewhere is a vector (being a product of a vector and a matrix). Thus, any vector can be written as a linear combination of the rows of , with coefficients taken from the vector . As a consequence, the space is no larger than the span of the rows of , whose dimension is . This implies that the dimension of is less than or equal to . Since the dimension of is the rank of , we haveThe two inequalitiesare satisfied if and only if

Another important fact is that the rank of a matrix does not change when we multiply it by a full-rank matrix.

Proposition Let be a matrix and a square matrix. If is full-rank, then

Proof

Remember that the rank of a matrix is the dimension of the linear space spanned by its columns (or rows). We are going to prove that the ranks of and are equal because the spaces generated by their columns coincide. Denote by the space generated by the columns of . Any vector can be written as a linear combination of the columns of :where is the vector of coefficients of the linear combination. Since is full-rank and square, it has linearly independent columns that span the space of all vectors (they are equivalent to the canonical basis). Therefore, there exists an vector such thatThusWe have just proved that any vector can be written as a linear combination of the columns of . Furthermore, the columns of do not generate any vector . To see this, note that for any vector of coefficients , if thenso that . Thus, we have proved that the space spanned by the columns of and that spanned by the columns of coincide. As a consequence, also their dimensions (which by definition are equal to the ranks of and ) coincide.

Proposition Let be a matrix and a square matrix. If is full-rank, then

Proof

The proof of this proposition is almost identical to that of the previous proposition. It is left as an exercise (see the exercise below with its solution).

An immediate corollary of the previous two propositions is that the product of two full-rank square matrices is full-rank.

Proposition Let and be two full-rank matrices. Then, their products and are full-rank.

Proof

Being full-rank, both matrices have rank . Therefore, by the previous two propositionsBut and are , so they are full-rank.

We now present a very useful result concerning the product of a non-square matrix and its transpose.

Proposition Let be a full-rank matrix with . Then, the product is full-rank.

Proof

Let be the space of all vectors. Suppose that there exists a non-zero vector such thatThen,ororwhere denotes the -th entry of the column vector . This is possible only if for , that is, only ifwhich is impossible because is full-rank, it has less columns than rows and, hence, its columns are linearly independent. Thus, the only vector that givesis , which implies that the columns of are linearly independent and is full-rank.

The matrix is called a Gram matrix.

Below you can find some exercises with explained solutions.

Let be a matrix and a full-rank matrix. Prove that if is full-rank, then

Solution

Keep in mind that the rank of a matrix is the dimension of the space generated by its rows. We are going to prove that the spaces generated by the rows of and coincide, so that they trivially have the same dimension, and the ranks of the two matrices are equal. Denote by the space spanned by the rows of . Any is a linear combination of the rows of :where is the vector of coefficients of the linear combination. Since is full-rank, it has linearly independent rows that span the space of all vectors. As a consequence, there exists a vector such thatThusThis means that any is a linear combination of the rows of . Moreover, the rows of do not generate any vector : for any vector of coefficients , if thenso that . Thus, the space spanned by the rows of and that spanned by the rows of coincide. As a consequence, also their dimensions coincide.

Please cite as:

Taboga, Marco (2021). "Matrix product and rank", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/matrix-product-and-rank.

The books

Most of the learning materials found on this website are now available in a traditional textbook format.

Featured pages

- Central Limit Theorem
- Characteristic function
- Uniform distribution
- F distribution
- Gamma distribution
- Law of Large Numbers

Explore

Main sections

- Mathematical tools
- Fundamentals of probability
- Probability distributions
- Asymptotic theory
- Fundamentals of statistics
- Glossary

About

Glossary entries

- Type II error
- IID sequence
- Convolutions
- Continuous mapping theorem
- Probability space
- Continuous random variable

Share

- To enhance your privacy,
- we removed the social buttons,
- but
**don't forget to share**.