Search for probability and statistics terms on Statlect
StatLect
Index > Matrix algebra

Commutation matrix

by , PhD

The commutation matrix (or vec-permutation matrix) is used to transform the vectorization of a matrix into the vectorization of its transpose and to commute the factors of a Kronecker product.

Table of Contents

Definition

We start with a definition.

Definition A $KL imes KL$ matrix $P_{KL}$ is a commutation matrix if and only if[eq1]for any $K imes L$ matrix A.

A commutation matrix is also called a vec-permutation matrix because, as we will demonstrate, it is a permutation matrix.

Example

As an example, let us consider the $2 imes 2$ matrix[eq2]

The two vectorizations are[eq3]

The commutation matrix is[eq4]

By carrying out the matrix multiplication, you can check that[eq5]

Existence

A commutation matrix always exists, for any K and $L$.

To prove its existence, note that [eq6] and [eq7] have the same dimension and contain the same entries, arranged in different orders.

In other words, [eq8] is obtained by permuting the rows of [eq9].

But we know that row permutations can be performed by pre-multiplying [eq10] by a permutation matrix.

Thus, the commutation matrix $P_{KL}$ is a permutation matrix obtained by performing on the $KL imes KL$ identity matrix the same row interchanges that transform [eq9] into [eq8].

Permutation matrix

The following properties of permutation matrices, which have been proved previously, apply:

  1. each row of $P_{KL}$ has one entry equal to 1 and all the other entries equal to 0;

  2. each column of $P_{KL}$ has one entry equal to 1 and all the other entries equal to 0;

  3. the rows of $P_{KL}$ form the standard basis of the space of $1 imes KL$ vectors;

  4. the columns of $P_{KL}$ form the standard basis of the space of $KL imes 1$ vectors;

  5. $P_{KL}$ is full-rank;

  6. $P_{KL}$ is orthogonal (i.e., [eq13]).

Orthogonality

By property 6 above (orthogonality) we have[eq14]which implies that[eq15]

Relation with the identity matrix

Let us provide a more precise characterization of the relation between the commutation matrix $P_{KL}$ and the $KL imes KL$ identity matrix $I_{KL}$.

Note that the $left( k,l ight) $-th entry of a $K imes L$ matrix A is equal to:

Therefore:

  1. row number [eq20] of $P_{KL}$ has a 1 in position [eq21] and 0s elsewhere;

  2. column number [eq22] of $P_{KL}$ has a 1 in position [eq20] and 0s elsewhere.

In other words:

  1. row number [eq20] of $P_{KL}$ is equal to row number [eq25] of $I_{KL}$;

  2. column number [eq26] of $P_{KL}$ is equal to column number [eq20] of $I_{KL}$.

Useful matrices and vectors

In order to prove some results about commutation matrices, we will use:

These matrices are such that[eq28]

Explicit formula

We can now provide an explicit formula for the commutation matrix.

Proposition A $KL imes KL$ commutation matrix $P_{KL}$ satisfies[eq29]where $otimes $ denotes the Kronecker product.

Proof

Let A be any $K imes L$ matrix. Then, [eq30]By taking the vectorization of both sides, we obtain[eq31]where in steps $ rame{A}$ and $ rame{B}$ we have used two properties of the vec operator.

Characterization as a block matrix

From the explicit formula above, we can see that the commutation matrix $P_{KL}$ is a block matrix having K rows and $L$ columns of blocks.

Each block has dimension $L imes K$, and the $left( k,l ight) $-th block is equal to $E_{kl}^{ op }$.

Example If $K=2$ and $L=3$, then[eq32]

Other explicit formulae

The next proposition provides two other explicit formulae.

Proposition A $KL imes KL$ commutation matrix $P_{KL}$ satisfies[eq33]where $I_{L}$ denotes the $L imes L$ identity matrix and [eq34] denotes the $K imes K$ identity matrix.

Proof

We have proved above that[eq35]Since the Kronecker product is associative and distributive, and the product of a column by a row is the same as their Kronecker product (in any order), we have:[eq36]Similarly,[eq37]

Special cases

Here are two special cases in which the commutation matrix has a simple form.

Proposition When $K=1$ or $L=1$, then the commutation matrix $P_{KL}$ is equal to the identity matrix.

Proof

For any Kx1 vector a, we have [eq38]which implies $P_{K1}=I$. By the same token, for any $1 imes L$ vector $b$, we have [eq39]which implies $P_{1L}=I$.

Commutation properties

The commutation matrix takes its name from the fact that it can be used to commute the factors of a Kronecker product.

Proposition Let A be a $K imes L$ matrix and $B$ an $M imes N$ matrix. Then,[eq40]

Proof

Take any $L imes N$ matrix $C$. Then,[eq41]where in steps $ rame{A}$ and $ rame{B}$ we have used a property of the vec operator. Hence,[eq42]for any matrix $C$, which implies that [eq43]

Proposition (general) Let A be a $K imes L$ matrix and $B$ an $M imes N$ matrix. Then,[eq44]

Proof

We have demonstrated above that[eq45]Now pre-multiply both sides of the equation by $P_{KM}^{-1}=P_{MK}$ to obtain the desired result.

Proposition Let a be a Kx1 vector and $b$ an $M imes 1$ vector. Then,[eq46]

Proof

By the previous proposition, we have[eq47]But $P_{11}=1$.

Proposition Let A be a $K imes L$ matrix and $b$ an $M imes 1$ vector. Then,[eq48]

Proof

These are all special cases of the general proposition above[eq49]in which one of the two commutation matrices is equal to the identity matrix.

Other properties

Commutation matrices enjoy several other useful properties that have not been presented in this lecture.

For more details, consult Abadir and Magnus (2005), Harville (2008), Magnus and Neudecker (2019).

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Explicitly write the commutation matrix $P_{24}$

Solution

We use the characterization as a block matrix:[eq50]

Exercise 2

Prove that, when the two indices coincide, the trace of the commutation matrix is [eq51]

Solution

The explicit formula for the vec-permutation matrix becomes [eq52]Since the trace is a linear operator and the trace of a Kronecker product equals the product of the traces, we have[eq53]The matrices $u_{k}u_{l}^{ op }$ and $u_{l}u_{k}^{ op }$ have a non-zero diagonal entry (which is unique and equal to 1) only when $k=l$. Therefore,[eq54]

References

Abadir, K. M., and Magnus, J. R. (2005) Matrix Algebra, Cambridge University Press.

Harville, D. A. (2008) Matrix Algebra From a Statistician's Perspective, Springer.

Magnus, J. R., and Neudecker, H. (2019) Matrix Differential Calculus with Applications in Statistics and Econometrics, Wiley.

How to cite

Please cite as:

Taboga, Marco (2021). "Commutation matrix", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/commutation-matrix.

The books

Most of the learning materials found on this website are now available in a traditional textbook format.

Featured pages
Explore
Main sections
About
Glossary entries
Share
  • To enhance your privacy,
  • we removed the social buttons,
  • but don't forget to share.