The vec operator is an operator that transforms a matrix into a column vector by vertically stacking the columns of the matrix.

In this lecture we define the vec operator and we prove some of its most important properties.

We start with a definition.

Definition Let be a matrix. Denote by the columns of . The vectorization of , denoted by , is the column vector

Here is an example of how the vec operator works.

Example Define the matrixIts vectorization is

The first two properties of the vec operator are immediate consequences of its definition.

Proposition If is a column vector, then

Proposition If is a row vector, thenwhere denotes the transpose of .

The vec operator is linear, that is, it preserves linear combinations.

Proposition Let and be two matrices and and two scalars. Then

Proof

Denote by the columns of and by the columns of . By the rules of matrix addition and multiplication of a matrix by a scalar, the -th column of is Therefore,

Several properties of the vec operator are also properties of the Kronecker product.

Remember that the Kronecker product is the block matrixwhere denotes the -th entry of .

A property of the Kronecker product that we have already proved and that we will use below is the so-called mixed-product property: if , , and are such that the products and are well-defined, then

The next property concerns outer products, that is, products between a column and a row vector.

Proposition Let be a column vector and a row vector. Then,

Proof

Denote the entries of by . Then,

The next property concerns matrix products.

Proposition Let be a matrix and an matrix. Denote by the columns of . Then,

Proof

Write as a block matrix:By the rules on the multiplication of block matrices, we havewhere each of the products , ..., is a column of . The stated result then follows by the definition of vectorization.

By using the previous proposition, we can prove the next one.

Proposition Let be a matrix and an matrix. Then,where is the identity matrix.

Proof

We havewhere in step we have used the result proved in the previous proposition.

The next property concerns the product of three matrices. We can think of it as a trick that allows us to free a matrix squeezed between two matrices and and bring it out of the product.

Proposition Let be a matrix, an matrix and an matrix. Then,where is the identity matrix.

Proof

Denote the columns of by . The matrix can be expressed aswhere is the -th vector of the canonical basis of the -dimensional vectors (i.e., an vector such that its -th entry is equal to and all its other entries are equal to ). Then, we havewhere: in steps and we have used the linearity of the vec operator; in steps and we have used the above result about the vec of outer products; in step we have used the mixed-product property of the Kronecker product.

Note that the previously derived result is a special case of the last result, obtained by setting . We can similarly obtain other expressions for :

Remember that the trace of a matrix is the sum of its diagonal entries.

The proposition below shows a connection between the vec operator and the trace.

Proposition Let be a matrix and an matrix. Then,where denotes the trace of .

Proof

This is proved as follows:

Below you can find some exercises with explained solutions.

Let be a matrix, an matrix and an vector. Prove that

Solution

Since is a column vector it equal its vec and the vec of its transpose:

Let be a block matrix with blocks , and . Can you express in terms of the vec of the blocks?

Solution

When we vectorize we stack its columns vertically starting from the first column on the left and ending with the last column of the right. Therefore, we first stack all the columns of , then all the columns of and finally those of . Therefore,

Please cite as:

Taboga, Marco (2017). "Vec operator", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/vec-operator.

The book

Most of the learning materials found on this website are now available in a traditional textbook format.

Featured pages

- Central Limit Theorem
- Characteristic function
- Uniform distribution
- F distribution
- Gamma distribution
- Law of Large Numbers

Explore

Main sections

- Mathematical tools
- Fundamentals of probability
- Probability distributions
- Asymptotic theory
- Fundamentals of statistics
- Glossary

About

Glossary entries

- Type II error
- IID sequence
- Convolutions
- Continuous mapping theorem
- Probability space
- Continuous random variable

Share