The vec operator is an operator that transforms a matrix into a column vector by vertically stacking the columns of the matrix.
In this lecture we define the vec operator and we prove some of its most important properties.
We start with a definition.
Definition Let be a matrix. Denote by the columns of . The vectorization of , denoted by , is the column vector
Here is an example of how the vec operator works.
Example Define the matrixIts vectorization is
The first two properties of the vec operator are immediate consequences of its definition.
Proposition If is a column vector, then
Proposition If is a row vector, thenwhere denotes the transpose of .
The vec operator is linear, that is, it preserves linear combinations.
Proposition Let and be two matrices and and two scalars. Then
Several properties of the vec operator are also properties of the Kronecker product.
A property of the Kronecker product that we have already proved and that we will use below is the so-called mixed-product property: if , , and are such that the products and are well-defined, then
The next property concerns outer products, that is, products between a column and a row vector.
Proposition Let be a column vector and a row vector. Then,
Denote the entries of by . Then,
The next property concerns matrix products.
Proposition Let be a matrix and an matrix. Denote by the columns of . Then,
Write as a block matrix:By the rules on the multiplication of block matrices, we havewhere each of the products , ..., is a column of . The stated result then follows by the definition of vectorization.
By using the previous proposition, we can prove the next one.
Proposition Let be a matrix and an matrix. Then,where is the identity matrix.
We havewhere in step we have used the result proved in the previous proposition.
The next property concerns the product of three matrices. We can think of it as a trick that allows us to free a matrix squeezed between two matrices and and bring it out of the product.
Proposition Let be a matrix, an matrix and an matrix. Then,where is the identity matrix.
Denote the columns of by . The matrix can be expressed aswhere is the -th vector of the canonical basis of the -dimensional vectors (i.e., an vector such that its -th entry is equal to and all its other entries are equal to ). Then, we havewhere: in steps and we have used the linearity of the vec operator; in steps and we have used the above result about the vec of outer products; in step we have used the mixed-product property of the Kronecker product.
Note that the previously derived result is a special case of the last result, obtained by setting . We can similarly obtain other expressions for :
Remember that the trace of a matrix is the sum of its diagonal entries.
The proposition below shows a connection between the vec operator and the trace.
Proposition Let be a matrix and an matrix. Then,where denotes the trace of .
This is proved as follows:
Below you can find some exercises with explained solutions.
Let be a matrix, an matrix and an vector. Prove that
Since is a column vector it equal its vec and the vec of its transpose:
Let be a block matrix with blocks , and . Can you express in terms of the vec of the blocks?
When we vectorize we stack its columns vertically starting from the first column on the left and ending with the last column of the right. Therefore, we first stack all the columns of , then all the columns of and finally those of . Therefore,
Please cite as:
Taboga, Marco (2017). "Vec operator", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/vec-operator.
Most of the learning materials found on this website are now available in a traditional textbook format.