StatlectThe Digital Textbook

# Covariance matrix

Let be a random vector. The covariance matrix of , or variance-covariance matrix of , is denoted by . It is defined as follows:provided the above expected values exist and are well-defined.

It is a multivariate generalization of the definition of variance for a scalar random variable :

## Structure

Let , ..., denote the components of the vector . From the definition of , it can easily be seen that is a matrix with the following structure:

Therefore, the covariance matrix of is a square matrix whose generic -th entry is equal to the covariance between and .

Since when , the diagonal entries of the covariance matrix are equal to the variances of the individual components of .

Example Suppose is a random vector with components and . LetBy the symmetry of covariance, it must also be Therefore, the covariance matrix of is

## Formula for computing the covariance matrix

The covariance matrix of a random vector can be computed as follows:

Proof

The above formula can be derived as follows:

This formula also makes clear that the covariance matrix exists and is well-defined only as long as the vector of expected values and the matrix of second cross-moments exist and are well-defined.

## More details

The following subsections contain more details about the covariance matrix.

Let be a constant vector and let be a random vector. Then,

Proof

This is a consequence of the fact that (by linearity of the expected value):

### Multiplication by constant matrices

Let be a constant matrix and let be a random vector. Then,

Proof

This is easily proved using the fact that (by linearity of the expected value):

### Linear transformations

Let be a constant vector, be a constant matrix and a random vector. Then, combining the two properties above, one obtains

### Symmetry

The covariance matrix is a symmetric matrix, i.e., it is equal to its transpose:

### Semi-positive definiteness

The covariance matrix is a positive-semidefinite matrix, i.e. for any vector :This is easily proved using the Multiplication by constant matrices property above:where the last inequality follows from the fact that variance is always positive.

### Covariance between linear transformations

Let and be two constant vectors and a random vector. Then, the covariance between the two linear transformations and can be expressed as a function of the covariance matrix:

Proof

This can be proved as follows:

### Cross-covariance

The term covariance matrix is sometimes also used to refer to the matrix of covariances between the elements of two vectors.

Let be a random vector and be a random vector. The covariance matrix between and , or cross-covariance between and is denoted by . It is defined as follows:provided the above expected values exist and are well-defined.

It is a multivariate generalization of the definition of covariance between two scalar random variables.

Let , ..., denote the components of the vector and , ..., denote the components of the vector . From the definition of , it can easily be seen that is a matrix with the following structure:Note that is not the same as . In fact, is a matrix equal to the transpose of :

## Solved exercises

Below you can find some exercises with explained solutions:

1. Exercise set 1 (use of the covariance matrix)

The book

Most learning materials found on this website are now available in a traditional textbook format.