This lecture discusses how to derive the marginal and conditional distributions of one or more entries of a multivariate normal vector.
Table of contents
A random vector is multivariate normal if its joint probability density function iswhere:
is a mean vector;
is a covariance matrix.
We partition into two sub-vectors and such that
The sub-vectors and have dimensions and respectively. Moreover, .
We partition the mean vector and the covariance matrix as follows:and
where:
is the mean of ;
is the mean of ;
is the covariance matrix of ;
is the covariance matrix of ;
is the cross-covariance between and .
The following proposition states that the marginal distributions of the two sub-vectors are also multivariate normal.
Proposition Both and have a multivariate normal distribution:
The random vector can be written as a linear transformation of :where is a matrix whose entries are either zero or one. Thus, has a multivariate normal distribution because it is a linear transformation of the multivariate normal random vector and multivariate normality is preserved by linear transformations (see the lecture on Linear combinations of normal random variables). Also has a multivariate normal distribution because it can be written as a linear transformation of :where is a matrix whose entries are either zero or one.
The following proposition states a necessary and sufficient condition for the independence of the two sub-vectors.
Proposition and are independent if and only if .
and are independent if and only if their joint moment generating function is equal to the product of their individual moment generating functions (see the lecture entitled Joint moment generating function). Since is multivariate normal, its joint moment generating function isThe joint moment generating function of isThe joint moment generating function of and , which is just the joint moment generating function of , isfrom which it is obvious that if and only if .
In order to derive the conditional distributions, we are going to rely on the following results, demonstrated in the lecture on Schur complements.
Proposition Let be invertible. Let be the Schur complement of in , defined asIf is invertible, then is invertible and
Proposition Let be invertible. Let be the Schur complement of in , defined asIf is invertible, then is invertible and
We will also need the following results on the determinant of a block matrix.
Proposition If is invertible, then
Proposition If is invertible, then
Another important result that we are going to use concerns the factorization of joint density functions.
Write the joint density of the multivariate normal vector as
Suppose that we are able to find a factorizationsuch that is a valid probability density function every time that we fix and we see as a function of .
Then, where:
is the conditional probability density function of given ;
is the marginal density of .
Similarly, if we find a factorization such that is a valid probability density function every time that we fix and we see as a function of , then
The blocks of the inverse of the covariance matrix (known as precision matrix) are denoted as follows:
We are now ready to derive the conditional distributions.
Proposition Suppose that and its Schur complement in are invertible. Then, conditional on , the vector has a multivariate normal distribution with meanand covariance matrix
First, defineand note thatwhere: in step we have used the partition of the precision matrix ; in step we have used the formulae for the blocks of the precision matrix based on the Schur complements; in step we have definedandWe can now factorize the joint density of and :where is the density of a multivariate normal vector with mean and covariance matrix , and is the density of a multivariate normal vector with mean and covariance matrix .
Proposition Suppose that and its Schur complement in are invertible. Then, conditional on , the vector has a multivariate normal distribution with meanand covariance matrix
Analogous to the previous proof.
Please cite as:
Taboga, Marco (2021). "Marginal and conditional distributions of a multivariate normal vector", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/probability-distributions/multivariate-normal-distribution-partitioning.
Most of the learning materials found on this website are now available in a traditional textbook format.