StatlectThe Digital Textbook
Index > Probability distributions > Normal distribution

Linear combinations of normal random variables

One property that makes the normal distribution extremely tractable from an analytical viewpoint is its closure under linear combinations: the linear combination of two independent random variables having a normal distribution also has a normal distribution. The following sections present a multivariate generalization of this elementary property and then discuss some special cases.

Linear transformation of a multivariate normal random vector

A linear transformation of a multivariate normal random vector also has a multivariate normal distribution, as illustrated by the following:

Proposition Let X be a Kx1 multivariate normal random vector with mean mu and covariance matrix V. Let A be an $L	imes 1$ real vector and $B$ an $L	imes K$ full-rank real matrix. Then the $L	imes 1$ random vector Y defined by:[eq1]has a multivariate normal distribution with mean[eq2]and covariance matrix[eq3]

Proof

This is proved using the formula for the joint moment generating function of the linear transformation of a random vector. The joint moment generating function of X is [eq4]Therefore, the joint moment generating function of Y is:[eq5]which is the moment generating function of a multivariate normal distribution with mean $A+Bmu $ and covariance matrix $BVB^{intercal }$. Note that $BVB^{intercal }$ needs to be positive definite in order to be the covariance matrix of a proper multivariate normal distribution, but this is implied by the assumption that $B$ is full-rank. Therefore, Y has a multivariate normal distribution with mean $A+Bmu $ and covariance matrix $BVB^{intercal }$, because two random vectors have the same distribution when they have the same joint moment generating function.

The following examples present some important special cases of the above property.

Example 1 - Sum of two independent normal random variables

The sum of two independent normal random variables has a normal distribution, as stated in the following:

Example Let X_1 be a random variable having a normal distribution with mean $mu _{1}$ and variance $sigma _{1}^{2}$. Let X_2 be a random variable, independent of X_1, having a normal distribution with mean $mu _{2}$ and variance $sigma _{2}^{2}$. Then, the random variable Y defined as:[eq6]has a normal distribution with mean [eq7]and variance [eq8]

Proof

First of all, we need to use the fact that mutually independent normal random variables are jointly normal: the $2	imes 1$ random vector X defined as[eq9]has a multivariate normal distribution with mean [eq10]and covariance matrix [eq11]

We can write:[eq12]where[eq13]

Therefore, according to the above proposition on linear transformations, Y has a normal distribution with mean:[eq14]and variance:[eq15]

Example 2 - Sum of more than two mutually independent normal random variables

The sum of more than two independent normal random variables also has a normal distribution, as shown in the following:

Example Let [eq16] be K mutually independent normal random variables, having means [eq17] and variances [eq18]. Then, the random variable Y defined as:[eq19]has a normal distribution with mean [eq20]and variance [eq21]

Proof

This can be obtained, either generalizing the proof of the proposition in Example 1, or using the proposition in Example 1 recursively (starting from the first two components of X, then adding the third one and so on).

Example 3 - Linear combinations of mutually independent normal random variables

The properties illustrated in the previous two examples can be further generalized to linear combinations of K mutually independent normal random variables:

Example Let [eq22] be K mutually independent normal random variables, having means [eq23] and variances [eq24]. Let [eq25] be K constants. Then, the random variable Y defined as:[eq26]has a normal distribution with mean [eq27]and variance [eq28]

Proof

First of all, we need to use the fact that mutually independent normal random variables are jointly normal: the Kx1 random vector X defined as[eq29]has a multivariate normal distribution with mean [eq30]and covariance matrix [eq31]

We can write:[eq32]where[eq33]

Therefore, according to the above proposition on linear transformations, Y has a (multivariate) normal distribution with mean:[eq34]and variance:[eq35]

Example 4 - Linear transformation of a normal random variable

A special case of the above proposition obtains when X has dimension $1	imes 1$ (i.e. it is a random variable):

Example Let X be a normal random variable with mean mu and variance sigma^2. Let a and $b$ be two constants (with $b
eq 0$). Then the random variable Y defined by:[eq36]has a normal distribution with mean[eq37]and variance[eq38]

Proof

This is just a special case ($K=1$) of the above proposition on linear transformations.

Example 5 - Linear combinations of mutually independent normal random vectors

The property illustrated in Example 3 can be generalized to linear combinations of mutually independent normal random vectors.

Example Let [eq39] be n mutually independent Kx1 normal random vectors, having means [eq40] and covariance matrices [eq41]. Let [eq42] be n real $L	imes K$ full-rank matrices. Then, the $L	imes 1$ random vector Y defined as:[eq43]has a normal distribution with mean [eq44]and covariance matrix [eq45]

Proof

This is a consequence of the fact that mutually independent normal random vectors are jointly normal: the $Kn	imes 1$ random vector X defined as[eq46]has a multivariate normal distribution with mean [eq47]and covariance matrix [eq48]

Therefore, we can apply the above proposition on linear transformations to the vector X.

Solved exercises

Below you can find some exercises with explained solutions:

  1. Exercise set 1

The book

Most learning materials found on this website are now available in a traditional textbook format.