StatlectThe Digital Textbook
Index > Additional topics in probability theory

Sums of independent random variables

This lecture discusses how to derive the distribution of the sum of two independent random variables. We explain first how to derive the distribution function of the sum and then how to derive its probability mass function (if the summands are discrete) or its probability density function (if the summands are continuous).

Distribution function of a sum

The following proposition characterizes the distribution function of the sum in terms of the distribution functions of the two summands:

Proposition Let X and Y be two independent random variables and denote by [eq1] and [eq2] their distribution functions. Let:[eq3]and denote the distribution function of Z by [eq4]. The following holds:[eq5]or:[eq6]

Proof

The first formula is derived as follows:[eq7]The second formula is symmetric to the first.

Example Let X be a uniform random variable with support [eq8] and probability density function:[eq9]and Y another uniform random variable, independent of X, with support [eq10] and probability density function:[eq11]The distribution function of X is:[eq12]The distribution function of $Z=X+Y$ is:[eq13]There are four cases to consider:

  1. If [eq14], then:[eq15]

  2. If [eq16], then:[eq17]

  3. If [eq18], then:[eq19]

  4. If $z>2$, then:[eq20]

Combining these four possible cases, we obtain:[eq21]

Probability mass function of a sum

When the two summands are discrete random variables, the probability mass function of their sum can be derived as follows:

Proposition Let X and Y be two independent discrete random variables and denote by [eq22] and [eq23] their respective probability mass functions and by R_X and $R_{Y}$ their supports. Let:[eq3]and denote the probability mass function of Z by [eq25]. The following holds:[eq26]or:[eq27]

Proof

The first formula is derived as follows:[eq28]The second formula is symmetric to the first.

The two summations above are called convolutions (of two probability mass functions).

Example Let X be a discrete random variable with support [eq29] and probability mass function:[eq30]and Y another discrete random variable, independent of X, with support [eq31] and probability mass function:[eq32]Define[eq33]Its support is: [eq34]The probability mass function of Z, evaluated at $z=0$ is:[eq35]Evaluated at $z=1$, it is:[eq36]Evaluated at $z=2$, it is:[eq37]Therefore, the probability mass function of Z is:[eq38]

Probability density function of a sum

When the two summands are absolutely continuous random variables, the probability density function of their sum can be derived as follows:

Proposition Let X and Y be two independent absolutely continuous random variables and denote by [eq39] and [eq40] their respective probability density functions. Let:[eq3]and denote the probability density function of Z by [eq42]. The following holds:[eq43]or:[eq44]

Proof

The distribution function of a sum of independent variables is:[eq5]Differentiating both sides and using the fact that the density function is the derivative of the distribution function, we obtain:[eq46]The second formula is symmetric to the first.

The two integrals above are called convolutions (of two probability density functions).

Example Let X be an exponential random variable with support [eq47] and probability density function:[eq48]and Y another exponential random variable, independent of X, with support [eq49] and probability density function:[eq50]Define: [eq33]The support of Z is:[eq52]When $zin R_{Z}$, the probability density function of Z is:[eq53]Therefore, the probability density function of Z is:[eq54]

More details

Sum of n independent random variables

We have discussed above how to derive the distribution of the sum of two independent random variables. How do we derive the distribution of the sum of more than two mutually independent random variables? Suppose X_1, X_2, ..., X_n are n mutually independent random variables and let Z be their sum:[eq55]The distribution of Z can be derived recursively, using the results for sums of two random variables given above:

  1. first, define:[eq56]and compute the distribution of $Y_{2}$;

  2. then, define:[eq57]and compute the distribution of $Y_{3}$;

  3. and so on, until the distribution of Z can be computed from:[eq58]

Solved exercises

Below you can find some exercises with explained solutions:

  1. Exercise set 1

The book

Most learning materials found on this website are now available in a traditional textbook format.