Search
STATLECT > Additional topics in probability theory

Sums of independent random variables

Let X and Y be two independent random variables. In this lecture we discuss how to derive the distribution of their sum, i.e. the distribution of the random variable Z defined as:[eq1]We first characterize the distribution function of Z; then, we characterize the probability mass function and the probability density function for the two special cases in which Z is respectively discrete or absolutely continuous.

Sum of two independent random variables - Distribution function

The following proposition characterizes the distribution function of the sum in terms of the distribution functions of the two summands:

Proposition (Distribution function of a sum)_ Let X and Y be two independent random variables and denote by [eq2] and [eq3] their respective distribution functions. Let:[eq4]and denote the distribution function of Z by [eq5]. The following holds:[eq6]or:[eq7]

nav_button Proof

The first formula is derived as follows:[eq8]The second formula is symmetric to the first.

Example_ Let X be a uniform random variable with support [eq9] and probability density function:[eq10]and Y another uniform random variable, independent of X, with support [eq11] and probability density function:[eq12]The distribution function of X is:[eq13]The distribution function of $Z=X+Y$ is:[eq14]There are four cases to consider:

  1. If [eq15], then:[eq16]

  2. If [eq17], then:[eq18]

  3. If [eq19], then:[eq20]

  4. If $z>2$, then:[eq21]

Therefore, combining these four possible cases, we obtain:[eq22]

Sum of two discrete independent random variables - Probability mass function

When the two summands are discrete random variables, the probability mass function of their sum can be derived as follows:

Proposition (Probability mass function of a sum)_ Let X and Y be two independent discrete random variables and denote by [eq23] and [eq24] their respective probability mass functions and by R_X and $R_{Y}$ their supports. Let:[eq4]and denote the probability mass function of Z by [eq26]. The following holds:[eq27]or:[eq28]

nav_button Proof

The first formula is derived as follows:[eq29]The second formula is symmetric to the first.

The two summations above are called convolutions (of two probability mass functions).

Example_ Let X be a discrete random variable with support [eq30] and probability mass function:[eq31]and Y another discrete random variable, independent of X, with support [eq32] and probability mass function:[eq33]Define[eq4]Its support is: [eq35]The probability mass function of Z, evaluated at $z=0$ is:[eq36]Evaluated at $z=1$, it is:[eq37]Evaluated at $z=2$, it is:[eq38]Therefore, the probability mass function of Z is:[eq39]

Sum of two absolutely continuous random variables - Probability density function

When the two summands are absolutely continuous random variables, the probability density function of their sum can be derived as follows:

Proposition (Probability density of a sum)_ Let X and Y be two independent absolutely continuous random variables and denote by [eq40] and [eq41] their respective probability density functions. Let:[eq4]and denote the probability density function of Z by [eq43]. The following holds:[eq44]or:[eq45]

nav_button Proof

The distribution function of a sum of independent variables is:[eq6]Differentiating both sides (and using the fact that the density function is the derivative of the distribution function), we obtain:[eq47]The second formula is symmetric to the first.

The two integrals above are called convolutions (of two probability density functions).

Example_ Let X be an exponential random variable with support [eq48] and probability density function:[eq49]and Y another exponential random variable, independent of X, with support [eq50] and probability density function:[eq51]Define: [eq4]The support of Z is:[eq53]When $zin R_{Z}$, the probability density function of Z is:[eq54]Therefore, the probability density function of Z is:[eq55]

More details

Sum of n independent random variables

We have discussed above how to derive the distribution of the sum of two independent random variables. How do we derive the distribution of the sum of more than two mutually independent random variables? Suppose X_1, X_2, ..., X_n are n mutually independent random variables and let Z be their sum:[eq56]The distribution of Z can be derived recursively, using the results for sums of two random variables given above:

  1. first, define:[eq57]and compute the distribution of $Y_{2}$;

  2. then, define:[eq58]and compute the distribution of $Y_{3}$;

  3. and so on, until the distribution of Z can be computed from:[eq59]

by
About | Contacts | Privacy and terms of use | Sitemap