Search
STATLECT > Fundamentals of probability

Conditional expectation

Let X and Y be two random variables defined on the same sample space Omega. In this lecture we discuss how to compute the expected value of $X $ when we observe the realization of Y, i.e. when we receive the information that Y=y. The expected value of X conditional on the information that Y=y is called conditional expectation of X given Y=y. As in the case of the expected value, giving a completely rigorous definition of conditional expected value (or conditional expectation) requires some background in measure theory. To make things simpler, we do not give a completely rigorous definition in this lecture. We rather give an informal definition and we show how conditional expectation is computed.

Definition (informal)_ The conditional expected value (or conditional expectation) of a random variable X given Y=y is the weighted average of the values that X can take on, where each possible value is weighted by its respective conditional probability (conditional on the information that Y=y).

The expectation of a random variable X conditional on Y=y is denoted by [eq1] and it is often called the conditional mean of X given Y=y.

Conditional expectation of a discrete random variable

In the case in which [eq2] is a discrete random vector (as a consequence X is a discrete random variable), the conditional expectation of X given Y=y is computed as follows:[eq3]where [eq4] is the conditional probability mass function of X given Y=y and R_X is the support of X. Note that the above summation is not guaranteed to be well-defined or can be infinite. In case it is not well-defined, X does not possess a conditional expectation (conditional on Y=y).

Thus, the formula for computing the conditional expectation of a discrete random variable is a straightforward implementation of the informal definition of conditional expectation we have given above: the conditional expectation of a discrete variable X given Y=y is the weighted average of the values that X can take on (the elements of R_X), where each possible value x is weighted by its respective conditional probability [eq5] (conditional on Y=y).

Example_ Let the support of [eq6] be: [eq7]and its joint probability mass function be:[eq8]Let us compute the conditional probability mass function of X given $Y=0$. The support of Y is:[eq9]The marginal probability mass function of Y evaluated at $y=0$ is:[eq10]The support of X is:[eq11]Thus, the conditional probability mass function of X given $Y=0$ is:[eq12] The conditional expectation of X given $Y=0$ is:[eq13]

In the next subsections we will explain how to compute the conditional expectation of absolutely continuous random variables and of random variables that are neither discrete nor absolutely continuous: to calculate the conditional expectation of these variables, we will need to compute integrals instead of summations; we can think of these integrals as the limiting cases of the summation [eq14].

Conditional expectation of an absolutely continuous random variable

In the case in which [eq15] is an absolutely continuous random vector (as a consequence X is an absolutely continuous random variable), the conditional expectation of X given Y=y is computed as follows:[eq16]where [eq17] is the conditional probability density function of X given Y=y.

As in the discrete case, the above integral is not guaranteed to be well-defined or can be infinite. In case it is not well-defined, X does not possess a conditional expectation (conditional on Y=y).

As we anticipated in the previous subsection, this integral can safely be thought of as the limiting case of the summation [eq18] found in the discrete case, where [eq19] is the (infinitesimal) conditional probability of x and [eq20] can be thought of as a summation sign.

Example_ Let the support of [eq21] be: [eq22]and its joint probability density function be:[eq23]Let us compute the conditional probability density function of X given $Y=2 $. The support of Y is:[eq24]When [eq25], the marginal probability density function of Y is 0; when [eq26], the marginal probability density function is:[eq27]Thus, the marginal probability density function of Y is:[eq28]When evaluated at $y=2$, it is:[eq29]The support of X is:[eq30]Thus, the conditional probability density function of X given $Y=2$ is:[eq31] The conditional expectation of X given $Y=2$ is:[eq32]

Conditional expectation in general

The general formula for computing the conditional expectation of X given Y=y does not require [eq15] to be discrete or absolutely continuous, but it is applicable to any random vector:[eq34]where the integral is a Riemann-Stieltjes integral and [eq35] is the conditional distribution function of X given Y=y. Of course, as in the two special cases above, the above integral is not guaranteed to be well-defined or can be infinite. In case it is not well-defined, X does not possess a conditional expectation (conditional on Y=y).

Again, the above integral can safely be thought of as the limiting case of the summation [eq36] found in the discrete case, where [eq37] is the conditional probability of x and [eq38] can be thought of as a summation sign.

The above formula follows the same logic of the formula already introduced in the lecture on the expected value: [eq39]with the only difference that the unconditional distribution function [eq40] has now been replaced with the conditional distribution function [eq41]. The reader who feels unfamiliar with this formula can go back to the lecture entitled Expected value to read an intuitive introduction to the Riemann-Stieltjes integral and its use in probability theory.

More details

Properties of conditional expectation

From the above sections, it should be clear that the conditional expectation is computed exactly as the expected value, with the only difference that probabilities and probability densities are replaced by conditional probabilities and conditional probability densities. Therefore, the properties enjoyed by the expected value are, in general, also enjoyed by the conditional expectation. For example, two important properties enjoyed by the expected value that are also enjoyed by the conditional expectation are:

  1. Transformation theorem

  2. Linearity

by
About | Contacts | Privacy and terms of use | Sitemap