The conditional expectation (or conditional mean, or conditional expected value) of a random variable is the expected value of the random variable itself, computed with respect to its conditional probability distribution.

As in the case of the expected value, a completely rigorous definition of conditional expected value requires a complicated mathematical apparatus. To make things simpler, we do not give a completely rigorous definition in this lecture. We rather give an informal definition and we show how conditional expectation can be computed. In particular, we discuss how to compute the expected value of a random variable when we observe the realization of another random variable , that is, when we receive the information that . The expected value of conditional on the information that is called conditional expectation of given .

The following informal definition is very similar to the definition of expected value we have given in the lecture entitled Expected value.

Definition (informal)
Let
and
be two random variables. The **conditional
expectation** of
given
is the weighted average of the values that
can take on, where each possible value is weighted by its respective
conditional probability (conditional on the information that
).

The expectation of a random variable conditional on is denoted by

In the case in which and are two discrete random variables and, considered together, they form a discrete random vector, the formula for computing the conditional expectation of given is a straightforward implementation of the above informal definition of conditional expectation: the weights of the average are given by the conditional probability mass function of .

Definition Let and be two discrete random variables. Let be the support of and let be the conditional probability mass function of given . The conditional expectation of given isprovided that

If you do not understand the symbol and the finiteness condition above (absolute summability) go back to the lecture entitled Expected value, where they are explained.

Example Let the support of the random vector be and its joint probability mass function beLet us compute the conditional probability mass function of given . The marginal probability mass function of evaluated at isThe support of isThus, the conditional probability mass function of given isThe conditional expectation of given is

When and are continuous random variables, forming an continuous random vector, the formula for computing the conditional expectation of given involves an integral, which can be thought of as the limiting case of the summation found in the discrete case above.

Definition Let and be two continuous random variables. Let be the support of and let be the conditional probability density function of given . The conditional expectation of given isprovided that

If you do not understand why an integration is required and why the finiteness condition above (absolute integrability) is imposed, you can find an explanation in the lecture entitled Expected value.

Example Let the support of the random vector be and its joint probability density function beLet us compute the conditional probability density function of given . The support of isWhen , the marginal probability density function of is ; when , the marginal probability density function isThus, the marginal probability density function of isWhen evaluated at , it isThe support of isThus, the conditional probability density function of given isThe conditional expectation of given is

The general formula for computing the conditional expectation of given does not require that the two variables form a discrete or a continuous random vector, but it is applicable to any random vector.

Definition Let be the conditional distribution function of given . The conditional expectation of given iswhere the integral is a Riemann-Stieltjes integral and the expected value exists and is well-defined only as long as the integral is well-defined.

The above formula follows the same logic of the formula for the expected value with the only difference that the unconditional distribution function has now been replaced with the conditional distribution function . The reader who feels unfamiliar with this formula can go back to the lecture entitled Expected value and read an intuitive introduction to the Riemann-Stieltjes integral and its use in probability theory.

The following subsections contain more details about conditional expectation.

From the above sections, it should be clear that the conditional expectation is computed exactly as the expected value, with the only difference that probabilities and probability densities are replaced by conditional probabilities and conditional probability densities. Therefore, the properties enjoyed by the expected value, such as linearity, are also enjoyed by the conditional expectation. For an exposition of the properties of the expected value, you can go to the lecture entitled Properties of the expected value.

Quite obviously, before knowing the realization of , the conditional expectation of given is unknown and can itself be regarded as a random variable. We denote it by . In other words, is a random variable such that its realization equals when is the realization of .

This random variable satisfies a very important property, known as **law
of iterated
expectations**:

Proof

For discrete random variables this is proved as follows:For continuous random variables the proof is analogous:

Below you can find some exercises with explained solutions.

Let be a discrete random vector with support and joint probability mass functionWhat is the conditional expectation of given ?

Solution

Let us compute the conditional probability mass function of given . The marginal probability mass function of evaluated at isThe support of isThus, the conditional probability mass function of given isThe conditional expectation of given is

Suppose is a continuous random vector with support and joint probability density functionCompute the expected value of conditional on .

Solution

We first need to compute the conditional probability density function of given , by using the formula Note that, by using indicator functions, we can writeThe marginal probability density function is obtained by marginalizing the joint density:When evaluated at , it isFurthermore, Thus, the conditional probability density function of given isThe conditional expectation of given is

Let and be two random variables. Remember that the variance of can be computed asIn a similar manner, the conditional variance of , given , can be defined asUse the law of iterated expectations to prove that

Solution

This is proved as follows:

where: in step we have used the law of iterated expectations; in step we have used the formula in step we have used the linearity of the expected value; in step we have used the formula

The book

Most of the learning materials found on this website are now available in a traditional textbook format.

Featured pages

- Point estimation
- Multivariate normal distribution
- Hypothesis testing
- Maximum likelihood
- Bayes rule
- Gamma function

Explore

Main sections

- Mathematical tools
- Fundamentals of probability
- Probability distributions
- Asymptotic theory
- Fundamentals of statistics
- Glossary

About

Glossary entries

- Alternative hypothesis
- Precision matrix
- Power function
- Distribution function
- Mean squared error
- Type II error

Share