A conditional probability is a probability assigned to an event after receiving some information about other relevant events.
Table of contents
The aims of this lecture are to:
introduce the concept of conditional probability in a mathematically sound manner;
provide two proofs of the formula used to compute conditional probabilities;
discuss some of the subtleties involved in the formula, including potential division-by-zero problems.
Let be a sample space and let denote the probability assigned to the events .
Suppose that, after assigning probabilities to the events in , we receive new information about the things that will happen (the possible outcomes).
In particular, suppose that we are told that the realized outcome will belong to a set .
How should we revise the probabilities assigned to the events in , to properly take the new information into account?
The answer to this question is provided by the conditional probability , the revised probability assigned to an event after learning that the realized outcome will be an element of .
Despite being an intuitive concept, conditional probability is quite difficult to define in a mathematically rigorous way.
We take a gradual approach in this lecture:
we first discuss conditional probability for the very special case in which all the sample points are equally likely;
we then give a more general definition;
finally, we refer the reader to other lectures where conditional probability is defined in even more abstract ways.
Suppose that a sample space has a finite number of sample points:
Suppose also that each sample point is assigned the same probability:
In such a simple space, the probability of a generic event is obtained aswhere denotes the cardinality of a set, that is, the number of its elements.
In other words, the probability of an event is obtained in two steps:
counting the number of "cases that are favorable to the event ", that is, the number of elements belonging to ;
dividing the number thus obtained by the number of "all possible cases", that is, the number of elements belonging to .
For example, if , then
When we learn that the realized outcome will belong to a set , we still apply the rule
However, the number of all possible cases is now equal to the number of elements of because only the outcomes belonging to are still possible.
Furthermore, the number of favorable cases is now equal to the number of elements of because the outcomes in are no longer possible.
As a consequence,
By dividing numerator and denominator by , we obtain
Therefore, when all sample points are equally likely, conditional probabilities are computed as
Let us make an example.
Suppose that we toss a die. Six numbers (from to can appear face up, but we do not yet know which one of them will appear.
The sample space is
Each of the six numbers is a sample point and is assigned probability .
Define the event as follows:where the event could be described as "an odd number appears face up".
Now define the event as follows:where the event could be described as "a number greater than three appears face up".
The probability of isSuppose we are told that the realized outcome will belong to . How do we have to revise our assessment of the probability of the event , according to the rules of conditional probability?
First of all, we need to compute the probability of the event :
Then, the conditional probability of given is
In the next section, we will show that the conditional probability formulais valid also for more general cases (i.e., when the sample points are not all equally likely).
However, this formula already allows us to understand why defining conditional probability is a challenging task.
In the conditional probability formula, a division by is performed. This division is impossible when is a zero-probability event (i.e., ).
If we want to be able to define also when , then we need to give a more complicated definition of conditional probability. We will return to this point later.
In this section we give a more general definition of conditional probability, by taking an axiomatic approach.
First, we list the mathematical properties that we would like conditional probability to satisfy.
Then, we prove that the conditional probability formula introduced above satisfies these properties.
The conditional probability is required to satisfy the following properties:
Probability measure. has to satisfy all the properties of a probability measure.
Sure thing. .
Impossible events. If (, the complement of with respect to , is the set of all elements of that do not belong to ), then .
Constant likelihood ratios on . If , and , then
These properties are very intuitive:
Property 1 requires that also conditional probability measures satisfy the fundamental properties that any other probability measure needs to satisfy.
Property 2 says that the probability of a sure thing must be : since we know that only things belonging to the set can happen, then the probability of must be .
Property 3 says that the probability of an impossible thing must be : since we know that things not belonging to the set will not happen, then the probability of the events that are disjoint from must be .
Property 4 is a bit more complex: it says that if is - say - two times more likely than before receiving the information , then remains two times more likely than , also after receiving the information because all the things in and remain possible (can still happen) and, hence, there is no reason to expect that the ratio of their likelihoods changes.
When the above properties are satisfied, then the conditional probability formula can be used.
Proposition Whenever , satisfies the four above properties if and only if
We first show thatsatisfies the four properties whenever . As far as property 1) is concerned, we have to check that the three requirements for a probability measure are satisfied. The first requirement for a probability measure is that . Since , by the monotonicity of probability we have thatHence,Furthermore, since and , also The second requirement for a probability measure is that . This is satisfied becauseThe third requirement for a probability measure is that for any sequence of disjoint sets the following holds:But,so that also the third requirement is satisfied. Property 2) is trivially satisfied:Property 3) is verified because, if , thenProperty 4) is verified because, if , and , thenSo, the "if" part has been proved. Now we prove the "only if" part. We prove it by contradiction. Suppose there exist another conditional probability that satisfies the four properties. Then, there exists an event , such thatIt can not be that , otherwise we would havewhich would be a contradiction, since if was a conditional probability it would satisfy:If is not a subset of then implies also becauseandbut this would also lead to a contradiction because .
In the previous section we have generalized the concept of conditional probability. However, we have not been able to define the conditional probability for the case in which . This case is discussed in the lectures:
We end this lecture by stating an important formula that allows us to write the probability of an event as a weighted sum of conditional probabilities.
Let , ..., be events having the following characteristics:
they are mutually disjoint: whenever ;
they cover all the sample space:
they have strictly positive probability: for any .
The events , ..., are called a partition of .
The law of total probability states that, for any event , the following holds:which can, of course, also be written as
The law of total probability is proved as follows:
Some solved exercises on conditional probability can be found below.
Consider a sample space comprising three possible outcomes , , :
Suppose the three possible outcomes are assigned the following probabilities:
Define the events
and denote by the complement of .
Compute , the conditional probability of given .
We need to use the conditional probability formula
The numerator isand the denominator is
As a consequence,
Consider a sample space comprising four possible outcomes , , , :
Suppose the four possible outcomes are assigned the following probabilities:
Define two events
Compute , the conditional probability of given .
We need to use the formula
Butwhile, by using additivity, we obtain
Therefore,
The Census Bureau has estimated the following survival probabilities for men:
probability that a man lives at least 70 years: 80%;
probability that a man lives at least 80 years: 50%.
What is the conditional probability that a man lives at least 80 years given that he has just celebrated his 70th birthday?
Given an hypothetical sample space , define the two events
We need to find the following conditional probability:
The denominator is known:
As far as the numerator is concerned, note that (if you live at least 80 years then you also live at least 70 years). But implies
Therefore,
Thus,
Please cite as:
Taboga, Marco (2021). "Conditional probability", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/fundamentals-of-probability/conditional-probability.
Most of the learning materials found on this website are now available in a traditional textbook format.