StatlectThe Digital Textbook
Index > Fundamentals of probability

Bayes' rule

Bayes' rule, named after the English mathematician Thomas Bayes, is a rule for computing conditional probabilities.

The rule

A formal statement of Bayes' rule follows.

Proposition Let A and $B$ be two events. Denote their probabilities by [eq1] and [eq2] and suppose that both [eq3] and [eq4]. Denote by [eq5] the conditional probability of A given $B$ and by [eq6] the conditional probability of $B$ given A. Bayes' rule states that[eq7]

Proof

By the conditional probability formula, we have that[eq8]and[eq9]The second formula can be re-arranged as follows:[eq10]which is then plugged into the first formula, so as to obtain[eq11]

The following example shows how Bayes' rule can be applied in a practical situation.

Example An HIV test gives a positive result with probability 98% when the patient is indeed affected by HIV, while it gives a negative result with 99% probability when the patient is not affected by HIV. If a patient is drawn at random from a population in which 0,1% of individuals are affected by HIV and he is found positive, what is the probability that he is indeed affected by HIV? In probabilistic terms, what we know about this problem can be formalized as follows:[eq12]Furthermore, the unconditional probability of being found positive can be derived using the law of total probability:[eq13]Therefore, Bayes' rule gives[eq14]Therefore, even if the test is conditionally very accurate, the unconditional probability of being affected by HIV when found positive is less than 10 per cent!

Terminology

The quantities involved in Bayes' rule[eq15]often take the following names:

  1. [eq16] is called prior probability or, simply, prior;

  2. [eq17] is called conditional probability or likelihood;

  3. [eq18] is called marginal probability;

  4. [eq19] is called posterior probability or, simply, posterior.

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

There are two urns containing colored balls. The first urn contains 50 red balls and 50 blue balls. The second urn contains 30 red balls and 70 blue balls. One of the two urns is randomly chosen (both urns have probability $50%$ of being chosen) and then a ball is drawn at random from one of the two urns. If a red ball is drawn, what is the probability that it comes from the first urn?

Solution

In probabilistic terms, what we know about this problem can be formalized as follows:[eq20]The unconditional probability of drawing a red ball can be derived using the law of total probability:[eq21]By using Bayes' rule, we obtain[eq22]

Exercise 2

An economics consulting firm has created a model to predict recessions. The model predicts a recession with probability 80% when a recession is indeed coming and with probability 10% when no recession is coming. The unconditional probability of falling into a recession is 20%. If the model predicts a recession, what is the probability that a recession will indeed come?

Solution

What we know about this problem can be formalized as follows:[eq23]The unconditional probability of predicting a recession can be derived by using the law of total probability:[eq24]Bayes' rule implies[eq25]

Exercise 3

Alice has two coins in her pocket, a fair coin (head on one side and tail on the other side) and a two-headed coin. She picks one at random from her pocket, tosses it and obtains head. What is the probability that she flipped the fair coin?

Solution

What we know about this problem can be formalized as follows:[eq26]The unconditional probability of obtaining head can be derived by using the law of total probability:[eq27]With Bayes' rule, we obtain[eq28]

The book

Most learning materials found on this website are now available in a traditional textbook format.