StatlectThe Digital Textbook
Index > Fundamentals of probability

Zero-probability events

The notion of a zero-probability event plays a special role in probability theory and statistics, because it underpins the important concepts of almost sure property and almost sure event. This lecture defines zero-probability events and discusses some counterintuitive aspects of their apparently simple definition, in particular the fact that a zero-probability event is not an event that never happens: there are common probabilistic settings where zero-probability events do happen all the time! After discussing this matter, the concepts of almost sure property and almost sure event are introduced.

Definition and discussion

Tautologically, zero-probability events are events whose probability is equal to zero:

Definition Let E be an event and denote its probability by [eq1]. We say that E is a zero-probability event if and only if:[eq2]

Despite the simplicity of this definition, there are some features of zero-probability events that might seem paradoxical. We illustrate these features with the following example.

Example Consider a probabilistic experiment whose set of possible outcomes, called sample space and denoted by Omega, is the unit interval:[eq3]It is possible to assign probabilities in such a way that each sub-interval has probability equal to its length:[eq4]The proof that such an assignment of probabilities can be consistently performed is beyond the scope of this example, but you can find it in any elementary measure theory book (e.g. Williams - 1991). As a direct consequence of this assignment, all the possible outcomes omega in Omega have zero probability:[eq5] Stated differently, every possible outcome is a zero-probability event. This might seem counterintuitive. In everyday language, a zero-probability event is an event that never happens. However, this example illustrates that a zero-probability event can indeed happen. Since the sample space provides an exhaustive description of the possible outcomes, one and only one of the sample points omega in Omega will be the realized outcome. But we have just demonstrated that all the sample points are zero-probability events: as a consequence, the realized outcome can only be a zero-probability event. Another apparently paradoxical aspect of this probability model is that the sample space Omega can be obtained as the union of disjoint zero-probability events:[eq6]where each omega in Omega is a zero-probability event and all events in the union are disjoint. If we forgot that the additivity property of probability applies only to countable collections of subsets, we would mistakenly deduce that:[eq7]and we would come to a contradiction: [eq8], when, by the properties of probability, it should be [eq9]. Of course, the fallacy in such an argument is that Omega is not a countable set and, hence, the additivity property cannot be used.

The main lesson to be taken from this example is that a zero-probability event is not an event that never happens: in some probability models, where the sample space is not countable, zero-probability events do happen all the time!

Almost sure and almost surely

Zero-probability events are of paramount importance in probability and statistics. Often, we want to prove that some property is almost always satisfied, or something happens almost always. "Almost always" means that the property is satisfied for all sample points, except possibly for a negligible set of sample points. The concept of zero-probability event is used to determine which sets are negligible: if a set is included in a zero-probability event, then it is negligible.

Definition Let $Phi $ be some property that a sample point omega in Omega can either satisfy or not satisfy. Let F be the set of all sample points that satisfy the property:[eq10]Denote its complement (the set of all points not satisfying property $Phi $) by $F^{c}$. Property $Phi $ is said to be almost sure if there exists a zero-probability event E such that $F^{c}subseteq E$.

An almost sure property is said to hold almost surely (often abbreviated as a.s.). Sometimes, an almost sure property is also said to hold with probability one (abbreviated w.p.1).

Almost sure events

Remember (see the lecture entitled Probability) that some subsets of the sample space may not be considered events. The above definition of almost sure property allows us to consider also sets F that are not, strictly speaking, events. However, in the case in which F is an event, F is called an almost sure event and we say that F happens almost surely. Furthermore, since there exists an event E such that $F^{c}subseteq E$ and [eq11], we can apply the monotonicity of probability:[eq12]which in turn implies [eq13]. Finally, recalling the formula for the probability of a complement:[eq14]Thus, an almost sure event is an event that happens with probability 1.

Example Consider the sample space [eq15] and the assignment of probabilities introduced in the previous example: [eq4]We want to prove that the event[eq17]is a zero-probability event. Since the set of rational numbers is countable and E is a subset of the set of rational numbers, E is countable. This implies that the elements of E can be arranged into a sequence:[eq18]Furthermore, E can be written as a countable union:[eq19]Applying the countable additivity property of probability, we obtain:[eq20]since [eq21] for every n. Therefore, E is a zero-probability event. This might seem surprising: in this probability model there are also zero-probability events comprising infinitely many sample points! It can also easily be proved that the event[eq22]is an almost sure event. In fact:[eq23]and applying the formula for the probability of a complement:[eq24]

References

Williams, D. (1991) Probability with martingales, Cambridge University Press.

The book

Most learning materials found on this website are now available in a traditional textbook format.