Search for probability and statistics terms on Statlect
StatLect

Almost sure convergence

by , PhD

This lecture introduces the concept of almost sure (a.s.) convergence, first for sequences of random variables and then for sequences of random vectors.

Table of Contents

What you need to know before starting

In order to understand this lecture, you should understand the concepts of:

We anyway quickly review both of these concepts below.

Weakening pointwise convergence

Almost sure convergence is defined by weakening the requirements for pointwise convergence.

Let [eq1] be a sequence of random variables defined on a sample space Omega.

Remember that [eq1] is pointwise convergent if and only if the sequence of real numbers [eq3] is convergent for all omega in Omega.

Achieving convergence for all omega in Omega is a very stringent requirement. We weaken it by requiring the convergence of [eq4] for a large enough subset of Omega, and not necessarily for all omega in Omega.

In particular, we require [eq5] to be a convergent sequence almost surely: if F is the set of all sample points omega for which the sequence [eq6] is convergent, its complement $F^{c}$ must be included in a zero-probability event, that is,[eq7]

In other words, almost sure convergence requires that the sequences [eq8] converge for all sample points omega in Omega, except, possibly, for a very small set $F^{c}$ of sample points.

The set $F^{c}$ is so small that must be included in a zero-probability event.

Definition for sequences of random variables

What we have said so far is summarized by the following definition.

Definition Let [eq1] be a sequence of random variables defined on a sample space Omega. We say that [eq1] is almost surely convergent to a random variable X defined on Omega if and only if the sequence of real numbers [eq11] converges to [eq12] almost surely, that is, if and only if there exists a zero-probability event E such that[eq13]

The variable X is called the almost sure limit of the sequence and convergence is indicated by[eq14]

Example

The following is an example of a sequence that converges almost surely.

Suppose that the sample space Omega is[eq15]

As discussed in the lecture on Zero-probability events, it is possible to build a probability measure $QTR{rm}{P}$ on Omega, such that $QTR{rm}{P}$ assigns to each sub-interval of $left[ 0,1
ight] $ a probability equal to its length:[eq16]

Remember that in this probability model all the sample points omega in Omega are assigned zero probability.

In other words, each sample point, when considered as an event, is a zero-probability event:[eq17]

Now, consider a sequence of random variables [eq1] defined as follows:[eq19]

When [eq20], the sequence of real numbers [eq8] converges to 0 because[eq22]

However, when $omega =0$, the sequence of real numbers [eq23] is not convergent to 0 because[eq24]

Define a constant random variable X as follows: [eq25]

We have that[eq26]

But [eq27] because [eq28]which means that the event[eq29]is a zero-probability event.

Therefore, the sequence [eq1] converges to X almost surely.

Note, however, that [eq1] does not converge pointwise to X because [eq32] does not converge to [eq12] for all omega in Omega.

How to generalize the definition to the multivariate case

Let [eq1] be a sequence of random vectors defined on a sample space Omega, where each random vector X_n has dimension Kx1.

Also in the case of random vectors, the concept of almost sure convergence is obtained from the concept of pointwise convergence by relaxing the assumption that the sequence [eq35] converges for all omega in Omega.

Remember that a sequence of real vectors [eq36] converges to a real vector [eq12] if and only if [eq38]where [eq39] denotes the Euclidean norm.

In the case of almost sure convergence, it is required that the sequence [eq40] converges for almost all omega (i.e., almost surely).

Definition for sequences of random vectors

Here is a formal definition for the multivariate case.

Definition Let [eq1] be a sequence of random vectors defined on a sample space Omega. We say that [eq1] is almost surely convergent to a random vector X defined on Omega if and only if the sequence of real vectors [eq43] converges to the real vector [eq12] almost surely, that is, if and only if there exists a zero-probability event E such that[eq13]

Also in the multivariate case, X is called the almost sure limit of the sequence and convergence is indicated by[eq14]

Relation between univariate and multivariate convergence

A sequence of random vectors is almost surely convergent if and only if all the sequences formed by their entries are almost surely convergent.

Proposition Let [eq1] be a sequence of random vectors defined on a sample space Omega. Denote by [eq48] the sequence of random variables obtained by taking the i-th entry of each random vector X_n. The sequence [eq1] converges almost surely to the random vector X if and only if [eq50] converges almost surely to the random variable $X_{ullet ,i}$ (the i-th entry of X) for each $i=1,ldots ,K$.

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Let the sample space be[eq51]

Sub-intervals of $left[ 0,1
ight] $ are assigned a probability equal to their length:[eq52]

Define a sequence of random variables [eq1] as follows:[eq54]

Define a random variable X as follows:[eq55]

Does the sequence [eq1] converge almost surely to X?

Solution

For a fixed sample point [eq57], the sequence of real numbers [eq58] has limit[eq59]

For $omega =1$, the sequence of real numbers [eq60] has limit[eq61]

Therefore, the sequence of random variables [eq1] does not converge pointwise to X because[eq63]for $omega =1$. However, the set of sample points omega such that [eq40] does not converge to [eq65] is a zero-probability event: [eq66]Therefore, the sequence [eq1] converges almost surely to X.

Exercise 2

Let [eq1] and [eq69] be two sequences of random variables defined on a sample space Omega.

Let X and Y be two random variables defined on Omega such that[eq70]

Prove that[eq71]

Solution

Denote by $F_{X}$ the set of sample points for which [eq72] converges to [eq12]:[eq74]The fact that [eq1] converges almost surely to X implies that[eq76]where [eq77].

Denote by $F_{Y}$ the set of sample points for which [eq78] converges to [eq79]:[eq80]The fact that [eq69] converges almost surely to Y implies that[eq82]where [eq83].

Now, denote by $F_{XY}$ the set of sample points for which [eq84] converges to [eq85]:[eq86]

Observe that if [eq87] then [eq88] converges to [eq89], because the sum of two sequences of real numbers is convergent if the two sequences are convergent. Therefore,[eq90]Taking the complement of both sides, we obtain[eq91]But [eq92]and as a consequence [eq93]. Thus, the set $F_{XY}^{c}$ of sample points omega such that [eq88] does not converge to [eq95] is included in the zero-probability event $E_{X}cup E_{Y}$, which means that[eq71]

Exercise 3

Let the sample space be[eq97]

Sub-intervals of $left[ 0,1
ight] $ are assigned a probability equal to their length:[eq52]

Define a sequence of random variables [eq1] as follows:[eq100]

Find an almost sure limit of the sequence.

Solution

If $omega =0$ or $omega =1$, then the sequence of real numbers [eq8] is not convergent:[eq102]

For [eq103], the sequence of real numbers [eq8] has limit[eq105]because for any omega we can find $n_{0}$ such that [eq106] for any $ngeq n_{0}$ (as a consequence [eq107] for any $ngeq n_{0}$).

Thus, the sequence of random variables [eq1] converges almost surely to the random variable X defined as[eq109]because the set of sample points omega such that [eq110] does not converge to [eq12] is a zero-probability event: [eq112]

How to cite

Please cite as:

Taboga, Marco (2021). "Almost sure convergence", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/asymptotic-theory/almost-sure-convergence.

The books

Most of the learning materials found on this website are now available in a traditional textbook format.