This lecture introduces the concept of almost sure (a.s.) convergence, first for sequences of random variables and then for sequences of random vectors.
Table of contents
In order to understand this lecture, you should understand the concepts of:
almost sure property and almost sure event, explained in the lecture on Zero-probability events;
pointwise convergence of a sequence of random variables, explained in the lecture on Pointwise convergence.
We anyway quickly review both of these concepts below.
Almost sure convergence is defined by weakening the requirements for pointwise convergence.
Let be a sequence of random variables defined on a sample space .
Remember that is pointwise convergent if and only if the sequence of real numbers is convergent for all .
Achieving convergence for all is a very stringent requirement. We weaken it by requiring the convergence of for a large enough subset of , and not necessarily for all .
In particular, we require to be a convergent sequence almost surely: if is the set of all sample points for which the sequence is convergent, its complement must be included in a zero-probability event, that is,
In other words, almost sure convergence requires that the sequences converge for all sample points , except, possibly, for a very small set of sample points.
The set is so small that must be included in a zero-probability event.
What we have said so far is summarized by the following definition.
Definition Let be a sequence of random variables defined on a sample space . We say that is almost surely convergent to a random variable defined on if and only if the sequence of real numbers converges to almost surely, that is, if and only if there exists a zero-probability event such that
The variable is called the almost sure limit of the sequence and convergence is indicated by
The following is an example of a sequence that converges almost surely.
Suppose that the sample space is
As discussed in the lecture on Zero-probability events, it is possible to build a probability measure on , such that assigns to each sub-interval of a probability equal to its length:
Remember that in this probability model all the sample points are assigned zero probability.
In other words, each sample point, when considered as an event, is a zero-probability event:
Now, consider a sequence of random variables defined as follows:
When , the sequence of real numbers converges to because
However, when , the sequence of real numbers is not convergent to because
Define a constant random variable as follows:
We have that
But because which means that the eventis a zero-probability event.
Therefore, the sequence converges to almost surely.
Note, however, that does not converge pointwise to because does not converge to for all .
Let be a sequence of random vectors defined on a sample space , where each random vector has dimension .
Also in the case of random vectors, the concept of almost sure convergence is obtained from the concept of pointwise convergence by relaxing the assumption that the sequence converges for all .
Remember that a sequence of real vectors converges to a real vector if and only if where denotes the Euclidean norm.
In the case of almost sure convergence, it is required that the sequence converges for almost all (i.e., almost surely).
Here is a formal definition for the multivariate case.
Definition Let be a sequence of random vectors defined on a sample space . We say that is almost surely convergent to a random vector defined on if and only if the sequence of real vectors converges to the real vector almost surely, that is, if and only if there exists a zero-probability event such that
Also in the multivariate case, is called the almost sure limit of the sequence and convergence is indicated by
A sequence of random vectors is almost surely convergent if and only if all the sequences formed by their entries are almost surely convergent.
Proposition Let be a sequence of random vectors defined on a sample space . Denote by the sequence of random variables obtained by taking the -th entry of each random vector . The sequence converges almost surely to the random vector if and only if converges almost surely to the random variable (the -th entry of ) for each .
Below you can find some exercises with explained solutions.
Let the sample space be
Sub-intervals of are assigned a probability equal to their length:
Define a sequence of random variables as follows:
Define a random variable as follows:
Does the sequence converge almost surely to ?
For a fixed sample point , the sequence of real numbers has limit
For , the sequence of real numbers has limit
Therefore, the sequence of random variables does not converge pointwise to becausefor . However, the set of sample points such that does not converge to is a zero-probability event: Therefore, the sequence converges almost surely to .
Let and be two sequences of random variables defined on a sample space .
Let and be two random variables defined on such that
Prove that
Denote by the set of sample points for which converges to :The fact that converges almost surely to implies thatwhere .
Denote by the set of sample points for which converges to :The fact that converges almost surely to implies thatwhere .
Now, denote by the set of sample points for which converges to :
Observe that if then converges to , because the sum of two sequences of real numbers is convergent if the two sequences are convergent. Therefore,Taking the complement of both sides, we obtainBut and as a consequence . Thus, the set of sample points such that does not converge to is included in the zero-probability event , which means that
Let the sample space be
Sub-intervals of are assigned a probability equal to their length:
Define a sequence of random variables as follows:
Find an almost sure limit of the sequence.
If or , then the sequence of real numbers is not convergent:
For , the sequence of real numbers has limitbecause for any we can find such that for any (as a consequence for any ).
Thus, the sequence of random variables converges almost surely to the random variable defined asbecause the set of sample points such that does not converge to is a zero-probability event:
Please cite as:
Taboga, Marco (2021). "Almost sure convergence", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/asymptotic-theory/almost-sure-convergence.
Most of the learning materials found on this website are now available in a traditional textbook format.