This lecture discusses convergence in probability, first for sequences of random variables, and then for sequences of random vectors.
As we have discussed in the lecture entitled Sequences of random variables and their convergence, different concepts of convergence are based on different ways of measuring the distance between two random variables (how "close to each other" two random variables are). The concept of convergence in probability is based on the following intuition: two random variables are "close to each other" if there is a high probability that their difference is very small.
Let be a sequence of random variables defined on a sample space . Let be a random variable and a strictly positive number. Consider the following probability:Intuitively, is considered far from when ; therefore, is the probability that is far from . If converges to , should become smaller and smaller as increases. In other words, the probability of being far from should go to zero when increases. Formally, we should haveNote that is a sequence of real numbers. Therefore, the above limit is the usual limit of a sequence of real numbers.
Furthermore, the condition should be satisfied for any (also for very small , which means that we are very restrictive on our criterion for deciding whether is far from ). This leads us to the following definition of convergence.
Definition Let be a sequence of random variables defined on a sample space . We say that is convergent in probability to a random variable defined on if and only iffor any . is called the probability limit of the sequence and convergence is indicated byor by
The following example illustrates the concept of convergence in probability.
Example Let be a discrete random variable with support and probability mass functionConsider a sequence of random variables whose generic term isWe want to prove that converges in probability to . Take any . Note that When , which happens with probability , we have thatand, of course, . When , which happens with probability , we have thatand only if (or only if ). Therefore,andThus, trivially converges to , because it is identically equal to zero for all such that . Since was arbitrary, we have obtained the desired result: for any .
The above notion of convergence generalizes to sequences of random vectors in a straightforward manner.
Let be a sequence of random vectors defined on a sample space , where each random vector has dimension . In the case of random variables, the sequence of random variables converges in probability if and only if for any , where is the distance of from . In the case of random vectors, the definition of convergence in probability remains the same, but distance is measured by the Euclidean norm of the difference between the two vectors:where the second subscript is used to indicate the individual components of the vectors and .
The following is a formal definition.
Definition Let be a sequence of random vectors defined on a sample space . We say that is convergent in probability to a random vector defined on if and only iffor any . is called the probability limit of the sequence and convergence is indicated byor by
Now, denote by the sequence of the -th components of the vectors . It can be proved that the sequence of random vectors is convergent in probability if and only if all the sequences of random variables are convergent in probability.
Proposition Let be a sequence of random vectors defined on a sample space . Denote by the sequence of random variables obtained by taking the -th component of each random vector . The sequence converges in probability to the random vector if and only if the sequence converges in probability to the random variable (the -th component of ) for each .
When the terms of the sequence depend on the value of a parameter that can take many different values, it is often necessary to use a slightly modified concept of convergence in probability. This is presented in the lecture entitled Uniform convergence in probability.
Below you can find some exercises with explained solutions:
Exercise set 1 (verify convergence in probability and find the probability limit).
Most learning materials found on this website are now available in a traditional textbook format.