Convergence in distribution

This lecture discusses convergence in distribution, first for sequences of random variables and then for sequences of random vectors.

Table of contents

The intuition
How to define convergence
Definition for sequences of random variables
Example - Maximum of uniform random variables
Proper distribution functions
Definition for sequences of random vectors
Relation between univariate and multivariate convergence
Solved exercises

The intuition

We have previously explained that different concepts of convergence are based on different ways of measuring the distance between two random variables (how "close to each other" two random variables are).

The concept of convergence in distribution is based on the following intuition: two random variables are "close to each other" if their distribution functions are "close to each other".

How to define convergence

Let us consider a sequence of random variables and a generic random variable belonging to the sequence.

Denote by the distribution function of . is a function .

Once we fix , the following facts are true:

the value associated to the point is a real number;
the sequence is a sequence of real numbers;
we can easily assess whether the sequence is convergent by using the standard definition of convergence of a sequence of real numbers.

If the sequence is convergent, its limit depends on the specific we have fixed.

A sequence of random variables is said to be convergent in distribution if and only if the sequence is convergent for any choice of , except possibly for some "special values" of where is not continuous in .

Definition for sequences of random variables

The following definition summarizes what we have said above.

Definition Let be a sequence of random variables. Denote by the distribution function of . We say that is convergent in distribution (or convergent in law) if and only if there exists a distribution function such that the sequence converges to for all points where is continuous.

If a random variable has distribution function , then is called the limit in distribution (or limit in law) of the sequence and convergence is indicated by

Note that convergence in distribution only involves the distribution functions of the random variables belonging to the sequence and that these random variables need not be defined on the same sample space.

On the contrary, the modes of convergence we have discussed in previous lectures (pointwise convergence, almost sure convergence, convergence in probability, mean-square convergence) require that all the variables in the sequence be defined on the same sample space.

Example - Maximum of uniform random variables

Let us make an example

Let be a sequence of IID random variables all having a uniform distribution on the interval .

In other words, the distribution function of is [eq24]

Define [eq25]

The distribution function of $Y_{n}$ is [eq26]

Thus, [eq27]

Since we have [eq29] where is the distribution function of an exponential random variable.

Therefore, the sequence converges in law to an exponential distribution.

Proper distribution functions

Let be a sequence of random variables and denote by the distribution function of .

Suppose that we find a function such that for all where is continuous.

How do we check that is a proper distribution function, so that we can say that the sequence converges in distribution?

As explained in the glossary entry on distribution functions, we need to check that satisfies the four properties characterizing proper distribution functions, that is, must be increasing, right-continuous and its limits at minus and plus infinity must be and

Definition for sequences of random vectors

The definition of convergence in distribution of a sequence of random vectors is almost identical to that given for random variables.

We just need to replace distribution functions in the above definition with joint distribution functions.

Definition Let be a sequence of random vectors. Denote by the joint distribution function of . We say that is convergent in distribution (or convergent in law) if and only if there exists a joint distribution function such that the sequence converges to for all points $xin U{211d} ^{K}$ where is continuous.

If a random vector has joint distribution function , then is called the limit in distribution (or limit in law) of the sequence and convergence is indicated by

Relation between univariate and multivariate convergence

It is important to note that for other notions of stochastic convergence (in probability, almost sure and in mean-square), the convergence of each single entry of the random vector is necessary and sufficient for their joint convergence, that is, for the convergence of the vector as a whole. Instead, for convergence in distribution, the individual convergence of the entries of the vector is necessary but not sufficient for their joint convergence.

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Let be a sequence of random variables having distribution functions [eq53]

Find the limit in distribution (if it exists) of the sequence .

Solution

If , then [eq56] If , then [eq57] We now need to verify that the function [eq58] is a proper distribution function. The function is increasing, continuous, its limit at minus infinity is and its limit at plus infinity is , hence it satisfies the four properties that a proper distribution function needs to satisfy. This implies that converges in distribution to a random variable having distribution function .

Exercise 2

Let be a sequence of random variables having distribution functions [eq62]

Find the limit in distribution (if it exists) of the sequence .

Solution

If , then [eq64] If , then [eq66] Therefore, the distribution functions converge to the function [eq68] which is not a proper distribution function, because it is not right-continuous at the point . However, note that the function [eq69] is a proper distribution function and it is equal to at all points except at the point . But this is a point of discontinuity of . As a consequence, the sequence converges in distribution to a random variable having distribution function .

Exercise 3

Let be a sequence of random variables having distribution functions [eq75]

Find the limit in distribution (if it exists) of the sequence .

Solution

The distribution functions converge to the function [eq68] This is the same limiting function found in the previous exercise. As a consequence, the sequence converges in distribution to a random variable having distribution function [eq80]

How to cite

Please cite as:

Taboga, Marco (2021). "Convergence in distribution", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/asymptotic-theory/convergence-in-distribution.