Delta method

The Delta method is a theorem that can be used to derive the distribution of a function of an asymptotically normal variable.

It is often used to derive standard errors and confidence intervals for functions of parameters whose estimators are asymptotically normal.

The starting point: an asymptotically normal sequence

Let be a sequence of random variables such thatwhere:

is said to be asymptotically normal, is called the asymptotic mean of and its asymptotic variance.

For example, could be a sequence of sample means that are asymptotically normal because a Central Limit Theorem applies. Or it could be a sequence of maximum likelihood estimators satisfying a set of conditions that are sufficient for asymptotic normality.

The transformation

Now, consider the sequence where is a function.

The delta method is a method that allows us to derive, under appropriate conditions, the asymptotic distribution of from the asymptotic distribution of .

The proposition

A formal statement of the delta method is given in the following proposition.

Proposition Let be a sequence of random variables such thatLet be a continuously differentiable function. Then,

Proof

By the mean value theorem, there exists a point lying between and , such thatBy subtracting from both sides and multiplying by , we obtainSince converges in probability to , and lies between and , also converges in probability to . Because the derivative is continuous, by the continuous mapping theoremwhere denotes convergence in probability. Therefore, the first term of the product converges in probability to a constant. By assumption, the second term converges in distribution to a normal random variable having mean and variance . As a consequence, Slutsky's theorem applies and the product converges in distribution toBy elementary rules on linear transformations of normal random variables, this has a normal distribution with mean and variance

Example

In this example we show how the delta method can be applied.

Suppose that a sequence is asymptotically normal with asymptotic mean and asymptotic variance , that is,

We want to derive the asymptotic distribution of the sequence .

The functionis continuously differentiable, so we can apply the delta method.

The asymptotic mean of the transformed sequence is

In order to compute the asymptotic variance, we need to take the first derivative of the function , which isand evaluate it at :

Therefore, the asymptotic variance isand we can write

Multivariate generalization

The delta method generalizes also to multivariate settings, as stated by the following proposition.

Proposition Let be a sequence of random vectors such thatwhere is a multivariate normal distribution with mean and covariance matrix , is a constant vector, and indicates convergence in distribution. Let . If all the entries of have continuous partial derivatives with respect to , then where is the Jacobian of , i.e., the matrix of partial derivatives of the entries of with respect to the entries of .

The next example shows how the multivariate delta method can be applied.

Example Suppose that a sequence of random vectors satisfieswhere the asymptotic mean isand the asymptotic covariance matrix isDenote the two components of by and . We want to derive the asymptotic distribution of the sequence . The functionis continuously differentiable, so we can apply the delta method. The asymptotic mean of the transformed sequence isIn order to compute the asymptotic covariance matrix, we need to compute the Jacobian of the function , which isand evaluate it at :Therefore, the asymptotic variance isand we can write

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Let be an asymptotically normal sequence with asymptotic mean and asymptotic variance , that is,

Derive the asymptotic distribution of the sequence .

Solution

The functionis continuously differentiable, so we can apply the delta method. The asymptotic mean of the transformed sequence isIn order to compute the asymptotic variance, we need to take the first derivative of the function , which isand evaluate it at :Therefore, the asymptotic variance isand we can write

Exercise 2

Let be a sequence of random vectors satisfyingwhere the asymptotic mean isand the asymptotic covariance matrix is

Denote the two entries of by and .

Derive the asymptotic distribution of the sequence of products

Solution

We can apply the delta method because the functionis continuously differentiable. The asymptotic mean of the transformed sequence isThe Jacobian of the function isBy evaluating it atwe obtainTherefore, the asymptotic covariance matrix isand we can write

Exercise 3

Let be a sequence of random vectors satisfyingwhere the asymptotic mean isand the asymptotic covariance matrix is

Denote the two entries of by and .

Derive the asymptotic distribution of the sequence of vectors where the two entries of satisfy

Solution

We can apply the delta method because the functionsare continuously differentiable. The asymptotic mean of the transformed sequence is a vector whose entries are The Jacobian of the function isBy evaluating it atwe obtainAs a consequence, the asymptotic covariance matrix isThus,where and have been calculated above.