StatlectThe Digital Textbook
Index > Fundamentals of statistics > Maximum likelihood - Hypothesis testing

Wald test

This lecture discusses the Wald test and how it is used to perform tests of hypotheses on parameters that have been estimated by maximum likelihood.

Before reading this lecture, the reader is strongly advised to read the lecture entitled Maximum likelihood - Hypothesis testing, which introduces the basics of hypothesis testing in a maximum likelihood (ML) framework.

In what follows, we are going to assume that an unknown parameter $	heta _{0}$ has been estimated by ML, that it belongs to a parameter space [eq1], and that we want to test the null hypothesis[eq2]where [eq3] is a vector valued function, with $rleq p$. The aforementioned lecture on hypothesis testing gives some examples of common null hypotheses that can be written in this form.

Furthermore, we are going to assume that the following technical conditions are satisfied:

The Wald statistic

Let [eq5] be the estimate of the $p	imes 1$ parameter $	heta _{0}$ obtained by maximizing the log-likelihood over the whole parameter space $Theta $:[eq6]where [eq7] is the likelihood function and $xi _{n}$ is the sample.

Let us assume that the sample and the likelihood function satisfy some set of conditions that are sufficient to guarantee consistency and asymptotic normality of [eq8] (see the lecture on maximum likelihood for a set of such conditions).

The test statistic employed in the Wald test is[eq9]where n is the sample size, and $widehat{V}_{n}$ is a consistent estimate of the asymptotic covariance matrix of [eq10] (see the lecture entitled Maximum likelihood - Covariance matrix estimation).

Asymptotically, the test statistic has a Chi-square distribution, as stated by the following proposition.

Proposition Under the null hypothesis that [eq11], the Wald statistic $W_{n}$ converges in distribution to a Chi-square distribution with $r$ degrees of freedom.

Proof

We have assumed that [eq12] is consistent and asymptotically normal, which implies that [eq13] converges in distribution to a multivariate normal random variable with mean $	heta _{0}$ and asymptotic covariance matrix V, that is,[eq14]Now, by the delta method, we have that[eq15]But [eq16], so that[eq17]We have assumed that [eq18] and $widehat{V}_{n}$ are consistent estimators, that is,[eq19]where [eq20] denotes convergence in probability. Therefore, by the continuous mapping theorem, we have that[eq21]Thus we can write the Wald statistic as a sequence of quadratic forms [eq22]where[eq23]converges in distribution to a normal random vector $G$ with mean zero, and [eq24] converges in probability to [eq25]. By a standard result (see Exercise 1.2 in Convergence of transformations - Exercise set 1), such a sequence of quadratic forms converges in distribution to a Chi-square random variable with a number of degrees of freedom equal to [eq26].

The test

In the Wald test, the null hypothesis is rejected if[eq27]where $z$ is a pre-determined critical value.

The size of the test can be approximated by its asymptotic value[eq28]

where $Fleft( z
ight) $ is the distribution function of a Chi-square random variable with $r$ degrees of freedom.

The critical value $z$ can be chosen so as to achieve a pre-determined size, as follows:[eq29]

Example

The following example shows how to use the Wald test to test a simple linear restriction.

Example Let the parameter space be the set of all $2$-dimensional vectors, i.e., [eq30]. Suppose we have obtained the following estimates of the parameter and of the asymptotic covariance matrix:[eq31]where $90$ is the sample size. We want to test the restriction [eq32]where $	heta _{0,1}$ and $	heta _{0,2}$ denote the first and second component of $	heta _{0}$. Then, the function [eq33] is a function [eq34] defined by[eq35]In this case, $r=1$. The Jacobian of $g$ is[eq36]which has rank $r=1$. Note also that it does not depend on $	heta $. We have that[eq37]As a consequence, the Wald statistic is[eq38]Our test statistic has a Chi-square distribution with $r=1$ degrees of freedom. Suppose we want our test to have size $lpha =5%$. Then, our critical value $z$ is[eq39]where $Fleft( z
ight) $ is the distribution function of a Chi-square random variable with 1 degree of freedom and the value of [eq40] can be calculated with any statistical software (for, example, in MATLAB with the command chi2inv(0.95,1)). Therefore, the test statistic does not exceed the critical value[eq41]and we do not reject the null hypothesis.

The book

Most learning materials found on this website are now available in a traditional textbook format.