Search for probability and statistics terms on Statlect
StatLect

Consistent estimator

by , PhD

An estimator of a given parameter is said to be consistent if it converges in probability to the true value of the parameter as the sample size tends to infinity.

Table of Contents

The main elements of an estimation problem

Before providing a definition of consistent estimator, let us briefly recall the main elements of a parameter estimation problem:

Sampling variability

Before being observed, the sample $xi _{n}$ is regarded as random.

Therefore, [eq2], which depends on $xi _{n}$, is a random variable.

When needed, we write[eq3]to highlight the fact that the estimator [eq4] is a function of the sample $xi _{n}$.

Definition

Now, imagine that we are able to collect new data and increase our sample size n indefinitely, so as to obtain a sequence of samples [eq5] and a sequence of estimators [eq6].

If this "imaginary" sequence of estimators converges in probability to the true parameter value, then it is said to be consistent.

Definition A sequence of estimators [eq7] is said to be consistent if and only if[eq8]where $QTR{rm}{plim}$ denotes convergence in probability.

Terminology

Note that we have defined "consistent sequences of estimators".

But what do we mean by "consistent estimator"? The latter locution is informally used to mean that:

  1. the same predefined rule is used to generate all the estimators in the sequence;

  2. the terms of the sequence converge in probability to the true parameter value.

Thus, the concept of consistency extends from the sequence of estimators to the rule used to generate it.

For instance, suppose that the rule is to "compute the sample mean", so that [eq9] is a sequence of sample means over samples of increasing size.

If [eq10] converges in probability to the mean of the distribution that generated the samples, then we say that [eq11] is consistent.

By a slight abuse of language, we also say that the sample mean is a consistent estimator.

Examples

The following table contains examples of consistent estimators (with links to lectures where consistency is proved).

Estimator Estimated parameter Lecture where proof can be found
Sample mean Expected value Estimation of the mean
Sample variance Variance Estimation of the variance
OLS estimator Coefficients of a linear regression Properties of the OLS estimator
Maximum likelihood estimator Any parameter of a distribution Maximum likelihood

Inconsistent estimator

An estimator which is not consistent is said to be inconsistent.

Consistent and asymptotically normal

You will often read that a given estimator is not only consistent but also asymptotically normal, that is, its distribution converges to a normal distribution as the sample size increases.

You might think that convergence to a normal distribution is at odds with the fact that consistency implies convergence in probability to a constant (the true parameter value).

In other words, you might ask yourself: "Is convergence to a constant or to a distribution?"

To answer this question, we should give a more precise definition of asymptotic normality.

Consider the ratio [eq12]

When [eq10] is consistent, both the difference [eq14] and the standard deviation [eq15] converge to zero as n tends to infinity. However, their ratio can converge to a distribution. When it converges to a standard normal distribution, then the sequence [eq10] is said to be asymptotically normal.

The practical consequence of asymptotic normality is that, when n is large, we can approximate the above ratio with a standard normal distribution.

It follows that [eq4] can be approximated by a normal distribution with mean $	heta _{0}$ and standard deviation [eq18]. But the latter converges to zero, so that the distribution becomes more and more concentrated around the mean, ultimately converging to a constant.

More details

Consistency is discussed in more detail in the lecture on Point estimation.

Keep reading the glossary

Previous entry: Conditional probability mass function

Next entry: Convergence criterion

How to cite

Please cite as:

Taboga, Marco (2021). "Consistent estimator", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/glossary/consistent-estimator.

The books

Most of the learning materials found on this website are now available in a traditional textbook format.