Search for probability and statistics terms on Statlect

Estimation methods

by , PhD

Estimation methods are general techniques that can be used to derive estimators in a parametric estimation problem.

Table of Contents

The general framework

Let us recall the main elements of a parametric estimation problem, which were explained in detailed in the lecture on point estimation:

The aim of an estimation method is to produce a parameter estimate $widehat{	heta }$ that is as close as possible to the true parameter $	heta _{0}$.

Extremum estimators

Several widely employed estimators fall within the class of extremum estimators.

An estimator $widehat{	heta }$ is an extremum estimator if it can be represented as the solution of a maximization problem:[eq6]where $Q$ is a function of both the parameter $	heta $ and the sample $xi $.

General conditions can be derived for the consistency and asymptotic normality of extremum estimators. We do not discuss them here (see, e.g., Hayashi 2000), but we give some examples of extremum estimation methods and we refer the reader to lectures that describe these examples in a more detailed manner.

Maximum likelihood

In maximum likelihood estimation, we maximize the likelihood of the sample:[eq7]where:

  1. if $Xi $ is discrete, the likelihood [eq8]is the joint probability mass function of $Xi $ associated to the distribution that corresponds to the parameter $	heta $;

  2. if $Xi $ is absolutely continuous, the likelihood [eq9]is the joint probability density function of $Xi $ associated to the distribution that corresponds to the parameter $	heta $.

The vector $widehat{	heta }$ is called the maximum likelihood estimator of $	heta _{0}$.

The maximum likelihood estimation method is discussed in more detail in the lecture entitled Maximum Likelihood.

Generalized method of moments

In the generalized method of moments (GMM) estimation method, the distribution associated to the parameter $	heta _{0}$ satisfies a moment condition:[eq10]where [eq11] is a (vector) function and [eq12] indicates that the expected value is computed using the distribution associated to $	heta _{0}$.

The GMM estimator $widehat{	heta }$ is obtained as[eq13]where [eq14] is a measure of the distance of [eq15] from its expected value of 0 and the estimator is an extremum estimator because[eq16]

Least squares

In the least squares estimation method, the sample $xi $ comprises:

It is postulated that there exists a function [eq18] such that[eq19]

The least squares estimator $widehat{	heta }$ is obtained as[eq20]

The estimator is an extremum estimator because[eq21]

A special case of the least squares estimator is analyzed in detail in the lecture on the properties of the OLS estimator.


Hayashi, F. (2000) Econometrics, Princeton University Press.

How to cite

Please cite as:

Taboga, Marco (2021). "Estimation methods", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix.

The books

Most of the learning materials found on this website are now available in a traditional textbook format.