 StatLect

# Mean squared error

In the theory of point estimation, the mean squared error is frequently used to assess the risk of an estimator, that is, how large are on average the losses generated by the estimation errors committed when employing the estimator in question.

When the parameter to be estimated is a scalar, the mean squared error is equal to the expected value of the square of the difference between the value taken by the estimator and the true value of the parameter.

When the parameter to be estimated is a vector, we take the Euclidean norm of the difference before computing the square. ## Acronyms

The acronym MSE is often employed.

## Definition

The following is a possible definition of mean squared error.

Definition Let be an estimator of an unknown parameter . The estimation error is When the squared error is used as a loss function, then the risk of the estimator (i.e., the expected value of the loss) is and it is called the mean squared error of the estimator .

When is a scalar, the squared error is because the Euclidean norm of a scalar is equal to its absolute value. Therefore, the MSE becomes ## Bias variance decomposition

The following decomposition is often used to distinguish between the two main sources of error, called bias and variance.

Proposition The mean squared error of an estimator can be written as where. is the trace of the covariance matrix of. and is the bias of the estimator, that is, the expected difference between the estimator and the true value of the parameter.

Proof

Suppose the true parameter and its estimator are column vectors. Then we can write: where: in step we have expanded the products; in step   we have used the linearity of the expected value operator; in step we have used the fact that the trace of a square matrix is equal to the sum of its diagonal elements.

When the parameter is a scalar, the above formula for the bias-variance decomposition becomes It is then clear that the mean squared error of an unbiased estimator (an estimator that has zero bias) is equal to the variance of the estimator itself.

## More details

More details about loss functions, statistical risk and the mean squared error can be found in the lecture entitled Point estimation.

Previous entry: Mean

Next entry: Multinomial coefficient