In the theory of point estimation, the mean squared error is frequently used to assess the risk of an estimator, that is, how large are on average the losses generated by the estimation errors committed when employing the estimator in question.
When the parameter to be estimated is a scalar, the mean squared error is equal to the expected value of the square of the difference between the value taken by the estimator and the true value of the parameter.
When the parameter to be estimated is a vector, we take the Euclidean norm of the difference before computing the square.
The acronym MSE is often employed.
The following is a possible definition of mean squared error.
Definition Let be an estimator of an unknown parameter . The estimation error isWhen the squared error is used as a loss function, then the risk of the estimator (i.e., the expected value of the loss) isand it is called the mean squared error of the estimator .
When is a scalar, the squared error isbecause the Euclidean norm of a scalar is equal to its absolute value. Therefore, the MSE becomes
The following decomposition is often used to distinguish between the two main sources of error, called bias and variance.
Proposition The mean squared error of an estimator can be written aswhere. is the trace of the covariance matrix of. andis the bias of the estimator, that is, the expected difference between the estimator and the true value of the parameter.
Suppose the true parameter and its estimator are column vectors. Then we can write:where: in step we have expanded the products; in step we have used the linearity of the expected value operator; in step we have used the fact that the trace of a square matrix is equal to the sum of its diagonal elements.
When the parameter is a scalar, the above formula for the bias-variance decomposition becomes
It is then clear that the mean squared error of an unbiased estimator (an estimator that has zero bias) is equal to the variance of the estimator itself.
More details about loss functions, statistical risk and the mean squared error can be found in the lecture entitled Point estimation.
Previous entry: Mean
Next entry: Multinomial coefficient
Most learning materials found on this website are now available in a traditional textbook format.