Search for probability and statistics terms on Statlect
StatLect

Fundamentals of mathematical statistics

Learn the mathematical foundations of statistics, through a series of rigorous but accessible lectures on the most frequently utilized statistical concepts.

The foundations

Statistical inference

Samples, statistical models, estimation, statistical decisions

Point estimation

Mean estimation

Examples of mean estimation and mathematical properties of common mean estimators

Point estimation

Estimates and estimators of a parameter and criteria to evaluate them

Variance estimation

Examples of variance estimation and mathematical properties of common variance estimators

The sample mean converges to the population mean as the sample size increases.

Interval estimation

Confidence interval for the mean

Examples of confidence intervals for the mean, with detailed derivations of their properties

Interval estimation

Confidence intervals, confidence coefficients, how to evaluate them

Confidence interval for the variance

Examples of confidence intervals for the variance, with detailed derivations of their properties

Hypothesis testing

Testing hypotheses about the mean

Examples of hypothesis tests about the mean, with detailed derivations of their properties

Hypothesis testing

Null and alternative hypothesis, types of errors, size and power

Testing hypotheses about the variance

Examples of hypothesis tests about the variance, with detailed derivations of their properties

Learn all the terminology used in the theory of hypothesis testing.

Estimation methods

Estimation methods

Introduction to estimators used in mathematical statistics, including ML, GMM, NLS

Maximum likelihood estimation

MLE - Covariance matrix estimation

How to estimate the covariance matrix of a maximum likelihood estimator

Maximum likelihood

The fundamentals of the theory of maximum likelihood estimation

MLE - Hypothesis testing

How to carry out tests of hypothesis in a maximum likelihood framework

Maximum likelihood algorithm

How to solve numerically the maximum likelihood optimization problem

Score test

A test of hypothesis involving only restricted ML estimates

Wald test

A test of hypothesis involving only unrestricted ML estimates

Model selection criteria

Criteria used to select the best model among a set of candidate models estimated by ML

Likelihood ratio test

A test of hypothesis involving both restricted and unrestricted ML estimates

EM algorithm

Recursive algorithm used for ML estimation of latent-variable models

Exponential family of distributions

Parametric families that are particularly important in maximum likelihood estimation

Learn the steps that you need to perform to solve a maximum likelihood estimation problem.

Conditional models

Conditional models

The fundamentals of conditional models, regression and classification

Linear regression

Properties of the OLS estimator

Asymptotic properties of the OLS estimators of regression coefficients

Linear regression models

Introduction to the mathematics of linear regression models: notation, assumptions, inference.

R squared of a linear regression

A measure of how well a linear regression fits the data

The Normal Linear Regression Model

A regression model in which errors are conditionally normal

Gauss-Markov theorem

The OLS estimator is the best among those that are linear and unbiased

Linear regression - Hypothesis testing

How to test hypotheses about coefficients estimated by OLS

Standardized linear regression

Linear regression where all the variables are centered and divided by their standard deviation

Generalized least squares

How to estimate the regression coefficients efficiently when the errors are heteroskedastic or correlated

Ridge regression

A biased estimator of linear regression coefficients whose MSE can be lower than that of OLS

Multicollinearity

If regressors are highly correlated, then OLS coefficient estimates have high variance

Partitioned regression

How to separately estimate the regression coefficients of two groups of regressors

Dummy variables

Variables used in regression models to encode categorical features

Linear regression calculator

Use our calculator to run your regressions effortlessly and without coding

The three assumptions made in the Gauss-Markov theorem: full-rank, strict exogeneity, sphericity.

Classification models

Logistic classification model

Binary classification model in which the logistic function is used to transform inputs

Classification models

Conditional models in which the output variable has a discrete distribution

Probit classification model

Binary model in which the cdf of a standard normal distribution is used to transform inputs

Topics in stochastic processes and time series

Autocorrelation

Definition of autocorrelation, autocorrelation function (ACF), sample ACF, ACF plots.

Markov chains

Sequences of random vectors whose future does not depend on the past conditional on the present

Read our introduction to Markov Chains, with explanations of concepts such as irreducibility, stationarity, homogeneity.

Markov Chain Monte Carlo (MCMC) methods

MCMC diagnostics

How to diagnose (and solve) problems with MCMC samples

Markov Chain Monte Carlo

Monte Carlo methods based on sequences of dependent draws from a distribution

Metropolis-Hastings algorithm

MCMC algorithm based on acceptance/rejection of draws from a proposal distribution

Bayesian statistics

Hierarchical Bayesian models

Bayesian models in which the parameters of the prior are assigned a hyper-prior

Bayesian inference

The fundamentals of Bayesian inference: prior, likelihood, posterior distributions

Bayesian linear regression

Bayesian inference about the parameters of a normal linear regression model

Normal distribution - Bayesian estimation

Bayesian inference about the parameters of a normal distribution

Posterior odds ratio

A simple and intuitive way of comparing two different models or hypotheses

Conjugate prior

When prior and posterior distribution belong to the same parametric family

Uninformative prior

An "objective" prior that has little influence on the posterior distribution

Jeffrey's scale

A scale used to translate the value of the Bayes factor into a qualitative judgement on the evidence

The books

Most of the learning materials found on this website are now available in a traditional textbook format.