Search
STATLECT > Additional topics in probability theory

Joint moment generating function

The concept of joint moment generating function (joint mgf) generalizes the concept of moment generating function to random vectors:

Definition_ Let X be a Kx1 random vector. If the expected value:[eq1]exists and is finite for all Kx1 real vectors $t$ belonging to a closed rectangle H:[eq2]with $h_{i}>0$ for all $i=1,ldots ,K$, then we say that X possesses a joint moment generating function and the function [eq3] defined by:[eq4]is called the joint moment generating function of X.

Example

As an example, we derive the joint mgf of a standard multivariate normal random vector.

Example_ Let X be a Kx1 standard multivariate normal random vector. Its support R_X is:[eq5]and its joint probability density function [eq6] is:[eq7]As explained in the lecture entitled Multivariate normal distribution, the K components of X are K mutually independent standard normal random variables, because the joint probability density function of X can be written as:[eq8]where $x_{i}$ is the i-th entry of x and [eq9] is the probability density function of a standard normal random variable:[eq10]Therefore, the joint mgf of X can be derived as follows:[eq11]Since the mgf of a standard normal random variable is:[eq12]then:[eq13][eq14] is defined for any $t_{i}in U{211d} $. As a consequence, [eq15] is defined for any $tin U{211d} ^{K}$.

Relation to cross-moments

The next proposition shows how the joint mgf can be used to derive the cross-moments of a random vector.

Proposition_ If a Kx1 random vector X possesses a joint mgf [eq16], then X possesses finite cross-moments of order n, for any $nin U{2115} $. Furthermore, if you define a cross-moment of order n as:[eq17]where [eq18] and [eq19], then:[eq20]where the derivative on the right-hand side is the n-th order partial derivative of [eq21] evaluated at the point [eq22].

nav_button Proof

We do not provide a rigorous proof of this proposition, but see e.g. Pfeiffer, P. E. (1978) Concepts of probability theory, Courier Dover Publications and DasGupta, A. (2010) Fundamentals of probability: a first course, Springer. The main intuition, however, is quite simple. Differentiation is a linear operation and the expected value is a linear operator. This allows us to differentiate through the expected value, provided appropriate technical conditions (omitted here) are satisfied:[eq23]Evaluating this derivative at the point [eq24], we obtain:[eq25]

The following example shows how the above proposition can be applied.

Example_ Let's continue with the previous example. The joint mgf of a $2 imes 1$ standard normal random vector X is:[eq26]The second cross-moment of X can be computed by taking the second cross partial derivative of [eq27]:[eq28]

Characterization of a joint distribution

One of the most important properties of the joint mgf is that it completely characterizes the joint distribution of a random vector:

Proposition_ Let X and Y be two Kx1 random vectors, possessing joint mgfs [eq29] and [eq30]. Denote by [eq31] and [eq32] their joint distribution functions. Then:[eq33]

In other words, X and Y have the same joint distribution if and only if they have the same joint mgfs.

This proposition is used very often in applications where one needs to demonstrate that two joint distributions are equal. In such applications, proving equality of the joint moment generating functions is often much easier than proving equality of the joint distribution functions.

More details

Joint moment generating function of a linear transformation

Let X be a Kx1 random vector possessing joint mgf [eq34]. Define:[eq35]where $A $is a $L imes 1$ constant vector and and $B $is an $L imes K$ constant matrix. Then, the $L imes 1$ random vector Y possesses a joint mgf [eq30] and:[eq37]

nav_button Proof

Using the definition of mgf:[eq38]If [eq39] is defined on a closed rectangle H, then [eq40] is defined on another closed rectangle whose shape and location depend on A and $B$.

Joint moment generating function of a random vector with independent entries

Let X be a Kx1 random vector. Let its entries X_1, ..., $X_{K}$ be K mutually independent random variables possessing a mgf. Denote the mgf of the i-th entry of X by [eq14].

Then, the joint mgf of X is:[eq42]

nav_button Proof

This fact is demonstrated as follows:[eq43]

Joint mgf of a sum of mutually independent random vectors

Let X_1, ..., X_n be n mutually independent random vectors, all of dimension Kx1. Let Z be their sum:[eq44]Then, the joint mgf of Z is the product of the joint mgfs of X_1, ..., X_n:[eq45]

nav_button Proof

This fact descends from the properties of mutually independent random vectors and from the definition of joint mgf:[eq46]

Solved exercises

Some solved exercises on joint moment generating functions can be found below:

  1. Exercise set 1

by
About | Contacts | Privacy and terms of use | Sitemap