StatlectThe Digital Textbook
Index > Probability distributions

Gamma distribution

The Gamma distribution can be thought of as a generalization of the Chi-square distribution. If a random variable Z has a Chi-square distribution with n degrees of freedom and $h$ is a strictly positive constant, then the random variable X defined as [eq1]has a Gamma distribution with parameters n and $h$.

Definition

Gamma random variables are characterized as follows.

Definition Let X be an absolutely continuous random variable. Let its support be the set of positive real numbers:[eq2]Let [eq3]. We say that X has a Gamma distribution with parameters n and $h$ if its probability density function is[eq4]where $c$ is a constant:[eq5]and [eq6] is the Gamma function.

A random variable having a Gamma distribution is also called a Gamma random variable.

To better understand the Gamma distribution, you can have a look at its density plots.

Expected value

The expected value of a Gamma random variable X is[eq7]

Proof

It can be derived as follows:[eq8]

Variance

The variance of a Gamma random variable X is[eq9]

Proof

It can be derived thanks to the usual variance formula ([eq10]):[eq11]

Moment generating function

The moment generating function of a Gamma random variable X is defined for any $frac{n}{2h}$:[eq12]

Proof

By using the definition of moment generating function, we obtain[eq13]where the integral equals 1 because it is the integral of the probability density function of a Gamma random variable with parameters n and [eq14]. Thus,[eq15]Of course, the above integrals converge only if [eq16], i.e. only if $frac{n}{2h}$. Therefore, the moment generating function of a Gamma random variable exists for all $frac{n}{2h} $.

Characteristic function

The characteristic function of a Gamma random variable X is[eq17]

Proof

It can be derived by using the definition of characteristic function and a Taylor series expansion:[eq18]

Distribution function

The distribution function of a Gamma random variable is[eq19]where the function[eq20]is called lower incomplete Gamma function and is usually evaluated using specialized computer algorithms.

Proof

This is proved as follows:[eq21]

More details

In the following subsections you can find more details about the Gamma distribution.

The Gamma distribution is a scaled Chi-square distribution

If a variable X has the Gamma distribution with parameters n and $h$, then[eq22]where Z has a Chi-square distribution with n degrees of freedom.

Proof

This can be easily proved using the formula for the density of a function of an absolutely continuous variable ([eq23] is a strictly increasing function of Z, since $frac{h}{n}$ is strictly positive):[eq24]The density function of a Chi-square random variable with n degrees of freedom is[eq25]where [eq26]Therefore,[eq27]which is the density of a Gamma distribution with parameters n and $h$.

Thus, the Chi-square distribution is a special case of the Gamma distribution because, when $h=n$, we have[eq28]

In other words, a Gamma distribution with parameters n and $h=n$ is just a Chi square distribution with n degrees of freedom.

A Gamma random variable times a strictly positive constant is a Gamma random variable

Multiplying a Gamma random variable by a strictly positive constant one obtains another Gamma random variable. If X is a Gamma random variable with parameters n and $h$, then the random variable Y defined as[eq29]has a Gamma distribution with parameters n and $ch$.

Proof

This can be easily seen using the result from the previous subsection:[eq30]where Z has a Chi-square distribution with n degrees of freedom. Therefore,[eq31]In other words, Y is equal to a Chi-square random variable with n degrees of freedom, divided by n and multiplied by $ch$. Therefore, it has a Gamma distribution with parameters n and $ch$.

A Gamma random variable is a sum of squared normal random variables

In the lecture entitled Chi-square distribution we have explained that a Chi-square random variable Z with n degrees of freedom (n integer) can be written as a sum of squares of n independent normal random variables $W_{1}$, ...,$W_{n}$ having mean 0 and variance 1:[eq32]

In the previous subsections we have seen that a variable X having a Gamma distribution with parameters n and $h$ can be written as[eq33]where Z has a Chi-square distribution with n degrees of freedom.

Putting these two things together, we obtain[eq34]where we have defined[eq35]But the variables $Y_{i}$ are normal random variables with mean 0 and variance $frac{h}{n}$. Therefore, a Gamma random variable with parameters $n $ and $h$ can be seen as a sum of squares of n independent normal random variables having mean 0 and variance $h/n$.

Density plots

This page collects some plots of the Gamma distribution. These plots help us to understand how the shape of the Gamma distribution changes when its parameters are changed.

Plot 1 - Same mean but different degrees of freedom

The following plot contains the graphs of two Gamma probability density functions.

Because $h=3$ in both cases, the two distributions have the same mean. However, by increasing the number of degrees of freedom from $n=6$ to $n=8$, the shape of the distribution changes (the more the degrees of freedom are increased the more the distribution resembles a normal distribution).

The thin vertical lines indicate the means of the two distributions.

Gamma density plot 1

Plot 2 - Different means but same number of degrees of freedom

The following plot contains the graphs of two Gamma probability density functions:

Increasing the parameter $h$ changes the mean of the distribution from $2$ to $4$. However, the two distributions have the same number of degrees of freedom ($n=6$). Therefore, they have the same shape (one is the "stretched version of the other" - it would look exactly the same on a different scale).

Gamma density plot 2

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Let X_1 and X_2 be two independent Chi-square random variables having $3$ and $5$ degrees of freedom respectively. Consider the following random variables:[eq36]What distribution do they have?

Solution

Being multiples of Chi-square random variables, the variables $Y_{1}$, $Y_{2} $ and $Y_{3}$ all have a Gamma distribution. The random variable $X_{1} $ has $n=3$ degrees of freedom and the random variable $Y_{1}$ can be written as[eq37]where $h=6$. Therefore $Y_{1}$ has a Gamma distribution with parameters $n=3$ and $h=6$. The random variable X_2 has $n=5$ degrees of freedom and the random variable $Y_{2}$ can be written as[eq38]where $h=5/3$. Therefore $Y_{2}$ has a Gamma distribution with parameters $n=5$ and $h=5/3$. The random variable $X_{1}+X_{2}$ has a Chi-square distribution with $n=3+5=8$ degrees of freedom, because X_1 and X_2 are independent (see the lecture entitled Chi-square distribution), and the random variable $Y_{3}$ can be written as[eq39]where $h=24$. Therefore $Y_{3}$ has a Gamma distribution with parameters $n=8 $ and $h=24$.

Exercise 2

Let X be a random variable having a Gamma distribution with parameters $n=4 $ and $h=2$. Define the following random variables:[eq40]

What distribution do these variables have?

Solution

Multiplying a Gamma random variable by a strictly positive constant one still obtains a Gamma random variable. In particular, the random variable $Y_{1}$ is a Gamma random variable with parameters $n=4$ and [eq41] The random variable $Y_{2}$ is a Gamma random variable with parameters $n=4$ and [eq42] The random variable $Y_{3}$ is a Gamma random variable with parameters $n=4$ and [eq43]The random variable $Y_{3}$ is also a Chi-square random variable with $4$ degrees of freedom (remember that a Gamma random variable with parameters n and $h$ is also a Chi-square random variable when $n=h$).

Exercise 3

Let X_1, X_2 and $X_{3}$ be mutually independent normal random variables having mean $mu =0$ and variance $sigma ^{2}=3$. Consider the random variable[eq44]What distribution does X have?

Solution

The random variable X can be written as [eq45]where $Z_{1}$, $Z_{2}$ and $Z_{3}$ are mutually independent standard normal random variables. The sum [eq46] has a Chi-square distribution with $3$ degrees of freedom (see the lecture entitled Chi-square distribution). Therefore X has a Gamma distribution with parameters $n=3$ and $h=18$.

The book

Most learning materials found on this website are now available in a traditional textbook format.