Two random variables are independent if they convey no information about each other and, as a consequence, receiving information about one of the two does not change our assessment of the probability distribution of the other.
This lecture provides a formal definition of independence and discusses how to verify whether two or more random variables are independent.
Recall (see the lecture entitled Independent events) that two events and are independent if and only if:
This definition is extended to random variables as follows.
Definition Two random variables and are said to be independent if and only if:for any couple of events and , where and .
In other words, two random variables are independent if and only if the events related to those random variables are independent events.
The independence between two random variables is also called statistical independence.
Checking the independence of all possible couples of events related to two random variables can be very difficult. This is the reason why the above definition is seldom used to verify whether two random variables are independent. The following criterion is more often used instead.
Proposition Two random variables and are independent if and only ifwhere is their joint distribution function and and are their marginal distribution functions.
Using some facts from measure theory (not proved here), it is possible to demonstrate that, when checking for the condition it is sufficient to confine attention to sets and taking the form:Thus, two random variables are independent if and only if:Using the definitions of joint and marginal distribution function, this condition can be written as:
Example Let and be two random variables with marginal distribution functionsand joint distribution function and are independent if and only if which is straightforward to verify. When or , then When and , then:
When the two variables, taken together, form a discrete random vector, independence can also be verified using the following proposition:
Proposition Two random variables and , forming a discrete random vector, are independent if and only if:where is their joint probability mass function and and are their marginal probability mass functions.
The following example illustrates how this criterion can be used:
Example Let be a discrete random vector with support: Let its joint probability mass function be:In order to verify whether and are independent, we first need to derive the marginal probability mass functions of and . The support of is:and the support of is:We need to compute the probability of each element of the support of :Thus, the probability mass function of is:We need to compute the probability of each element of the support of :Thus, the probability mass function of is:The product of the marginal probability mass functions is:which is obviously different from . Therefore, and are not independent.
When the two variables, taken together, form an absolutely continuous random vector, independence can also be verified using the following proposition:
Proposition Two random variables and , forming an absolutely continuous random vector, are independent if and only if:where is their joint probability density function and and are their marginal probability density functions.
The following example illustrates how this criterion can be used:
Example Let the joint probability density function of and be:Its marginals are:and:Verifying that is straightforward. When or , then . When and , then:
The following subsections contain more details about statistical independence.
The definition of mutually independent random variables extends the definition of mutually independent events to random variables:
Definition We say that random variables , ..., are mutually independent (or jointly independent) if and only if for any sub-collection of random variables , ..., (where ) and for any collection of events , where .
In other words, random variables are mutually independent if the events related to those random variables are mutually independent events.
Denote by a random vector whose components are , ..., . The above condition for mutual independence can be replaced:
in general, by a condition on the joint distribution function of :
for discrete random variables, by a condition on the joint probability mass function of :
for absolutely continuous random variables, by a condition on the joint probability density function of :
It can be proved that random variables , ..., are mutually independent if and only if:for any functions , ..., such that the above expected values exist and are well-defined.
If two random variables and are independent, then their covariance is zero:
This is an immediate consequence of the fact that, if and are independent then:(see the Mutual independence via expectations property above). When and are identity functions ( and ), then:Therefore, by the covariance formula:
The converse is not true: two random variables that have zero covariance are not necessarily independent.
The above notions are easily generalized to the case in which and are two random vectors, having dimensions and respectively. Denote their joint distribution functions by and and the joint distribution function of and together by Also, if the two vectors are discrete or absolutely continuous replace with or to denote the corresponding probability mass or density functions.
Definition Two random vectors and are independent if and only if one of the following equivalent conditions is satisfied:
Condition 1:for any couple of events and , where and :
Condition 2:for any and (replace with or when the distributions are discrete or absolutely continuous)
Condition 3:for any functions and such that the above expected values exist and are well-defined.
Also the definition of mutual independence extends in a straightforward manner to random vectors:
Definition We say that random vectors , ..., are mutually independent (or jointly independent) if and only if for any sub-collection of random vectors , ..., (where ) and for any collection of events .
All the equivalent conditions for the joint independence of a set of random variables (see above) apply with obvious modifications also to random vectors.
Below you can find some exercises with explained solutions:
Exercise set 1 (verify independence between two random variables).
Most learning materials found on this website are now available in a traditional textbook format.