This lecture introduces the concepts of eigenvalues and eigenvectors of a square matrix. These are amongst the most useful concepts in linear algebra: studying the eigenvalues and eigenvectors of a square matrix is very frequent in applied work.

Let us first develop some intuition about eigenvalues and eigenvectors. To do so, we start from some concepts we explained in the lecture on the Determinant of a matrix.

Consider the linear space of all real vectors, which can be represented as a Cartesian plane. A vector is a point in the plane, and the first an the second entries of are the - and -coordinates of the point respectively. Now, consider a matrix . The matrix transforms any set of points (e.g., a rectangle, a circle) into another set of points :

If the points of form a region whose area is equal to , and the area of the transformed region is , then

In other words, the determinant tells us by how much the linear transformation
associated with the matrix
scales up or down the area of shapes. Eigenvalues and eigenvectors provide us
with another useful piece of information. **They tell us by how much the
linear transformation scales up or down the sides of certain
parallelograms**.

Consider the parallelograms that have one vertex at the origin of the Cartesian plane. The four vertices arewhere and are vectors and is the zero vector. There are two vectors and , called the eigenvectors of , such that the associated parallelogram is transformed by into a new parallelogram having verticeswhere and are two scalars called the eigenvalues of . In other words, the linear transformation multiplies the length of one pair of parallel sides by and the length of the other pair by , but it keeps the angles of the parallelogram unchanged. The next figure provides an illustration of this kind of transformation: the original parallelogram (in blue) is transformed into another parallelogram (in red) by a matrix whose eigenvalues are equal to and .

Since a pair of parallel sides is scaled by and the other pair by , the area of the parallelogram is scaled by a factor of . But we also know that the area of the parallelogram is scaled by . As a consequence,that is, the determinant of a matrix is equal to the product of its eigenvalues, a fact that holds in general.

The definition of eigenvalues and eigenvectors we are going to provide below generalizes these concepts to linear spaces that can have more than two dimensions.

We are now ready to define eigenvalues and eigenvectors.

Definition Let be a matrix. If there exist a vector and a scalar such thatthen is called an eigenvalue of and an eigenvector corresponding to .

This definition fits with the example above about the vertices of the parallelogram. The two vertices and are eigenvectors corresponding to the eigenvalues and becauseFurthermore, these two equations can be added so as to obtain the transformation of the vertex :

Note that the eigenvalue
equationcan
be written
aswhere
is the
identity matrix. The latter
equation has a non-zero solution only if the columns of the matrix
are linearly dependent,
that is, if the matrix is
singular. But
a matrix is singular if and
only if its determinant is zero. As a consequence, any eigenvalue of
must satisfy the
equationwhich
is called **characteristic equation**.

By using the fundamental theorem of algebra, it is possible to prove that the characteristic equation can be written aswhere are the solutions of the equation. The fundamental theorem of algebra guarantees that exactly solutions exist, but these solutions are not guaranteed to be real (i.e., they can be complex numbers), even when the entries of are all real. They are also not guaranteed to be distinct, that is, two solutions could be equal.

In the previous section we have explained that a matrix has not necessarily distinct and possibly complex eigenvalues. The set of all eigenvalues of is called the spectrum of .

Note that ifthen you can multiply both sides of the equation by a non-zero scalar and get In other words, if is an eigenvalue of and is an eigenvector corresponding to , then any multiple of is an eigenvector corresponding to . Thus, the eigenvector corresponding to a given eigenvalue is not unique. In this section we prove that the set of all eigenvectors corresponding to a given eigenvalue is a linear space.

Definition Let be a matrix and one of its eigenvalues. The union of the zero vector and the set of all the eigenvectors corresponding to the eigenvalue is called the eigenspace of .

Note that we include the zero vector in the eigenspace because eigenvectors are required to be non-zero.

The next proposition shows that an eigenspace is closed with respect to linear combinations, that is, it is a linear space.

Proposition The eigenspace corresponding to an eigenvalue is a linear space.

Proof

Suppose that is an eigenvalue of a square matrix and take any two vectors and belonging to the eigenspace of . Then,Now take a linear combination of the two eigenvectorswhere and are two scalars. Then,Thus, is an eigenvector corresponding to . In other words, any linear combination of the vectors of the eigenspace belongs to the eigenspace.

Clearly, once an eigenvalue has been found (e.g., by solving the characteristic equation), the eigenspace of can be found by solving the linear system

Below you can find some exercises with explained solutions.

Consider the matrix Show that is an eigenvector of and find its corresponding eigenvalue.

Solution

We have thatThus, is an eigenvector of corresponding to the eigenvalue .

DefineFind the eigenvalues of by solving the characteristic equation.

Solution

The characteristic equation isTherefore, the eigenvalues of are and .

The book

Most of the learning materials found on this website are now available in a traditional textbook format.

Featured pages

- Multivariate normal distribution
- Student t distribution
- Binomial distribution
- Mean square convergence
- Beta distribution
- Maximum likelihood

Explore

Main sections

- Mathematical tools
- Fundamentals of probability
- Probability distributions
- Asymptotic theory
- Fundamentals of statistics
- Glossary

About

Glossary entries

- Binomial coefficient
- Estimator
- Integrable variable
- Distribution function
- Mean squared error
- Probability mass function

Share