A square matrix is positive definite if pre-multiplying and post-multiplying it by the same vector always gives a positive number as a result, independently of how we choose the vector.
Positive definite symmetric matrices have the property that all their eigenvalues are positive.
Table of contents
We begin by defining quadratic forms. For the time being, we confine our attention to real matrices and real vectors. At the end of this lecture, we discuss the more general complex case.
Definition Let be a real matrix. A quadratic form in is a transformation where is a vector and is its transpose.
The transformation is a scalar because is a row vector and its product with the column vector gives a scalar as a result.
Example DefineGiven a vector , the quadratic form defined by the matrix is
When we study quadratic forms, we can confine our attention to symmetric matrices without loss of generality.
Remember that a matrix is symmetric if and only if
Any quadratic form can be written aswhere in step we have used the fact that is a scalar and the transpose of a scalar is equal to the scalar itself.
The matrixis symmetric because
Thus, we have proved that we can always write a quadratic form aswhere is symmetric.
Square matrices can be classified based on the sign of the quadratic forms that they define.
In what follows iff stands for "if and only if".
Definition Let be the space of all vectors having real entries. A real matrix is said to be:
positive definite iff for any non-zero ;
positive semi-definite iff for any ;
negative definite iff for any non-zero ;
negative semi-definite iff for any ;
indefinite iff there exist such that and .
Let us make an example.
Example DefineGiven a vector , the quadratic form defined by the matrix isSince the sumwhenever and (hence ), the matrix is positive definite.
We note that many textbooks and papers require that a positive definite matrix be symmetric. We keep the requirement distinct: every time that symmetry is needed, we will explicitly say so.
From now on, we will mostly focus on positive definite and semi-definite matrices. The results obtained for these matrices can be promptly adapted to negative definite and semi-definite matrices. As a matter of fact, if is negative (semi-)definite, then is positive (semi-)definite. Thus, results can often be adapted by simply switching a sign.
An important fact follows.
Proposition Let be a matrix. If is positive definite, then it is full-rank.
The proof is by contradiction. Suppose that is not full-rank. Then its columns are not linearly independent. As a consequence, there is a vector such thatWe can pre-multiply both sides of the equation by and obtainSince is positive definite, this is possible only if , a contradiction. Thus must be full-rank.
The following proposition provides a criterion for definiteness.
Proposition A real symmetric matrix is positive definite if and only if all its eigenvalues are strictly positive real numbers.
Let us prove the "only if" part, starting from the hypothesis that is positive definite. Let be an eigenvalue of and one of its associated eigenvectors. The symmetry of implies that is real (see the lecture on the properties of eigenvalues and eigenvectors). Moreover, can be chosen to be real since a real solution to the equationis guaranteed to exist (because is rank-deficient by the definition of eigenvalue). Then, we havewhere is the norm of . Since is an eigenvector, . Moreover, by the definiteness property of the norm, . Thus, we havebecause by the hypothesis that is positive definite (we have demonstrated above that the quadratic form involves a real vector , which is required in our definition of positive definiteness). We have proved that any eigenvalue of is strictly positive, as desired. Let us now prove the "if" part, starting from the hypothesis that all the eigenvalues of are strictly positive real numbers. Since is real and symmetric, it can be diagonalized as follows:where is orthogonal and is a diagonal matrix having the eigenvalues of on the main diagonal (as proved in the lecture on normal matrices). The eigenvalues are strictly positive, so we can writewhere is a diagonal matrix such that its -th entry satisfiesfor . Therefore, and, for any vector , we haveThe matrix , being orthogonal, is invertible (hence full-rank). The matrix is diagonal (hence triangular) and its diagonal entries are strictly positive, which implies that is invertible (hence full-rank) by the properties of triangular matrices. The product of two full-rank matrices is full-rank. Therefore, is full-rank. Thus,because . By the positive definiteness of the norm, this implies that and, as a consequence,Thus, is positive definite.
A very similar proposition holds for positive semi-definite matrices.
In what follows positive real number means a real number that is greater than or equal to zero.
Proposition A real symmetric matrix is positive semi-definite if and only if all its eigenvalues are positive real numbers.
We do not repeat all the details of the proof and we just highlight where the previous proof (for the positive definite case) needs to be changed. The first change is in the "only if" part, where we now havebecause by the hypothesis that is positive semi-definite. The second change is in the "if part", where we have because the entries of are no longer guaranteed to be strictly positive and, as a consequence, is not guaranteed to be full-rank. It follows that
When the matrix and the vectors are allowed to be complex, the quadratic form becomeswhere denotes the conjugate transpose of .
Let be the space of all vectors having complex entries. A complex matrix is said to be:
positive definite iff is real (i.e., it has zero complex part) and for any non-zero ;
positive semi-definite iff is real (i.e., it has zero complex part) and for any .
The negative definite and semi-definite cases are defined analogously.
Note that conjugate transposition leaves a real scalar unaffected. As a consequence, if a complex matrix is positive definite (or semi-definite), thenfor any , which implies that . In other words, if a complex matrix is positive definite, then it is Hermitian.
Also in the complex case, a positive definite matrix is full-rank (the proof above remains virtually unchanged).
Moreover, since is Hermitian, it is normal and its eigenvalues are real. We still have that is positive semi-definite (definite) if and only if its eigenvalues are positive (resp. strictly positive) real numbers. The proofs are almost identical to those we have seen for the real case. When adapting those proofs, we just need to remember that in the complex case
Below you can find some exercises with explained solutions.
Let be a complex matrix and one of its eigenvectors. Can you write the quadratic form in terms of ?
Let be the eigenvalue associated to . Then,
Can you tell whether the matrix is positive definite?
Let be a vector. Denote its entries by and . Then,Then, if and is positive definite.
Suppose that is a complex negative definite matrix. What can you say about the sign of its eigenvalues?
If is negative definite, thenfor any . As a consequence,In other words, the matrix is positive definite. It follows that the eigenvalues of are strictly positive. If is an eigenvalue of associated to an eigenvector , thenThe latter equation is equivalent toSo, if is an eigenvalue of , then is an eigenvalue of . Thus, the eigenvalues of are strictly negative.
Please cite as:
Taboga, Marco (2017). "Positive definite matrix", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/positive-definite-matrix.
Most of the learning materials found on this website are now available in a traditional textbook format.