A basis is orthonormal if its vectors:
have unit norm;
are orthogonal to each other (i.e., their inner product is equal to zero).
The representation of a vector as a linear combination of an orthonormal basis is called Fourier expansion. It is particularly important in applications.
Let us start by defining orthonormality for a set of vectors (not necessarily a basis).
Definition Let be a vector space equipped with an inner product . A set of vectors is said to be an orthonormal set if and only if
Thus, all vectors in an orthonormal set are orthogonal to each other and have unit norm:
Let us make a simple example.
Example Consider the space of all column vectors having real entries, together with the inner productwhere and denotes the transpose of . Consider the set of two vectors The inner product of with itself isThe inner product of with itself isThe inner product of and isTherefore, and form an orthonormal set.
The next proposition shows a key property of orthonormal sets.
Proposition Let be a vector space equipped with an inner product . The vectors of an orthonormal set are linearly independent.
The proof is by contradiction. Suppose that the vectors are linearly dependent. Then, there exist scalars , not all equal to zero, such that Thus, for any ,where: in step we have used the additivity and homogeneity of the inner product in its first argument; in step we have used the fact that we are dealing with an orthonormal set, so that if ; in step we have used the fact that the vectors have unit norm. Therefore, all the coefficients must be equal to zero. We have arrived at a contradiction and, as a consequence, the hypothesis that are linearly dependent is false. Hence, they are linearly independent.
If an orthonormal set is a basis for its space, then it is called an orthonormal basis.
Definition Let be a vector space equipped with an inner product . A set of vectors are called an orthonormal basis of if and only if they are a basis for and they form an orthonormal set.
In the next example we show that the canonical basis of a coordinate space is an orthonormal basis.
Example As in the previous example, consider the space of all column vectors having real entries, together with the inner productfor . Let us consider the three vectorswhich constitute the canonical basis of . We can clearly see thatFor instance,andThus, the canonical basis is an orthonormal basis.
It is incredibly easy to derive the representation of a given vector as a linear combination of an orthonormal basis.
Proposition Let be a vector space equipped with an inner product . Let be an orthonormal basis of . Then, for any , we have
Suppose the unique representation of in terms of the basis iswhere are scalars. Then, for , we have thatwhere: in step we have used the additivity and homogeneity of the inner product in its first argument; in step we have used the fact that we are dealing with an orthonormal basis, so that if ; in step we have used the fact that the vectors have unit norm. Thus, we have found that for any , which proves the proposition.
The linear combination above is called Fourier expansion and the coefficients are called Fourier coefficients.
In other words, we can find the coefficient of by simply calculating the inner product of with .
Example Let be the space of all column vectors with complex entries, together with the inner productwhere and is the conjugate transpose of . Consider the orthonormal basisConsider the vectorThen, the first Fourier coefficient of isand the second Fourier coefficient isWe can check that can indeed be written as a linear combination of the basis with the coefficients just derived:
Below you can find some exercises with explained solutions.
Use the orthonormal basis of two complex vectors introduced in the previous example to derive the Fourier coefficients of the vector
The first Fourier coefficient is derived by computing the inner product of and :The second Fourier coefficient is found by calculating the inner product of and :
Verify that the Fourier coefficients found in the previous exercise are correct.
In particular, check that using them to linearly combine the two vectors of the basis gives as a result.
The Fourier representation of iswhich is the desired result.
Please cite as:
Taboga, Marco (2021). "Orthonormal basis", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/orthonormal-basis.
Most of the learning materials found on this website are now available in a traditional textbook format.