An orthonormal basis is a basis whose vectors have unit norm and are orthogonal to each other.

Orthonormal bases are important in applications because the representation of a vector in terms of an orthonormal basis, called Fourier expansion, is particularly easy to derive.

In order to understand this lecture, we need to be familiar with the concepts of inner product and norm.

Recall that two vectors are orthogonal if their inner product is equal to zero.

Definition Let be a vector space equipped with an inner product . A set of vectors is said to be an orthonormal set if and only if

Thus, all vectors in an orthonormal set are orthogonal to each other and have unit norm:

Let us make a simple example.

Example Consider the space of all column vectors having real entries, together with the inner productwhere and denotes the transpose of . Consider the set of two vectors The inner product of with itself isThe inner product of with itself isThe inner product of and isTherefore, and form an orthonormal set.

The next proposition shows a key property of orthonormal sets.

Proposition Let be a vector space equipped with an inner product . The vectors of an orthonormal set are linearly independent.

Proof

The proof is by contradiction. Suppose that the vectors are linearly dependent. Then, there exists scalars , not all equal to zero, such that Thus, for any ,where: in step we have used the additivity and homogeneity of the inner product in its first argument; in step we have used the fact that we are dealing with an orthonormal set, so that if ; in step we have used the fact that the vectors have unit norm. Therefore, all the coefficients must be equal to zero. We have arrived at a contradiction and, as a consequence, the hypothesis that are linearly dependent is false. Hence, they are linearly independent.

If an orthonormal set is a basis for its space, then it is called an orthonormal basis.

Definition Let be a vector space equipped with an inner product . A set of vectors are called an orthonormal basis of if and only if they are a basis for and they form an orthonormal set.

In the next example we show that the canonical basis of a coordinate space is an orthonormal basis.

Example As in the previous example, consider the space of all column vectors having real entries, together with the inner productfor . Let us consider the three vectorswhich constitute the canonical basis of . We can clearly see thatFor instance,andThus, the canonical basis is an orthonormal basis.

It is incredibly easy to derive the representation of a given vector as a linear combination of an orthonormal basis.

Proposition Let be a vector space equipped with an inner product . Let be an orthonormal basis of . Then, for any , we have

Proof

Suppose the unique representation of in terms of the basis iswhere are scalars. Then, for , we have thatwhere: in step we have used the additivity and homogeneity of the inner product in its first argument; in step we have used the fact that we are dealing with an orthonormal basis, so that if ; in step we have used the fact that the vectors have unit norm. Thus, we have found that for any , which proves the proposition.

The linear combination above is called Fourier expansion and the coefficients are called Fourier coefficients.

In other words, we can find the coefficient of by simply calculating the inner product of with .

Example Let be the space of all column vectors with complex entries, together with the inner productwhere and is the conjugate transpose of . Consider the orthonormal basisConsider the vectorThen, the first Fourier coefficient of isand the second Fourier coefficient isWe can check that can indeed be written as a linear combination of the basis with the coefficients just derived:

Below you can find some exercises with explained solutions.

Use the orthonormal basis of two complex vectors introduced in the previous example to derive the Fourier coefficients of the vector

Solution

The first Fourier coefficient is derived by computing the inner product of and :The second Fourier coefficient is found by calculating the inner product of and :

Verify that the Fourier coefficients found in the previous exercise are correct. In particular, check that using them to linearly combine the two vectors of the basis gives as a result.

Solution

The Fourier representation of iswhich is the desired result.

Please cite as:

Taboga, Marco (2017). "Orthonormal basis", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/orthonormal-basis.

The book

Most of the learning materials found on this website are now available in a traditional textbook format.

Featured pages

- Point estimation
- Set estimation
- Chi-square distribution
- Law of Large Numbers
- Conditional probability
- Convergence in probability

Explore

Main sections

- Mathematical tools
- Fundamentals of probability
- Probability distributions
- Asymptotic theory
- Fundamentals of statistics
- Glossary

About

Glossary entries

Share