Search for probability and statistics terms on Statlect
StatLect
Index > Matrix algebra

Orthogonal complement

by , PhD

Given a vector space $S$, the orthogonal complement of a subset $R$ is the subspace of $S$ formed by all the vectors that are orthogonal to the vectors of $R$.

Table of Contents

Definition

Remember that two vectors $s$ and $r$ are said to be orthogonal if their inner product is equal to zero:[eq1]

Definition Let $S$ be a vector space. Let $R$ be a subset of $S$. The orthogonal complement of $R$, denoted by $R^{ot }$, is [eq2]

Let us make a simple example.

Example Let $S$ be the space of all $2	imes 1$ column vectors having real entries. The inner product between two vectors[eq3]is[eq4]Consider the set [eq5] formed by the single vector[eq6]Then, the orthogonal complement of $R$ is[eq7]Thus, $R^{ot }$ is formed by all the vectors $sin S$ whose second entry $s_{2}$ is equal to the first entry $s_{1}$.

Orthogonal complements are subspaces

No matter how the subset $R$ is chosen, its orthogonal complement $R^{ot }$ is a subspace, that is, a set closed with respect to taking linear combinations.

Proposition Let $S$ be a vector space. Let $R$ be a subset of $S$. Then, the orthogonal complement $R^{ot }$ is a subspace of $S$.

Proof

Arbitrarily choose $rin R,$ [eq8] and two scalars $lpha _{1}$ and $lpha _{2}$. Then,[eq9]where: in step $rame{A}$ we have used the linearity of the inner product in in its first argument: in step $rame{B}$ we have used the fact that $s_{1}$ and $s_{2}$ belong to $R^{ot }$ and are therefore orthogonal to each vector of $R$. Thus, we have proved that any linear combination of vectors of $R^{ot }$ is orthogonal to each element of $r$. Hence, it belongs to $R^{ot }$. As a consequence, $R^{ot }$ is closed with respect to taking linear combinations. Thus, it is a subspace.

Complementarity

Remember that, given two subspaces $S_{1}$ and $S_{2}$ of $S$, their sum is the set[eq10]

Moreover, when [eq11], then we say that the sum is direct and we write $S_{1}oplus S_{2}$.

When, additionally, we have that[eq12]then the two subspaces $S_{1}$ and $S_{2}$ are said to be complementary subspaces. In other words, two subspaces are complementary if their direct sum gives the whole vector space $S$ as a result.

It turns out that if $R$ is a subspace, then $R^{ot }$ is one of its complementary subspaces. This is the reason why $R^{ot }$ is called an orthogonal "complement".

Proposition Let $S$ be a finite-dimensional vector space. Let $R$ be a subspace of $S$. Then, the orthogonal complement $R^{ot }$ is a complementary subspace of $R$.

Proof

We first prove that [eq13]that is, any vector $sin S$ can be written as a sum of two vectors, one taken from $R$ and one taken from $R^{ot }$. Since $S$ and, a fortiori, $R$ are finite-dimensional, we can find a basis [eq14] of $R$. By the Gram-Schmidt process, we can transform it into an orthonormal basis [eq15]. Moreover, as explained in the lecture on the Gram-Schmidt process, any vector $sin S$ can be decomposed as follows:[eq16]where $arepsilon _{s}$ is orthogonal to all the vectors of the basis [eq17]. This implies that $arepsilon _{s}$ is orthogonal to every vector of $R$. Therefore, [eq18]. Moreover, the vector[eq19]belongs to $R$, because $R$, being a subspace, contains all linear combinations of its vectors. In other words, we can write any vector $s$ as a sum of a vector of $R$ and a vector of $R^{ot }$. Thus,[eq20]Now, if a vector $r$ belongs to both $R$ and $R^{ot }$, it must be orthogonal to itself, that is,[eq21]But by the definiteness property of the inner product, the only such vector is the zero vector. Therefore, [eq22]and the sum is direct. Thus,[eq23]

In the lecture on complementary subspaces, we have discussed the fact that complements are not necessarily unique, that is, there can be many different complements to a subspace $R$. On the contrary, the orthogonal complement is unique, as $R^{ot }$ is precisely identified by the condition that it must contain all the vectors $s$ that satisfy[eq24]

Double complementation

If we take the orthogonal complement twice, we get back to the original subspace.

Proposition Let $S$ be a finite-dimensional vector space. Let $R$ be a subspace of $S$. Then,[eq25]

Proof

By the definition of $R^{ot }$, any vector $rin R$ is orthogonal to all vectors of $R^{ot }$ and therefore belongs to [eq26]. Thus,[eq27]Now, choose any vector [eq28]. Since $S$ is finite-dimensional and $R$ is a subspace, [eq29] and $t$ has the decomposition[eq30]where $rin R$ and $sin R^{ot }$. We have that[eq31]because $t$, being in [eq32], is orthogonal to all elements of $R^{ot }$. Moreover, [eq33]because $r$, being in $R$, is orthogonal to all elements of $R^{ot }$. Therefore, [eq34]By the definiteness property of the inner product, this implies that $s=0$. Therefore, $t=rin R$. Thus, we have proved that the initial assumption that [eq28] implies that $tin R$. In other words,[eq36]By putting the inclusion relations (1) and (2) together, we obtain [eq37].

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Let $S$ be the space of all $3	imes 1$ column vectors having real entries. Let $R$ be the subspace containing all vectors of the form[eq38]where $lpha $, $eta $ and $gamma $ can be any real numbers satisfying [eq39]. What is the orthogonal complement of $R$? In particular, what constraints do the entries of the vectors in $R^{ot }$ need to satisfy? Can you find a vector that spans $R^{ot }$?

Solution

The vectors of $R$ can be written as[eq40]In other words, $R$ is spanned by the two vectors[eq41]The orthogonal complement $R^{ot }$ contains all the vectors $sin S$ that satisfy[eq42]for any two scalars $lpha $ and $eta $. Since this equation needs to be satisfied for every $lpha $ and $eta $, it must be that[eq43]Denote by [eq44] the three components of $s$:[eq45]Then,[eq46]and[eq47]Thus, the orthogonal complement $R^{ot }$ contains all vectors $s$ whose coordinates [eq44] satisfy the two constraints[eq49]and[eq50]These constraints are satisfied only by the vectors of the form[eq51]In other words, $R^{ot }$ is spanned by the vector[eq52]

How to cite

Please cite as:

Taboga, Marco (2017). "Orthogonal complement", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/orthogonal-complement.

The book

Most of the learning materials found on this website are now available in a traditional textbook format.