The norm is a function, defined on a vector space, that associates to each vector a measure of its length. In abstract vector spaces, it generalizes the notion of length of a vector in Euclidean spaces.
There is a tight connection between norms and inner products, as every inner product can be used to induce a norm on its space.
Before reading this lecture you should be familiar with inner products and with the basics of the algebra of complex vectors and matrices.
Table of contents
We are going to give an abstract, axiomatic definition of norm. Later we will show some examples of norm to clarify its meaning.
Definition Let be a vector space. A norm on is a function that associates to each a positive real number, denoted by , which has the following properties.
Definiteness:
Absolute homogeneity:where is the field over which the vector space is defined (i.e., the set of scalars used for scalar multiplication); denotes the absolute value if and the modulus if .
Triangle inequality:
These properties are pretty intuitive.
As the norm is a measure of the length of a vector, it is reasonable to require that it should always be a positive number.
The definiteness property imposes that all vectors except the zero vector should have a strictly positive length.
Absolute homogeneity means that if you scale up (or down) a vector by a factor , then its length is re-scaled accordingly.
Finally, in the triangle inequality, is to be interpreted as the side of a triangle, while and are the other two sides. A well-known fact from geometry is that the length of one side of a triangle is less than the sum of the lengths of the other two sides. The triangle inequality axiom extends this property to the notion of length in an abstract vector space.
Before providing some examples of normed vector spaces, we need to introduce an important connection between inner products and norms.
Definition Let be a vector space and an inner product on . Then, the function defined byis called an induced norm on .
We will prove that an induced norm indeed satisfies all the properties of a norm. But before doing that, we need to prove some preliminary results.
We highly recommend reading the proofs of the next three propositions as not only they are very short and simple to understand, but also they are a good way to hone our skills in dealing with norms and inner products.
In order to understand the following generalization of the well-known Pythagoras' theorem, we need to remember that two vectors are said to be orthogonal if and only if their inner product is equal to zero.
Proposition Let be a vector space equipped with an inner product and its induced norm . If two vectors are orthogonal (i.e., ), then
The proof is as follows:where: in step we have used the additivity of the inner product; in step we have used the orthogonality of and ; in step we have used the definition of induced norm.
Pythagoras' theorem says that the squared length of the hypotenuse () of a right triangle is equal to the sum of the squared lengths of the other two sides of the triangle (). The triangle is right because we assume orthogonality (a 90 degrees angle) between the two sides and .
Given two vectors, we can always write the first as a scalar multiple of the second plus a third vector orthogonal to the second.
Proposition Let be a vector space equipped with an inner product and its induced norm . Given two vectors , if , then there exist a scalar and a vector such that
It suffices to verify the first equality:and the second one:where: in step we have used the additivity and homogeneity in the first argument of the inner product; in step we have used the definition of induced norm.
The inequality presented in the next proposition is known as Cauchy-Schwarz inequality.
Proposition Let be a vector space equipped with an inner product and its induced norm . For any two vectors , the following inequality holds:
If , then , and, as a consequence, the inequality holds. We can thus focus on the case in which . In this case, we can use the orthogonal decomposition shown in the previous section:where is orthogonal to . Thanks to the orthogonality of , we can apply Pythagoras' theorem:Since , the equality just derived implies thatwhich becomes, by taking square roots on both sides,If we multiply both sides by , we obtained the desired inequality.
We now have almost all the tools that we need to prove that a norm induced by an inner product indeed satisfies all the properties of a norm.
We just need to remember a couple of facts about complex numbers.
First, if we add to a complex numberits complex conjugatewe getwhere is the so-called real part of the complex number.
Second, the modulus of satisfies the inequality because
We are now good to go!
Proposition Let be a vector space and an inner product on . Then, the norm induced by the inner productis a norm on .
Let us first prove positivity and definiteness. For any , by the positivity of inner products, we have thatand by the definiteness of inner products, we have thatNow, let us prove absolute homogeneity. Suppose is defined over the complex field . Then, for every , we have where: in step we have used the homogeneity in the first argument of the inner product; in step we have used the conjugate homogeneity in the second argument of the inner product ( is the complex conjugate of ); in step denotes the modulus of . Absolute homogeneity holds also when the vector space is over the real field because in that case, so that is the absolute value of . Finally, let us prove the triangle inequality. For any , we have thatwhere: in step we have used the additivity of the inner product in its two arguments; in step we have used the definition of induced norm; in step we have used the conjugate symmetry of the inner product; in step and we have used the elementary properties of complex numbers discussed in the previous section; in step we have used the Cauchy-Schwarz inequality. By taking the square root of both sides of the inequality derived above, we obtain
Wow! This was a long sequence of theoretical facts without ever presenting a single example. The wait was worthwhile because we can now make examples involving norms induced by inner products, which are the most practically relevant in linear algebra.
In this example we are going to discuss the norm of a column vector having real entries.
Let be the space of all real vectors (on the real field ).
In the lecture on inner products, we have explained that the most common way of defining an inner product between two real vectors iswhere is the transpose of , are the entries of and are the entries of .
The norm induced by this inner product is
Thus, the norm of a real vector is equal to the square root of the sum of the squares of its entries. This is the ordinary way to compute the length of a vector in Euclidean space.
Example DefineThen
Another important example is the norm of a column vector having complex entries.
Let be the space of all complex vectors (on the complex field ).
In the lecture on inner products, the inner product between two vectors was defined to bewhere is the conjugate transpose of , are the entries of and are the complex conjugates of the entries of .
The norm induced by this inner product iswhere we have used the fact that the modulus is equal to the square root of times its complex conjugate .
Thus, the formula for the norm of a complex vector is not very different from the formula for real vectors: we just need to replace the absolute values of the entries of the vector with their moduli.
Example DefineThen
Below you can find some exercises with explained solutions.
Let be the space of all real vectors.
For any , define where are the entries of . Show that is a norm.
The function is positive since it is the sum of positive terms . It is definite because if and only if for , which happens if and only if . Absolute homogeneity is satisfied becauseNote that for any couple of real numbers , we haveTherefore, for any two vectors so that the triangle inequality holds. Thus, all the properties of a norm are satisfied.
Let be the space of all real vectors.
For any , define where are the entries of . Show that is a norm.
The function is positive since it is the maximum over a set of positive terms . It is definite because if and only if for , which happens if and only if . Absolute homogeneity is satisfied becauseNote that given two real numbers , we haveFor any couple of vectors , we haveso that the triangle inequality holds. Thus, all the properties of a norm are satisfied.
Please cite as:
Taboga, Marco (2017). "Norm of a vector", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/vector-norm.
Most of the learning materials found on this website are now available in a traditional textbook format.