Linear combinations

This lecture is about linear combinations of vectors and matrices. Linear combinations are obtained by multiplying matrices by scalars, and by adding them together. Therefore, in order to understand this lecture you need to be familiar with the concepts introduced in the lectures on Matrix addition and Multiplication of a matrix by a scalar.

Table of contents

Definition
Linear combinations of vectors
Solved exercises

Definition

Let us start by giving a formal definition of linear combination.

Definition Let $A_{1}, \dots, A_{n}$ be $n$ matrices having dimension $K \times L$ . A $K \times L$ matrix $B$ is a linear combination of $A_{1}, \dots, A_{n}$ if and only if there exist $n$ scalars $α_{1}, \dots, α_{n}$ , called coefficients of the linear combination, such that

B = α_{1} A_{1} + \dots + α_{n} A_{n}

In other words, if you take a set of matrices, you multiply each of them by a scalar, and you add together all the products thus obtained, then you obtain a linear combination.

Note that all the matrices involved in a linear combination need to have the same dimension (otherwise matrix addition would not be possible).

Example Let $A_{1}$ and $A_{2}$ be $2 \times 2$ matrices defined as follows:

A_{1} A_{2} = 0121 = 3101

Let $α_{1} = 2$ and $α_{2} = - 1$ be two scalars. Then, the matrix

B = α_{1} A_{1} + α_{2} A_{2}

is a linear combination of $A_{1}$ and $A_{2}$ . It is computed as follows:

B = α_{1} A_{1} + α_{2} A_{2} = 2 \cdot 0121 - 1 \cdot 3101 = 2 \cdot 0 2 \cdot 1 2 \cdot 2 2 \cdot 1 + - 1 \cdot 3 - 1 \cdot 1 - 1 \cdot 0 - 1 \cdot 1 = 0242 + - 3 - 1 0 - 1 = 0 - 3 2 - 1 4 + 0 2 - 1 = - 3 1 41

Linear combinations of vectors

Most of the times, in linear algebra we deal with linear combinations of column vectors (or row vectors), that is, matrices that have only one column (or only one row).

Example Let $A_{1}$ , $A_{2}$ and $A_{3}$ be $3 \times 1$ column vectors defined as follows:

A_{1} = 012 A_{2} = 10 - 1 A_{3} = 100

Let $B$ be another column vector defined as

B = 111

Is $B$ a linear combination of $A_{1}$ , $A_{2}$ and $A_{3}$ ? In order to answer this question, note that a linear combination of $A_{1}$ , $A_{2}$ and $A_{3}$ with coefficients $α_{1}$ , $α_{2}$ and $α_{3}$ has the following form:

α_{1} A_{1} + α_{2} A_{2} + α_{3} A_{3} = α_{1} \cdot 012 + α_{2} \cdot 10 - 1 + α_{3} \cdot 100 = α_{1} \cdot 0 + α_{2} \cdot 1 + α_{3} \cdot 1 α_{1} \cdot 1 + α_{2} \cdot 0 + α_{3} \cdot 0 α_{1} \cdot 2 + α_{2} \cdot (- 1) + α_{3} \cdot 0 = α_{2} + α_{3} α_{1} 2 α_{1} - α_{2}

Now, $B$ is a linear combination of $A_{1}$ , $A_{2}$ and $A_{3}$ if and only if we can find $α_{1}$ , $α_{2}$ and $α_{3}$ such that

B = α_{1} A_{1} + α_{2} A_{2} + α_{3} A_{3}

which is equivalent to

111 = α_{2} + α_{3} α_{1} 2 α_{1} - α_{2}

But we know that two vectors are equal if and only if their corresponding elements are all equal to each other. This means that the above equation is satisfied if and only if the following three equations are simultaneously satisfied:

⎩ ⎨ ⎧ α_{2} + α_{3} = 1 α_{1} = 1 2 α_{1} - α_{2} = 1

The second equation gives us the value of the first coefficient:

α_{1} = 1

By substituting this value in the third equation, we obtain

α_{2} = 1

Finally, by substituting the value of $α_{2}$ in the first equation, we get

α_{3} = 0

You can easily check that these values really constitute a solution to our problem:

1 \cdot A_{1} + 1 \cdot A_{2} + 0 \cdot A_{3} = 1 \cdot 012 + 1 \cdot 10 - 1 + 0 \cdot 100 = 1 \cdot 0 + 1 \cdot 1 + 0 \cdot 1 1 \cdot 1 + 1 \cdot 0 + 0 \cdot 0 1 \cdot 2 + 1 \cdot (- 1) + 0 \cdot 0 = 111 = B

Therefore, the answer to our question is affirmative.

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Define two $3 \times 2$ matrices $A_{1}$ and $A_{2}$ as follows:

A_{1} A_{2} = 024204 = 115100

Let $α_{1} = 1/2$ and $α_{2} = 1$ be two scalars. Compute the linear combination

B = α_{1} A_{1} + α_{2} A_{2}

Solution

It is computed as follows:

B = α_{1} A_{1} + α_{2} A_{2} = \frac{1}{2} \cdot 024204 + 1 \cdot 115100 = \frac{1}{2} \cdot 0 \frac{1}{2} \cdot 2 \frac{1}{2} \cdot 4 \frac{1}{2} \cdot 2 \frac{1}{2} \cdot 0 \frac{1}{2} \cdot 4 + 1 \cdot 1 1 \cdot 1 1 \cdot 5 1 \cdot 1 1 \cdot 0 1 \cdot 0 = 012102 + 115100 = 0 + 1 1 + 1 2 + 5 1 + 1 0 + 0 2 + 0 = 127202

Exercise 2

Let $A_{1}$ and $A_{2}$ be $2 \times 1$ vectors:

A_{1} = 21 A_{2} = 01

Compute the value of the linear combination

3 A_{1} - 2 A_{2}

Solution

This is done as follows:

3 A_{1} - 2 A_{2} = 3 \cdot 21 - 2 \cdot 01 = 3 \cdot 2 3 \cdot 1 + - 2 \cdot 0 - 2 \cdot 1 = 63 + 0 - 2 = 6 + 0 3 - 2 = 61

Exercise 3

Let $A$ be the following $3 \times 2$ matrix:

A = 210201

Is the $2 \times 1$ zero vector

[00]

a linear combination of the rows of $A$ ?

Solution

Denote the rows of $A$ by $A_{1}$ , $A_{2}$ and $A_{3}$ . A linear combination of $A_{1}$ , $A_{2}$ and $A_{3}$ with coefficients $α_{1}$ , $α_{2}$ and $α_{3}$ can be written as

α_{1} A_{1} + α_{2} A_{2} + α_{3} A_{3} = α_{1} \cdot [22] + α_{2} \cdot [10] + α_{3} \cdot [01] = [α_{1} \cdot 2 + α_{2} \cdot 1 + α_{3} \cdot 0 α_{1} \cdot 2 + α_{2} \cdot 0 + α_{3} \cdot 1] = [2 α_{1} + α_{2} 2 α_{1} + α_{3}]

Now, the $2 \times 1$ zero vector is a linear combination of $A_{1}$ , $A_{2}$ and $A_{3}$ if and only if there exist coefficients $α_{1}$ , $α_{2}$ and $α_{3}$ such that

[00] = α_{1} A_{1} + α_{2} A_{2} + α_{3} A_{3}

which is the same as

[00] = [2 α_{1} + α_{2} 2 α_{1} + α_{3}]

Because two vectors are equal if and only if their corresponding entries are all equal to each other, this equation is satisfied if and only if the following system of two equations is satisfied:

⎩ ⎨ ⎧ 2 α_{1} + α_{2} = 0 2 α_{1} + α_{3} = 0

This can be rewritten as

⎩ ⎨ ⎧ α_{2} = - 2 α_{1} α_{3} = - 2 α_{1}

This means that, whatever value we choose for $α_{1}$ , the system is satisfied provided we set $α_{2} = - 2 α_{1}$ and $α_{3} = - 2 α_{1}$ . For example, if we choose $α_{1} = 0$ , then we need to set

α_{2} = - 2 \cdot 0 = 0 α_{3} = - 2 \cdot 0 = 0

Therefore, one solution is

⎩ ⎨ ⎧ α_{1} = 0 α_{2} = 0 α_{3} = 0

If we choose a different value, say $α_{1} = 1$ , then we have a different solution:

⎩ ⎨ ⎧ α_{1} = 1 α_{2} = - 2 α_{3} = - 2

In the same manner, you can obtain infinitely many solutions by choosing different values of $α_{1}$ and changing $α_{2}$ and $α_{3}$ accordingly. You can easily check that any of these linear combinations indeed give the zero vector as a result. For example, the solution proposed above ( $α_{1} = 1$ , $α_{2} = - 2$ , $α_{3} = - 2$ ) gives

1 \cdot A_{1} - 2 \cdot A_{2} - 2 \cdot A_{3} = 1 \cdot [22] - 2 \cdot [10] - 2 \cdot [01] = [1 \cdot 2 - 2 \cdot 1 - 2 \cdot 0 1 \cdot 2 - 2 \cdot 0 - 2 \cdot 1] = [00]

How to cite

Please cite as:

Taboga, Marco (2021). "Linear combinations", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/linear-combinations.