So far in this chapter, we have restricted our attention to vectors in two or three dimensions, where we are able to visualize things geometrically. Something that you may have noticed is that the algebraic rules for the addition and scalar multiplication vectors are the same in both dimensions.
In this section, we consider this algebra of vectors abstractly, which will allow us to move beyond three dimensions to vectors with an arbitrary number of components.
Despite the fact that we cannot easily visualize them, higher-dimensional vectors frequently arise in applications where there are many variables involved. The fact that the rules of algebra remain the same mean that we can continue to manipulate these objects, even though we can no longer picture them.
For each positive integer \(n\text{,}\) a column vector \(\vec{x}\) is formed by arranging \(n\) real numbers \(x_1, x_2, \ldots, x_n\) into a column \(\vec{x} = \bbm x_1\\x_2\\\vdots \\x_n\ebm\text{.}\) In Chapter 4 we will see that this is a special type of matrix; for now, we can think of a column vector as an alternative to the notation \(\bbm x_1\\x_2\\\ldots, x_n\ebm\) encountered earlier in this chapter. In particular, we still refer to the numbers \(x_1,x_2,\ldots, x_n\) in \(\vec{x}\) as the components of \(\vec{x}\text{,}\) and we define addition and scalar multiplication of column vectors in terms of their components, as we did for vectors in \(\mathbb{R}^2\) and \(\mathbb{R}^3\text{.}\)
That is, given vectors \(\vec{x} = \bbm x_1\\x_2\\\vdots\\x_n\ebm\) and \(\vec{y} = \bbm y_1\\y_2\\\vdots \\y_n\ebm\text{,}\) and a scalar \(c\text{,}\) we define
As with the vectors in \(\mathbb{R}^2\) and \(\mathbb{R}^3\) we encountered in Chapter 2, we allow the notation \(\mathbb{R}^n\) to represent both the space of points \((x_1,x_2, \ldots, x_n)\text{,}\) and the set of vectors defined within that space. Since we can identify any point \(P\) with the position vector \(\overrightarrow{OP}\text{,}\) the difference between viewing \(\mathbb{R}^n\) as a set of points or as a set of vectors is primarily one of perspective.
When \(n\geq 4\) we can no longer visualize vectors in \(\mathbb{R}^n\) as we did in Chapter 2, but we can handle them algebraically exactly as we did before, and we can extend the definitions of Chapter 2 to apply to vectors in \(\mathbb{R}^n\text{.}\)
Having defined the dot product, we can still declare two vectors \(\vec x\) and \(\vec y\) to be orthogonal if \(\dotp xy = 0\text{,}\) and define the angle between two vectors by requiring that the identity
remain valid. Using these definitions, along with Theorem 4.1.10, we can see that all of the properties of vector operations given in Theorem 2.2.16 remain valid in \(\mathbb{R}^n\text{.}\)
Then ten properties listed in Theorem 2.7.2 are known as the vector space axioms. Any set of objects satisfying these axioms is known as a vector space. There are many interesting examples of vector spaces other than \(\mathbb{R}^n\text{,}\) but we will not study vector spaces in general in this text.
One of the key insights of linear algebra is that a space such as \(\mathbb{R}^n\text{,}\) which contains infinitely many objects, can be generated using the operations of addition and scalar multiplication from a finite set of basic objects. We saw in Chapter 2, for example, that every vector in \(\mathbb{R}^3\) can be written in terms of just three basic unit vectors \(\veci\text{,}\)\(\vecj\text{,}\) and \(\veck\text{.}\)
Since addition and scalar multiplication are the main operations of linear algebra, it’s not too surprising (if a little unimaginative) that any combination of these operations is called a linear combination.
Let \(\vec{u} = \bbm 2\\-1\\3\ebm, \vec{v} = \bbm -4\\ 6\\ 3\ebm, \vec{w} = \bbm 2\\3\\12\ebm\) be vectors in \(\mathbb{R}^3\text{.}\) Form the following linear combinations:
To simplify the linear combination, we first take care of the scalar multiplication, and then perform the addition. (We choose to interpret this expression as \(3\vec{u}+(-4)\vec{w}\text{,}\) and multiply by \(-4\) in the first step, and add in the second step, rather than multiplying by 4 and then subtracting.)
Notice that in the last example above, our linear combination works out to be the zero vector. Let’s think about this geometrically for a second: using the “tip-to-tail” method for adding vectors and beginning with the tail of \(3\vec{u}\) at the origin, if we add the vector \(\vec{v}\) at the tip of \(3\vec{u}\text{,}\) and then subtract \(\vec{w}\text{,}\) we end up back at the origin. The vectors \(3\vec{u}\text{,}\)\(\vec{v}\text{,}\) and \(\vec{w}\) must therefore lie in the same plane, since they form three sides of a triangle, as depicted in Figure 2.7.5.
What this tells us is that when we’re being asked to form linear combinations of the vectors \(\vec{u}, \vec{v}\text{,}\) and \(\vec{w}\) in Example 2.7.4, then vector \(\vec{w}\) is redundant. Suppose the vector \(\vec{x}\) is an arbitrary linear combination of these vectors; that is,
These ideas come up frequently enough in Linear Algebra that they have associated terminology. The definitions that follow seem innocent enough, but their importance to the theory of Linear Algebra cannot be understated.
Let \(A = \{\vec{v}_1, \vec{v}_2, \ldots, \vec{v}_k\}\) be a set of vectors in \(\mathbb{R}^n\text{.}\) The span of the vectors in \(A\text{,}\) denoted \(\operatorname{span}(A)\text{,}\) is the set \(S\) of all possible linear combinations of the vectors in \(A\text{.}\) That is,
If we think back to Section 2.5, we can do a bit better with our description. The set \(\operatorname{span}\{\vec{u}\}\) consists of all vectors \(\bbm x\\y\\z\ebm\) such that
so \(\operatorname{span}\{\vec{u},\vec{v}\}\) consists of all vectors of the form \(\bbm 2s-4t\\-s+6t\\3s+3t\ebm\text{,}\) where \(s\) and \(t\) can be any real numbers. Again, with a bit of thought, we can come up with a geometric description of this set. Consider an arbitrary vector
Any such vector can be obtained by moving some distance (measured by the scalar \(s\)) in the direction of \(\vec{u}\text{,}\) and then moving another distance (measured by the scalar \(t\)) in the direction of \(\vec{v}\text{.}\) We now have two directions in which to move, and if we haven’t forgotten what we learned in Section 2.6, this probably reminds us of the description of a plane.
which we know is orthogonal to both \(\vec{u}\) and \(\vec{v}\text{.}\) It follows from the properties of the dot product that for any other vector \(\vec{x}=s\vec{u}+t\vec{v}\) we have
In the discussion following Example 2.7.4 we saw that any vector that can be written as a linear combination of \(\vec{u}\text{,}\)\(\vec{v}\text{,}\) and \(\vec{w}\) can be written as a linear combination of \(\vec{u}\) and \(\vec{v}\) alone. Thus, the span of \(\vec{u}, \vec{v}\text{,}\) and \(\vec{w}\) doesn’t contain anything we didn’t already have in the span of \(\vec{u}\) and \(\vec{v}\text{;}\) that is,
Given the vectors \(\vec{u} = \bbm 2\\-1\\1\ebm\text{,}\)\(\vec{v} = \bbm 3\\2\\5\ebm\text{,}\) and \(\vec{w} = \bbm -2\\5\\3\ebm\text{,}\) determine whether or not the following vectors belong to \(\operatorname{span}\{\vec{u},\vec{v},\vec{w}\}\text{:}\)
We do not yet have a general technique for solving problems of this type. Notice that the question “Does \(\vec{x}\) belong to the span of \(\{\vec{u},\vec{v},\vec{w}\}\text{?}\)” is equivalent to the question, “Do there exist scalars \(a,b,c\) such that
We will develop systematic techniques for solving such systems in the next chapter. Until then, is there anything we can say? The very astute reader might notice that the vectors \(\vec{u},\vec{v}, \vec{w}\) all have something in common: their third component is the sum of the first two: \(1=2+(-1)\) for \(\vec{u}\text{,}\)\(5=3+2\) for \(\vec{v}\text{,}\) and \(3=-2+5\) for \(\vec{w}\text{.}\) Thus, all three vectors are of the form \(\bbm x\\y\\x+y\ebm\text{.}\) Now, notice what happens if we combine two such vectors:
For the vector \(\vec{x}\text{,}\) we check that \(3+6=9\text{,}\) so \(\vec{x}\) has the correct form, and indeed (with a bit of “guess and check” work), we find that \(a=b=c=1\) works, since \(\vec{u}+\vec{v}+\vec{w}=\vec{x}\text{.}\) Thus, we can conclude that
For the vector \(\vec{y}\text{,}\) we add the first two components, getting \(4+1=5\neq -3\text{.}\) Since the third component is not the sum of the first two, there is no way that \(\vec{y}\) could belong to the span of \(\vec{u}\text{,}\)\(\vec{v}\text{,}\) and \(\vec{w}\text{.}\)
Notice in Example 2.7.7 that the span did not change when we added the vector \(\vec{w}\) to the set of spanning vectors. This was probably not too surprising, since we saw that \(\vec{w} = 3\vec{u}+\vec{v}\text{,}\) meaning that \(\vec{w}\) is a linear combination of \(\vec{u}\) and \(\vec{v}\text{,}\) and thus,
We don’t get anything new when we include the vector \(\vec{w}\) since it lies in the plane spanned by \(\vec{u}\) and \(\vec{v}\text{.}\) We say that the vector \(\vec{w}\)depends on \(\vec{u}\) and \(\vec{v}\text{,}\) in the same way that the total of a sum depends on the numbers being added. Since this dependence is defined in terms of linear combinations, we say that the vectors \(\vec{u},\vec{v},\vec{w}\) are linearly dependent.
In general, a set of vectors \(\vec{v}_1,\vec{v}_2\ldots, \vec{v}_k\) is linearly dependent of one of the vectors can be written as a linear combination of the others. If this is impossible, we say that the vectors are linearly independent.
\begin{equation*}
A = \{\vec{v}_1,\vec{v}_2,\ldots, \vec{v}_k\}
\end{equation*}
in \(\mathbb{R}^n\) is linearly dependent if \(\vec{0}\in A\text{,}\) or if one of the vectors \(\vec{v}_i\) in \(A\) can be written as a linear combination of the other vectors in \(A\text{.}\) If the set \(A\) is not linearly dependent, we say that it is linearly independent.
Like problems involving span, a general approach to answering questions like these about linear independence will have to wait until we develop methods for solving systems of equations in the next chapter. However, for these two sets of vectors, we can reason our way to an answer.
Here, we noticed that all three vectors satisfy the condition \(z=x+y\text{,}\) if we label their respective components as \(x\text{,}\)\(y\text{,}\) and \(z\text{.}\) But this condition is simply the equation of a plane; namely, \(x+y-z=0\text{.}\) Intuition tells us that any plane can be written as the span of two vectors, so we can expect that any one of the three vectors can be written in terms of the other two, and indeed, this is the case. With a bit of guesswork (or by asking a computer), we can determine that
showing that \(\vec{w}\) can be written as a linear combination of \(\vec{u}\) and \(\vec{v}\text{,}\) and thus, that our vectors are linearly dependent.
Here, we make the useful observation that two of our three vectors have zero as their third component. Since \(\vec{x}\) and \(\vec{z}\) have third component zero, it is impossible for \(\vec{y}\) to be written as a linear combination of \(\vec{x}\) and \(\vec{z}\text{,}\) since any such linear combination would still have a zero in the third component. To see that \(\vec{x}\) cannot be written in terms of \(\vec{y}\) and \(\vec{z}\text{,}\) notice that for any \(a\) and \(b\text{,}\)
If this is to equal \(\vec{x}\text{,}\) then we must have \(a=0\text{,}\) giving us \(\vec{x}=b\vec{z}\text{,}\) but it’s clear that \(\vec{x}\) is not a scalar multiple of \(\vec{z}\text{.}\) A similar argument shows that \(\vec{z}\) cannot be written in terms of \(\vec{x}\) and \(\vec{y}\text{,}\) and thus our vectors are linearly independent.
This is always possible of course, since we can simply set each of the scalars equal to zero. The condition of linear independence tells us that if our vectors are independent, then this is the only way to obtain a linear combination equal to the zero vector.
To see why this rule works, suppose we can choose our scalars in Equation (2.7.1) so that at least one of them is non-zero. For simplicity, let’s say \(c_1\neq 0\text{.}\) Then we can rewrite Equation (2.7.1) as
and since \(c_1\neq 0\text{,}\) we can divide both sides by \(c_1\text{,}\) and we’ve written \(\vec{v}_1\) as a linear combination of the remaining vectors.
For example, from Example 2.7.10 we can conclude (with a bit of rearranging) that the vectors \(\vec{u}\text{,}\)\(\vec{v}\text{,}\) and \(\vec{w}\) satisfy the relationship
Linear independence can be a difficult concept at first, but in three dimensions we can use Equation (2.7.1) to provide a visual interpretation on a case-by-case basis.
Key Idea2.7.11.Linearly independent sets of vectors in \(\mathbb{R}^3\).
Any set \(\{\vec{u}\}\) containing a single vector in \(\mathbb{R}^3\) is linearly dependent if \(\vec{u}=\vec{0}\text{,}\) and independent otherwise. (Here \eqref{eq-linearindep} becomes \(c\vec{u}=\vec{0}\text{.}\) If \(\vec{u}\neq\vec{0}\text{,}\) the only solution is to take \(c=0\text{.}\))
Any set \(\{\vec{u},\vec{v}\}\) containing two non-zero vectors in \(\mathbb{R}^3\) is linearly dependent if \(\vec{u}\) is parallel to \(\vec{v}\text{,}\) and independent otherwise. (In other words, two dependent vectors lie on the same line. Two independent vectors span a plane.)
Any set \(\{\vec{u},\vec{v},\vec{w}\}\) of three non-zero vectors in \(\mathbb{R}^3\) is linearly dependent if all three vectors lie in the same plane, and independent otherwise.
This section introduced several new ideas. Some, like linear combinations, are straightforward. Others, like span and linear independence, take some getting used to. There remain two very obvious questions to address:
How do we tell whether or not a given vector belongs to the span of a set of vectors?
It turns out that both questions lead to systems of linear equations. As we saw in Example 2.7.8, we are currently unable to systematically solve such problems. In Chapters Chapter 3 and Chapter 4 we will develop the techniques needed to systematically solve such systems, at which point we will be able to easily answer questions about linear independence and span.
To see how such systems arise, suppose we want to know whether or not the vector \(\vec w = \bbm 2\\-1\\3\\0\ebm \in\mathbb{R}^4\) belongs to the set \(V = \operatorname{span}\{\vec{v}_1, \vec{v}_2, \vec{v}_3\}\text{,}\) where
By definition, \(V\) is the set of all possible linear combinations of the vectors \(\vec{v}_1, \vec{v}_2, \vec{v}_3\text{,}\) so saying that \(\vec w \in V\) is the same as saying that we can write \(\vec w\) as a linear combination of these vectors. Thus, what we want to know is whether or not there exist scalars \(x_1, x_2, x_3\) such that
\begin{equation*}
\vec w = x_1\vec{v}_1 + x_2\vec{v}_2 + x_3\vec{v}_3\text{.}
\end{equation*}
Substituting in the values for our vectors, this gives
Thus, the question “Is the vector \(\vec w\) an element of \(V\text{?}\)” is equivalent to the question “Is there a solution to the above system of equations?”
Questions about linear independence are similar, but not quite the same. With the above example involving span, what we wanted to know is “Does a solution exist?” With linear independence, it is not whether a solution exists that is in doubt, but whether or not that solution is unique. For example, suppose we wanted to know if the vectors in our span example above are linearly independent. We would start with the vector equation
This vector equation leads to a system just like the one above, except that the numbers to the right of the \(=\) signs would all be zeros. The techniques needed to answer these and other questions will be developed beginning in Chapter 3.
If the vector \(\vec{w}\) belongs to the span of the set \(\{\vec{v}_1,\ldots, \vec{v}_k\}\text{,}\) then the set \(\{\vec{w},\vec{v}_1,\ldots, \vec{v}_k\}\) is linearly dependent.
If the set \(\{\vec{w},\vec{v}_1,\ldots, \vec{v}_k\}\) is linearly dependent, then \(\vec{w}\) belongs to the span of \(\{\vec{v}_1,\ldots, \vec{v}_k\}\text{.}\)
