Elementary Linear Algebra For University of Lethbridge Math 1410

So far in this chapter, we’ve learned a systematic method for solving systems of linear equations. Some of the applied examples we considered in the previous section led naturally to systems of equations, and had solutions that were best interpreted in that context. Other examples, such as those involving span and linear independence, were stated in terms of vectors. In this section, we discuss how to write a system of linear equations in terms of vectors and matrices, and express solutions as vectors.

Expressing the solutions of linear systems in terms of vectors will give us additional insight into the behaviour of those systems, and provides a stepping-off point for the study of the algebra of matrices.

We have often relied on previous algebra experience to help us understand linear algebra concepts. We do that again here. Consider the equation \(ax=b\text{,}\) where \(a=3\) and \(b=6\text{.}\) If we asked one to “solve for \(x\text{,}\)” what exactly would we be asking? We would want to find a number, which we call \(x\text{,}\) where \(a\) times \(x\) gives \(b\text{;}\) in this case, it is a number, when multiplied by 3, returns 6. As long as \(a\neq 0\) (what if \(a\) does equal zero?), we know that we can multiply both sides of the equation by \(\frac{1}{a}\) to get \(x = \frac{1}{a}(b) = \frac{b}{a}\text{.}\)

The definition of the product \(A\vec{x}\) is straightforward. We want the result to be a column vector of size \(m\) (that is, with \(m\) entries), so that we can set it equal to the column vector \(\vec{b}\text{.}\) Furthermore, each entry in \(\vec{b}\) is the right-hand side of an equation in (3.6.1), so we want the corresponding entry in \(A\vec{x}\) to be the left-hand side. We are immediately forced to adopt the following rule.

A few remarks about Definition 3.6.1 are needed here. First, note that the number of columns in the matrix \(A\) matches the number of entries in the vector \(\vec{x}\text{,}\) and the number of rows in \(A\) matches the number of entries in the vector \(\vec{b}\text{.}\) Moreover,

That is, since each row of \(A\) has \(n\) entries, as does the vector \(\vec{x}\text{,}\) we can form the dot product of \(\vec{x}\) with any of the rows of \(A\text{.}\) Each such dot product forms the corresponding entry in the vector \(A\vec{x}\text{.}\)

Or equivalently, using the language of Section 2.7, the vector \(A\vx\) belongs to the span of the columns of \(A\text{.}\) This will be a useful observation when solving questions about span.

This should look familiar; it is a system of linear equations! Given the matrix-vector equation \(A\vx=\vb\text{,}\) we can recognize \(A\) as the coefficient matrix from a linear system and as the vector of the constants from the linear system. Given a system of equations, rewriting it in matrix form is equally straightforward.

To solve a matrix--vector equation (and the corresponding linear system), we simply augment the matrix \(A\) with the vector \(\vb\text{,}\) put this matrix into reduced row echelon form, and interpret the results.

This probably seems all well and good. While asking one to solve the equation \(A\vx=\vb\) for \(\vx\) seems like a new problem, in reality it is just asking that we solve a system of linear equations. Our variables \(x_1\text{,}\) etc., appear not individually but as the entries of our vector \(\vx\text{.}\) We are simply writing an old problem in a new way.

In line with this new way of writing the problem, we have a new way of writing the solution. Instead of listing, individually, the values of the unknowns, we simply list them as the elements of our vector\(\vx\text{.}\)

These are important ideas, so we state the basic principle once more: solving the equation \(A\vx=\vb\) for \(\vx\) is the same thing as solving a linear system of equations. Equivalently, any system of linear equations can be written in the form \(A\vx=\vb\) for some matrix \(A\) and vector\(\vb\text{.}\)

Since these ideas are equivalent, we’ll refer to \(A\vx=\vb\) both as a matrix–vector equation and as a system of linear equations: they are the same thing.

We’ve seen two examples illustrating this idea so far, and in both cases the linear system had exactly one solution. We know from Theorem 3.4.2 that any linear system has either one solution, infinitely many solutions, or no solution. So how does our new method of writing a solution work with infinitely many solutions and no solutions?

Certainly, if \(A\vx=\vb\) has no solution, we simply say that the linear system has no solution. There isn’t anything special to write. So the only other option to consider is the case where we have infinitely many solutions. We’ll learn how to handle these situations through examples.

A few comments are in order here. First, matrix equations (or the corresponding system of linear equations) such as the above where the vector on the right-hand side is the zero vector form a special case that is important enough to have its own name: these are known as homogeneous systems of equations. The formal definition is as follows.

The term homogeneous comes from two Greek words; homo meaning “same” and genus meaning “type.” A homogeneous system of equations is a system in which each equation is of the same type — all constants are 0. In mathematics, objects that are considered homogeneous often have some sort of scale invariance, and that is the case here: if \(\vx\) is a solution to \(A\vx=\zero\text{,}\) then so is \(c\vx\) for any scalar \(c\text{.}\) (Do you see why?)

Notice that the line \(\vx =t\vvv\) in the solution of Example 3.6.3 above passes through the origin. This is an important characteristic of homogeneous systems: since \(A\vec 0 = \vec 0\) for any matrix \(A\text{,}\) we always have (at least) the solution \(\vx = \vec 0\text{.}\) (We’ll have more to say about this below.)

In the last two examples, we saw that the general solution could be written in the form \(\vx = t\vvv\) for a vector \(\vvv\) such that \(A\vvv =\vec 0\text{.}\) Such vectors are known as the basic solutions to a homogeneous linear system.

To help clarify Definition 3.6.8, let’s do one more example where we have more than one basic solution.

Let’s practice finding vector solutions again; this time, we won’t solve a system of the form \(A\vx=\zero\text{,}\) but instead \(A\vx = \vb\text{,}\) for some vector \(\vb\neq \vec 0\text{.}\) Such systems are known (unsurprisingly) as non-homogeneous systems.

The previous examples illustrate some important concepts. One is that (at least, when \(\vec x\in \R^2\) or \(\R^3\)) we can visualize the solution to a system of linear equations. Before, when we had infinitely many solutions, we knew we could arbitrarily pick values for our free variables and get different solutions. We knew this to be true, and we even practised it, but the result was not very “tangible.” Now, we can view our solution as a vector; by picking different values for our free variables, we see this as multiplying certain important vectors by a scalar which gives a different solution.

Recall that for a homogeneous system of linear equations, we know that \(\vx = \vec 0\) will be a solution, since no matter what the matrix \(A\) is, we can be certain that \(A\zero = \zero\text{.}\) This fact is important; the zero vector is always a solution to a homogeneous linear system. Therefore a homogeneous system is always consistent; we need only to determine whether we have exactly one solution (just \(\zero\)) or infinitely many solutions. This idea is important, so we give it its own box.

How do we determine if we have exactly one or infinitely many solutions? Recall Key Idea 3.4.7: if the solution has any free variables, then it will have infinitely many solutions. How can we tell if the system has free variables? Form the augmented matrix \(\bbm A \amp \zero \ebm\text{,}\) put it into reduced row echelon form, and interpret the result.

It may seem that we’ve brought up a new question, “When does \(A\vx=\zero\) have exactly one or infinitely many solutions?” only to answer with “Look at the reduced row echelon form of \(A\) and interpret the results, just as always.” Why bring up a new question if the answer is an old one?

So we wonder: does this mean that \(A\vx = \vb\) will have infinitely many solutions? After all, if \(\vec{x}_p\) and \(\vec{x}_p+\vvv\) are both solutions, don’t we have infinitely many solutions?

No. If \(A\vx = \zero\) has exactly one solution, then \(\vvv = \zero\text{,}\) and \(\vec{x}_p = \vec{x}_p +\vvv\text{;}\) we only have one solution.

So here is the culmination of all of our fun that started a few pages back. If \(\vvv\) is a solution to \(A\vx=\zero\) and \(\vec{x}_p\) is a solution to \(A\vx=\vb\text{,}\) then \(\vec{x}_p+\vvv\) is also a solution to \(A\vx=\vb\text{.}\) If \(A\vx=\zero\) has infinitely many solutions, so does \(A\vx = \vb\text{;}\) if \(A\vx=\zero\) has only one solution, so does \(A\vx = \vb\text{.}\) This culminating idea is of course important enough to be stated again.

A key word in the above statement is consistent. If \(A\vx = \vb\) is inconsistent (the linear system has no solution), then it doesn’t matter how many solutions \(A\vx = \zero\) has; \(A\vx=\vb\) has no solution.

We can elaborate on Key Idea 3.6.13 above, as well as Key Idea 3.4.7 from Section 3.4 by introducing one more piece of important terminology. By now it is probably clear that the leading 1s in the reduced row echelon form of a matrix play a key role in understanding the system. In fact, it turns out that we can describe all of the different possibilities for a linear system in terms of one number: the number of leading 1s in the reduced row echelon form of a matrix.

Although we do not prove it in this textbook, the reduced row echelon form of any matrix is unique; it follows from this fact that the rank of a matrix is a well-defined number. The importance of rank is outlined in the following result.

Items 2 and 3 in Theorem 3.6.15 simply give another way of stating the fact that the free variables are those variables that do not have a leading 1 in their column. This seems like an obvious fact, but it is very important. We will see in Section 5.4 that this observation leads to a major theorem, sometimes known as the Fundamental Theorem of Linear Transformations: see Theorem 5.4.12.

Let us explore this result with a series of examples.

The previous example may seem to violate the principle of Key Idea 3.6.13. After all, it seems that having infinitely many solutions to \(A\vx = \zero\) should imply infinitely many solutions to \(A\vx = \vb\text{.}\) However, we remind ourselves of the key word in the idea that we observed before: consistent. If \(A\vx = \vb\) is consistent and \(A\vx = \zero\) has infinitely many solutions, then so will \(A\vx = \vb\text{.}\) But if \(A\vx = \vb\) is not consistent, it does not matter how many solutions \(A\vx = \zero\) has; \(A\vx = \vb\) is still inconsistent.

In this chapter, we developed a systematic method for solving systems of linear equations. A key tool in this method was the augmented matrix corresponding to a given system. In this final section, we’ve seen that further insight into the structure of solutions can be gained by considering our systems in terms of matrices and vectors.

In the next chapter, we will begin the study of matrices as objects unto themselves. We will see that they can be added and multiplied by scalars in exactly the same way as vectors, and in addition to this, matrices of the correct sizes can be multiplied in a way that reproduces Definition 3.6.1 above as a special case.

One question that may have occurred to you as you worked through this section is the following: in the one-variable linear equation \(ax=b\text{,}\) we know that as long as \(a\neq 0\text{,}\) we can divide both sides by \(a\text{,}\) giving us the solution \(x=b/a\text{.}\) Now, given the matrix equation \(A\vec{x}=\vec{b}\text{,}\) is there some equivalent means of “dividing by \(A\)” to obtain the solution \(\vec{x}\text{?}\) The short answer is no. Indeed, there is no such thing as matrix division; the algebraic rules for matrix multiplication are much more complicated than they are for numbers. (In particular, we’ll see that for matrices, \(AB\) is usually not the same thing as \(BA\text{!}\))

The slightly longer answer to our question might be phrased as “Sometimes. Well, sort of.” To obtain the correct (and much longer) answer, we will be led in the next chapter to the definition of the inverse of a matrix.