Differential Equations - Systems: Equations

Because we are going to be working almost exclusively with systems of equations in which the number of unknowns equals the number of equations we will restrict our review to these kinds of systems.

All of what we will be doing here can be easily extended to systems with more unknowns than equations or more equations than unknowns if need be.

Let’s start with the following system of $n$ equations with the $n$ unknowns, $x_1$, $x_2$,…, $x_n$.

$$\begin{array}{}\begin{array}{cc}{a}_{11}{x}_1+a_{12}{x}_2+\cdots +a_{1n}{x}_n& =b_1\\ a_{21}{x}_1+a_{22}{x}_2+\cdots +a_{2n}{x}_n& =b_2\\ ⋮& \\ a_{n1}{x}_1+a_{n2}{x}_2+\cdots +a_{nn}{x}_n& =b_n\end{array}& \text{(1)}\end{array}$$

Note that in the subscripts on the coefficients in this system, $a_{ij}$, the $i$ corresponds to the equation that the coefficient is in and the $j$ corresponds to the unknown that is multiplied by the coefficient.

To use linear algebra to solve this system we will first write down the augmented matrix for this system. An augmented matrix is really just all the coefficients of the system and the numbers for the right side of the system written in matrix form. Here is the augmented matrix for this system.

$$\left(\begin{array}{ccccc}{a}_{11}& a_{12}& \cdots & a_{1n}& b_1\\ a_{21}& a_{22}& \cdots & a_{2n}& b_2\\ ⋮& ⋮& \ddots & ⋮& ⋮\\ a_{n1}& a_{n2}& \cdots & a_{nn}& b_n\end{array}\right)$$

To solve this system we will use elementary row operations (which we’ll define these in a bit) to rewrite the augmented matrix in triangular form. The matrix will be in triangular form if all the entries below the main diagonal (the diagonal containing $a_{11}$, $a_{22}$, …,$a_{nn}$) are zeroes.

Once this is done we can recall that each row in the augmented matrix corresponds to an equation. We will then convert our new augmented matrix back to equations and at this point solving the system will become very easy.

Before working an example let’s first define the elementary row operations. There are three of them.

1. Interchange two rows. This is exactly what it says. We will interchange row $i$ with row $j$. The notation that we’ll use to denote this operation is : $R_i\leftrightarrow R_j$
2. Multiply row $i$ by a constant, $c$. This means that every entry in row $i$ will get multiplied by the constant $c$. The notation for this operation is : $c{R}_i$
3. Add a multiple of row $i$ to row j. In our heads we will multiply row $i$ by an appropriate constant and then add the results to row $j$ and put the new row back into row $j$ leaving row $i$ in the matrix unchanged. The notation for this operation is : $c{R}_i+R_j$

It’s always a little easier to understand these operations if we see them in action. So, let’s solve a couple of systems.

Example 1 Solve the following system of equations.
$$\begin{array}{cc}-2x_1+x_2-x_3& =4\\ x_1+2x_2+3x_3& =13\\ 3x_1+x_3& =-1\end{array}$$

The first step is to write down the augmented matrix for this system. Don’t forget that coefficients of terms that aren’t present are zero.
$$\left(\begin{array}{cccc}-2& 1& -1& 4\\ 1& 2& 3& 13\\ 3& 0& 1& -1\end{array}\right)$$Now, we want the entries below the main diagonal to be zero. The main diagonal has been colored red so we can keep track of it during this first example. For reasons that will be apparent eventually we would prefer to get the main diagonal entries to all be ones as well.
We can get a one in the upper most spot by noticing that if we interchange the first and second row we will get a one in the uppermost spot for free. So let’s do that.
$$\left(\begin{array}{cccc}-2& 1& -1& 4\\ 1& 2& 3& 13\\ 3& 0& 1& -1\end{array}\right)\begin{array}{c}{R}_1\leftrightarrow R_2\\ \to \end{array}\left(\begin{array}{cccc}1& 2& 3& 13\\ -2& 1& -1& 4\\ 3& 0& 1& -1\end{array}\right)$$Now we need to get the last two entries (the -2 and 3) in the first column to be zero. We can do this using the third row operation. Note that if we take 2 times the first row and add it to the second row we will get a zero in the second entry in the first column and if we take -3 times the first row to the third row we will get the 3 to be a zero. We can do both of these operations at the same time so let’s do that.
$$\left(\begin{array}{cccc}1& 2& 3& 13\\ -2& 1& -1& 4\\ 3& 0& 1& -1\end{array}\right)\begin{array}{c}2R_1+R_2\\ -3R_1+R_3\\ \to \end{array}\left(\begin{array}{cccc}1& 2& 3& 13\\ 0& 5& 5& 30\\ 0& -6& -8& -40\end{array}\right)$$Before proceeding with the next step, let’s make sure that you followed what we just did. Let’s take a look at the first operation that we performed. This operation says to multiply an entry in row 1 by 2 and add this to the corresponding entry in row 2 then replace the old entry in row 2 with this new entry. The following are the four individual operations that we performed to do this.
$$\begin{array}{cc}2\left(1\right)+\left(-2\right)& =0\\ 2\left(2\right)+1& =5\\ 2\left(3\right)+\left(-1\right)& =5\\ 2\left(13\right)+4& =30\end{array}$$Okay, the next step optional, but again is convenient to do. Technically, the 5 in the second column is okay to leave. However, it will make our life easier down the road if it is a 1. We can use the second row operation to take care of this. We can divide the whole row by 5. Doing this gives,
$$\left(\begin{array}{cccc}1& 2& 3& 13\\ 0& 5& 5& 30\\ 0& -6& -8& -40\end{array}\right)\begin{array}{c}\frac{1}{5}{R}_2\\ \to \end{array}\left(\begin{array}{cccc}1& 2& 3& 13\\ 0& 1& 1& 6\\ 0& -6& -8& -40\end{array}\right)$$The next step is to then use the third row operation to make the -6 in the second column into a zero.
$$\left(\begin{array}{cccc}1& 2& 3& 13\\ 0& 1& 1& 6\\ 0& -6& -8& -40\end{array}\right)\begin{array}{c}6R_2+R_3\\ \to \end{array}\left(\begin{array}{cccc}1& 2& 3& 13\\ 0& 1& 1& 6\\ 0& 0& -2& -4\end{array}\right)$$Now, officially we are done, but again it’s somewhat convenient to get all ones on the main diagonal so we’ll do one last step.
$$\left(\begin{array}{cccc}1& 2& 3& 13\\ 0& 1& 1& 6\\ 0& 0& -2& -4\end{array}\right)\begin{array}{c}-\frac{1}{2}{R}_3\\ \to \end{array}\left(\begin{array}{cccc}1& 2& 3& 13\\ 0& 1& 1& 6\\ 0& 0& 1& 2\end{array}\right)$$We can now convert back to equations.
$$\left(\begin{array}{cccc}1& 2& 3& 13\\ 0& 1& 1& 6\\ 0& 0& 1& 2\end{array}\right)\Rightarrow \begin{array}{cc}{x}_1+2x_2+3x_3& =13\\ x_2+x_3& =6\\ x_3& =2\end{array}$$At this point the solving is quite easy. We get $x_3$ for free and once we get that we can plug this into the second equation and get $x_2$. We can then use the first equation to get $x_1$. Note as well that having 1’s along the main diagonal helped somewhat with this process.
The solution to this system of equation is
$$x_1=-1x_2=4x_3=2$$

The process used in this example is called Gaussian Elimination. Let’s take a look at another example.

Example 2 Solve the following system of equations.$$\begin{array}{cc}{x}_1-2x_2+3x_3& =-2\\ -x_1+x_2-2x_3& =3\\ 2x_1-x_2+3x_3& =1\end{array}$$

First write down the augmented matrix.
$$\left(\begin{array}{cccc}1& -2& 3& -2\\ -1& 1& -2& 3\\ 2& -1& 3& 1\end{array}\right)$$We won’t put down as many words in working this example. Here’s the work for this augmented matrix.
$$\left(\begin{array}{cccc}1& -2& 3& -2\\ -1& 1& -2& 3\\ 2& -1& 3& 1\end{array}\right)\begin{array}{c}{R}_1+R_2\\ -2R_1+R_3\\ \to \end{array}\left(\begin{array}{cccc}1& -2& 3& -2\\ 0& -1& 1& 1\\ 0& 3& -3& 5\end{array}\right)$$$$\begin{array}{c}-R_2\\ \to \end{array}\left(\begin{array}{cccc}1& -2& 3& -2\\ 0& 1& -1& -1\\ 0& 3& -3& 5\end{array}\right)\begin{array}{c}-3R_2+R_3\\ \to \end{array}\left(\begin{array}{cccc}1& -2& 3& -2\\ 0& 1& -1& -1\\ 0& 0& 0& 8\end{array}\right)$$We won’t go any farther in this example. Let’s go back to equations to see why.
$$\left(\begin{array}{cccc}1& -2& 3& -2\\ 0& 1& -1& -1\\ 0& 0& 0& 8\end{array}\right)\Rightarrow \begin{array}{cc}{x}_1-2x_2+3x_3& =-2\\ x_2-x_3& =-1\\ 0& =8\end{array}$$The last equation should cause some concern. There’s one of three options here. First, we’ve somehow managed to prove that 0 equals 8 and we know that’s not possible. Second, we’ve made a mistake, but after going back over our work it doesn’t appear that we have made a mistake.
This leaves the third option. When we get something like the third equation that simply doesn’t make sense we immediately know that there is no solution. In other words, there is no set of three numbers that will make all three of the equations true at the same time.

Let’s work another example. We are going to get the system for this new example by making a very small change to the system from the previous example.

Example 3 Solve the following system of equations.$$\begin{array}{cc}{x}_1-2x_2+3x_3& =-2\\ -x_1+x_2-2x_3& =3\\ 2x_1-x_2+3x_3& =-7\end{array}$$

So, the only difference between this system and the system from the second example is we changed the 1 on the right side of the equal sign in the third equation to a -7.
Now write down the augmented matrix for this system.
$$\left(\begin{array}{cccc}1& -2& 3& -2\\ -1& 1& -2& 3\\ 2& -1& 3& -7\end{array}\right)$$The steps for this problem are identical to the steps for the second problem so we won’t write them all down. Upon performing the same steps we arrive at the following matrix.
$$\left(\begin{array}{cccc}1& -2& 3& -2\\ 0& 1& -1& -1\\ 0& 0& 0& 0\end{array}\right)$$This time the last equation reduces to
$$0=0$$and unlike the second example this is not a problem. Zero does in fact equal zero!
We could stop here and go back to equations to get a solution and there is a solution in this case. However, if we go one more step and get a zero above the one in the second column as well as below it our life will be a little simpler. Doing this gives,
$$\left(\begin{array}{cccc}1& -2& 3& -2\\ 0& 1& -1& -1\\ 0& 0& 0& 0\end{array}\right)\begin{array}{c}2R_2+R_1\\ \Rightarrow \end{array}\left(\begin{array}{cccc}1& 0& 1& -4\\ 0& 1& -1& -1\\ 0& 0& 0& 0\end{array}\right)$$If we now go back to equation we get the following two equations.
$$\left(\begin{array}{cccc}1& 0& 1& -4\\ 0& 1& -1& -1\\ 0& 0& 0& 0\end{array}\right)\Rightarrow \begin{array}{c}{x}_1+x_3=-4\\ x_2-x_3=-1\\ \end{array}$$We have two equations and three unknowns. This means that we can solve for two of the variables in terms of the remaining variable. Since $x_3$ is in both equations we will solve in terms of that.
$$\begin{array}{cc}{x}_1& =-x_3-4\\ x_2& =x_3-1\end{array}$$What this solution means is that we can pick the value of $x_3$ to be anything that we’d like and then find values of $x_1$ and $x_2$. In these cases, we typically write the solution as follows,
$$\begin{array}{cc}{x}_1& =-t-4\\ x_2& =t-1t=\text{any real number}\\ & x_3=t\end{array}$$In this way we get an infinite number of solutions, one for each and every value of $t$.

These three examples lead us to a nice fact about systems of equations.

Fact

Given a system of equations, $\text{(1)}$, we will have one of the three possibilities for the number of solutions.

1. No solution.
2. Exactly one solution.
3. Infinitely many solutions.

Before moving on to the next section we need to take a look at one more situation. The system of equations in $\text{(1)}$ is called a nonhomogeneous system if at least one of the b_i’s is not zero. If however all of the $b_i$'s are zero we call the system homogeneous and the system will be,

$$\begin{array}{}\begin{array}{cc}{a}_{11}{x}_1+a_{12}{x}_2+\cdots +a_{1n}{x}_n& =0\\ a_{21}{x}_1+a_{22}{x}_2+\cdots +a_{2n}{x}_n& =0\\ ⋮& \\ a_{n1}{x}_1+a_{n2}{x}_2+\cdots +a_{nn}{x}_n& =0\end{array}& \text{(2)}\end{array}$$

Now, notice that in the homogeneous case we are guaranteed to have the following solution.

$$x_1=x_2=\cdots =x_n=0$$

This solution is often called the trivial solution.

For homogeneous systems the fact above can be modified to the following.

Fact

Given a homogeneous system of equations, $\text{(2)}$, we will have one of the two possibilities for the number of solutions.

1. Exactly one solution, the trivial solution
2. Infinitely many non-zero solutions in addition to the trivial solution.

In the second possibility we can say non-zero solution because if there are going to be infinitely many solutions and we know that one of them is the trivial solution then all the rest must have at least one of the $x_i$'s be non-zero and hence we get a non-zero solution.