Today we continue Section 3.5. Continue reading Section 3.5, and also Section 8.1. Work through suggested exercises.
Homework 7 is on WeBWorK and is due on Friday at 11:55pm.
Math Help Centre: M-F 12:30-5:30 in PAB48/49 and online 6pm-8pm.
My next office hour is Friday 2:30-3:20 in MC130.
The midterm is on Saturday, November 9, 2-4pm. It will cover until the end of Section 3.5, which we will finish this week.
Undocumented absences: Remember that these cannot be used for any aspect of this course. You should also not submit medical documentation for assignments, as our built-in flexibility and reweighting covers all situations.
Definition: A subspace of $\R^n$ is any collection $S$ of
vectors in $\R^n$ such that:
1. The zero vector $\vec 0$ is in $S$.
2. $S$ is closed under addition:
If $\vu$ and $\vv$ are in $S$, then $\vu + \vv$ is in $S$.
3. $S$ is closed under scalar multiplication:
If $\vu$ is in $S$ and $c$ is any scalar, then $c \vu$ is in $S$.
Definition: A basis for a subspace $S$ of $\R^n$ is a
set of vectors $\vv_1, \ldots, \vv_k$ such that:
1. $S = \span(\vv_1, \ldots, \vv_k)$, and
2. $\vv_1, \ldots, \vv_k$ are linearly independent.
Definition: Let $A$ be an $m \times n$ matrix.
1. The row space of $A$ is the subspace $\row(A)$ of $\R^n$ spanned
by the rows of $A$.
2. The column space of $A$ is the subspace $\col(A)$ of $\R^m$ spanned
by the columns of $A$.
3. The null space of $A$ is the subspace $\null(A)$ of $\R^n$
consisting of the solutions to the system $A \vx = \vec 0$.
Theorem 3.20: Let $A$ and $R$ be row equivalent matrices. Then $\row(A) = \row(R)$.
Also, $\null(A) = \null(R)$. But elementary row operations change the column space! So $\col(A) \neq \col(R)$.
Theorem: If $R$ is a matrix in row echelon form, then the nonzero rows of $R$ form a basis for $\row(R)$.
1. Find the reduced row echelon form $R$ of $A$.
2. The nonzero rows of $R$ form a basis for $\row(A) = \row(R)$.
3. The columns of $A$ that correspond to the columns of $R$ with leading 1's
form a basis for $\col(A)$.
4. Use back substitution to solve $R \vx = \vec 0$; the vectors that
arise are a basis for $\null(A) = \null(R)$.
Row echelon form is in fact enough. Then you look at the columns with leading nonzero entries (the pivot columns).
Solution: Row reduction gives \[ R = \bmat{rrrrr} 0 & 1& -4& 0& 3\\ 0& \phantom{-}0& \phantom{-1}0& \phantom{-}1& \phantom{1}2\\ 0& 0& 0& 0& 0\\ \emat \] Therefore, a basis for $\row(A)$ is \[ \{ [0,\, 1,\, -4,\, 0,\, 3],\, [0,\, 0,\, 0,\, 1,\, 2] \} . \] A basis for $\col(A)$ is \[ \{ \colll 1 3 2 , \colll {-1} 0 2 \} \] To find a basis for $\null(A)$, we do back substitution. Our system is \[ x_2 - 4 x_3 + 3 x_5 = 0 \quad\text{and}\quad x_4 + 2 x_5 = 0 . \] The variables $x_1 = r$, $x_3 = s$ and $x_5 = t$ are free, and the equations above give formulas for $x_2$ and $x_4$, so we find the general solution to be \[ \ccolllll r {4s-3t} s {-2t} t = r \ccolllll 1 0 0 0 0 + s \ccolllll 0 4 1 0 0 + t \ccolllll 0 {-3} 0 {-2} 1 . \] So a basis for the null space is given by the three vectors shown.
These methods can be used to compute a basis for a subspace $S$ spanned by some vectors $\vv_1, \ldots, \vv_k$.
The row method:
1. Form the matrix $A$ whose rows are $\vv_1, \ldots, \vv_k$, so $S = \row(A)$.
2. Reduce $A$ to row echelon form $R$.
3. The nonzero rows of $R$ will be a basis of $S = \row(A) = \row(R)$.
The column method:
1. Form the matrix $A$ whose columns are $\vv_1, \ldots, \vv_k$, so $S = \col(A)$.
2. Reduce $A$ to row echelon form $R$.
3. The columns of $A$ that correspond to the columns of $R$ with leading entries
form a basis for $S = \col(A)$.
Theorem 3.23: Let $S$ be a subspace of $\R^n$. Then any two bases for $S$ have the same number of vectors.
Definition: The number of vectors in a basis for a subspace $S$ is called the dimension of $S$, denoted $\dim S$.
We say that the dimension of the zero subspace $\{ \vec 0 \}$ is $0$.
Example: If $S$ is a line through the origin in $\R^2$ or $\R^3$, then $\dim S = \query{1}$
Example: If $S$ is a plane through the origin in $\R^3$, then $\dim S = \query{2}$
Example: If $S = \span(\colll 3 0 2, \colll {-2} 1 1, \colll 1 1 3)$, then $\dim S = \query{2}$
Theorem 3.23: Let $S$ be a subspace of $\R^n$. Then any two bases for $S$ have the same number of vectors.
Idea of proof: Suppose that $\{ \vu_1, \vu_2 \}$ and $\{ \vv_1, \vv_2, \vv_3 \}$ were both bases for $S$. We'll show that this is impossible, by showing that $\vv_1, \vv_2, \vv_3$ are linearly dependent. Since $\{ \vu_1, \vu_2 \}$ is a basis, we can express each $\vv_i$ in terms of the $\vu_j$'s: $$ \begin{aligned} \vv_1 &\ = a_{11} \vu_1 + a_{21} \vu_2 \\ \vv_2 &\ = a_{12} \vu_1 + a_{22} \vu_2 \\ \vv_3 &\ = a_{13} \vu_1 + a_{23} \vu_2 \end{aligned} $$ Then $$ \kern-8ex \begin{aligned} &c_1 \vv_1 + c_2 \vv_2 + c_3 \vv_3 \\ =\ &c_1 ( a_{11} \vu_1 + a_{21} \vu_2 ) + c_2 ( a_{12} \vu_1 + a_{22} \vu_2 ) + c_3 ( a_{13} \vu_1 + a_{23} \vu_2 ) \\ =\ &(c_1 a_{11} + c_2 a_{12} + c_3 a_{13}) \vu_1 + (c_1 a_{21} + c_2 a_{22} + c_3 a_{23}) \vu_2 \end{aligned} $$ But the homogenous system in the variables $c_1$, $c_2$, $c_3$ $$ \begin{aligned} c_1 a_{11} + c_2 a_{12} + c_3 a_{13} &\ = 0 \\ c_1 a_{21} + c_2 a_{22} + c_3 a_{23} &\ = 0 \end{aligned} $$ has nontrivial solutions! (Why?) Therefore, we can find nontrivial $c_1$, $c_2$, $c_3$ such that $$ c_1 \vv_1 + c_2 \vv_2 + c_3 \vv_3 = \vec 0 \qquad \Box $$
A very similar argument works for the general case.
True/false: Every subspace of $\R^3$ has dimension 0, 1, 2 or 3.
True/false: If a matrix $A$ has row echelon form $$ R=\bmat{rrr} 0 & 2 & 3 \\ 0 & 0 & 4 \\ \emat $$ then a basis for $\col(A)$ is given by $\coll 2 0$ and $\coll 3 4$.
True! In fact, in this case the method would be incorrect, but it actually gives the correct answer, because $\col(A)$ must be all of $\R^2$, and those columns of $R$ do give a basis for $\R^2$.
Question: For the same $A$ and $R$, what's a basis for $\row(A)$?
Example: Let $A$ be the matrix from last class whose reduced row echelon form is $R$: $$ \kern-8ex A = \bmat{rrrrr} 1 & 1 & 3 & 1 & 6 \\ 2 & -1 & 0 & 1 & -1 \\ -3 & 2 & 1 & -2 & 1 \\ 4 & 1 & 6 & 1 & 3 \emat \qquad R = \bmat{rrrrr} 1 & 0 & 1 & 0 & -1 \\ 0 & 1 & 2 & 0 & 3 \\ 0 & 0 & 0 & 1 & 4 \\ 0 & 0 & 0 & 0 & 0 \emat $$ Then: $\ \dim \row(A) = \query{3}$ $\ \dim \col(A) = \query{3}$ $\ \dim \null(A) = \query{2}$
Note that $\dim \row(A) = \rank(A)$, since we defined the rank of $A$ to be the number of nonzero rows in $R$. The above theorem shows that this number doesn't depend on how you row reduce $A$.
We call the dimension of the null space the nullity of $A$ and write $\nullity(A) = \dim \null(A)$. This is what we called the "number of free variables" in Chapter 2.
From the way we find the basis for $\row(A)$, $\col(A)$ and $\null(A)$, can you deduce any relationships between their dimensions?
Theorems 3.24 and 3.26: Let $A$ be an $m \times n$ matrix. Then $$ \dim \row(A) = \dim \col(A) = \rank(A) $$ and $$ \rank(A) + \nullity(A) = n . $$
Very important!
True/false: for any $A$, $\rank(A) = \rank(A^T)$.
True/false: if $A$ is $2 \times 5$, then the nullity of $A$ is $3$.
True/false: if $A$ is $5 \times 2$, then $\nullity(A) \geq 3$.
Example: Find the nullity of $$ M = \bmat{rrrrrrr} 1 & 2 & 3 & 4 & 5 & 6 & 7 \\ 8 & 9 & 10 & 11 & 12 & 13 & 14 \emat $$ and of $M^T$. Any guesses? The rows of $M$ are linearly independent, so the rank is $2$, so the nullity is $7 - 2 = 5$. The rank of $M^T$ is also $2$, so the nullity of $M^T$ is $2 - 2 = 0$.
For larger matrices, you would compute the rank by row reduction.
Theorem 3.27:
Let $A$ be an $n \times n$ matrix. The following are equivalent:
a. $A$ is invertible.
b. $A \vx = \vb$ has a unique solution for every $\vb \in \R^n$.
c. $A \vx = \vec 0$ has only the trivial (zero) solution.
d. The reduced row echelon form of $A$ is $I_n$.
e. $A$ is a product of elementary matrices.
f. $\rank(A) = n$
g. $\nullity(A) = 0$
h. The columns of $A$ are linearly independent.
i. The columns of $A$ span $\R^n$.
j. The columns of $A$ are a basis for $\R^n$.
k. The rows of $A$ are linearly independent.
l. The rows of $A$ span $\R^n$.
m. The rows of $A$ are a basis for $\R^n$.