Today we finish 5.2 and start 5.3.
**Read** Sections 5.3 and 5.4 for next class.
Work through recommended homework questions.

**Tutorials:** Review and questions.

**Office hour:** Mon 1:30-2:30 and Wed 10:30-11:15, MC103B.

**Help Centers:** Monday-Friday 2:30-6:30 in MC 106.

**Final exam:** Covers whole course, with an emphasis on the
material after the midterm.
Our course will end with Section 5.4.

**Question:** If $W = \R^n$, then $W^\perp = \query{\{ \vec 0 \}}$

**T/F:** An orthogonal basis $\{ \vv_1, \ldots, \vv_k \}$ must have
$$
\vv_i \cdot \vv_j =
\begin{cases} 0 & \text{if } i \neq j \\
1 & \text{if } i = j
\end{cases}
$$

**Definition:** Let $W$ be a subspace of $\R^n$.
A vector $\vv$ is **orthogonal** to $W$ if $\vv$ is orthogonal
to every vector in $W$.
The **orthogonal complement** of $W$ is the set of all vectors
orthogonal to $W$ and is denoted $W^\perp$. So
$$
\kern-4ex
W^\perp = \{ \vv \in \R^n : \vv \cdot \vw = 0 \text{ for all } \vw \text{ in } W \}
$$

An example to keep in mind is where $W$ is a plane through the origin in $\R^3$ and $W^\perp$ is $\span(\vn)$, where $\vn$ is the normal vector to $W$.

**Theorem 5.9:** Let $W$ be a subspace of $\R^n$. Then:

a. $W^\perp$ is a subspace of $\R^n$.

b. $(W^\perp)^\perp = W$

c. $W \cap W^\perp = \{ \vec 0 \}$

d. If $W = \span(\vw_1, \ldots, \vw_k)$, then $\vv$ is in $W^\perp$
if and only if $\vv \cdot \vw_i = 0$ for all $i$.

We proved all of these except part (b), which will come today.

**Theorem 5.10:** Let $A$ be an $m \times n$ matrix. Then
$$
\kern-4ex
(\row(A))^\perp = \null(A) \qtext{and} (\col(A))^\perp = \null(A^T)
$$

The first two are in $\R^n$ and the last two are in $\R^m$.
These are the **four fundamental subspaces** of $A$.

If we write $W = \span(\vu)$, then $\vw = \proj_{\vu}(\vv)$ is in $W$, $\vw^\perp = \Perp_{\vu}(\vv)$ is in $W^\perp$, and $\vv = \vw + \vw^\perp$. We can do this more generally:

**Definition:** Let $W$ be a subspace of $\R^n$
and let $\{ \vu_1, \ldots, \vu_k \}$ be an orthogonal basis for $W$.
For $\vv$ in $\R^n$, the **orthogonal projection** of $\vv$ onto $W$
is the vector
$$
\kern-8ex
\proj_W(\vv) = \proj_{\vu_1}(\vv) + \cdots + \proj_{\vu_k}(\vv)
$$
The **component of $\vv$ orthogonal to $W$** is the vector
$$
\kern-8ex
\Perp_W(\vv) = \vv - \proj_W(\vv)
$$

We will show soon that $\Perp_W(\vv)$ is in $W^\perp$.

Note that multiplying $\vu$ by a scalar in the earlier example doesn't change $W$, $\vw$ or $\vw^\perp$. We'll see later that the general definition also doesn't depend on the choice of orthogonal basis.

**Theorem:** $\Perp_W(\vv)$ is in $W^\perp$.

Explain on board.

Now we will see that $\proj$ and $\Perp$ don't depend on the choice of orthogonal basis. Here and in the rest of the section, we assume that every subspace has at least one orthogonal basis.

**Theorem 5.11:**
Let $W$ be a subspace of $\R^n$ and let $\vv$ be a vector in $\R^n$.
Then there are **unique** vectors $\vw$ in $W$ and $\vw^\perp$ in $W^\perp$
such that $\vv = \vw + \vw^\perp$.

**Proof:**
We saw above that such a decomposition exists, by taking
$\vw = \proj_W(\vv)$ and $\vw^\perp = \Perp_W(\vv)$, using an
orthogonal basis for $W$.

We now show that this decomposition is unique. So suppose $\vv = \vw_1 + \vw_1^\perp$ is another such decomposition. Then $\vw + \vw^\perp = \vw_1 + \vw_1^\perp$, so $$\vw - \vw_1 = \vw_1^\perp - \vw^\perp$$ The left hand side is in $W$ and the right hand side is in $W^\perp$ (why?), so both sides must be zero (why?). So $\vw = \vw_1$ and $\vw^\perp = \vw_1^\perp$.$\quad\Box$

Note that $\perp$ is an operation on subspaces, but is not an operation on vectors.

Now we can prove part (b) of Theorem 5.9.

**Corollary 5.12:**
If $W$ is a subspace of $\R^n$, then $(W^\perp)^\perp = W$.

**Proof:** If $\vw$ is in $W$ and $\vx$ is in $W^\perp$,
then $\vw \cdot \vx = 0$. This means that $\vw$ is in $(W^\perp)^\perp$.
So $W \subseteq (W^\perp)^\perp$.

We need to show that every vector in $(W^\perp)^\perp$ is in $W$. So let $\vv$ be a vector in $(W^\perp)^\perp$. By the previous result, we can write $\vv$ as $\vw + \vw^\perp$, where $\vw$ is in $W$ and $\vw^\perp$ is in $W^\perp$. Then $$ \kern-4ex \begin{aligned} 0 &= \vv \cdot \vw^\perp = (\vw + \vw^\perp)\cdot\vw^\perp \\ &= \vw \cdot \vw^\perp + \vw^\perp \cdot \vw^\perp = 0 + \vw^\perp \cdot \vw^\perp = \vw^\perp \cdot \vw^\perp \end{aligned} $$ So $\vw^\perp = \vec 0$ and $\vv = \vw$ is in $W$.$\quad\Box$

This next result is related to the Rank Theorem:

**Theorem 5.13:**
If $W$ is a subspace of $\R^n$, then
$$
\dim W + \dim W^\perp = n
$$

**Proof:**
Let $\{ \vu_1, \ldots, \vu_k \}$ be an orthogonal basis of $W$
and let $\{ \vv_1, \ldots, \vv_\ell \}$ be an orthogonal basis of $W^\perp$.
Then $\{ \vu_1, \ldots, \vu_k, \vv_1, \ldots, \vv_\ell \}$ is
an orthogonal basis for $\R^n$. (Explain.)
The result follows.$\quad\Box$

**Example:** For $W$ a plane in $\R^3$, $2 + 1 = 3$.

The Rank Theorem follows if we take $W = \row(A)$, since then $W^\perp = \null(A)$:

**Corollary 5.14 (The Rank Theorem, again):**
If $A$ is an $m \times n$ matrix, then
$$
\rank(A) + \nullity(A) = n
$$

**Note:** The logic here can be reversed. We can *use*
the rank theorem to prove Theorem 5.13, and Theorem 5.13 can
be used to prove Corollary 5.12.

**Example:** Let $W = \span(\vx_1, \vx_2)$ where
$\vx_1 = \colll 1 1 0$ and $\vx_2 = \colll {-2} 0 1$.
Find an orthogonal basis for $W$.

**Solution:** Ideas? Do on board.

**Question:** What if we had a third basis vector $\vx_3$?

**Theorem 5.15 (The Gram-Schmidt Process):**
Let $\{ \vx_1, \ldots, \vx_k \}$ be a basis for a subspace $W$ of $\R^n$.
Write $W_1 = \span(\vx_1)$, $W_2 = \span(\vx_1, \vx_2)$, $\ldots$, $W_k = \span(\vx_1, \ldots, \vx_k)$.
Define:
$$
\kern-9ex
\begin{aligned}
\vv_1 &= \vx_1 \\
\vv_2 &= \Perp_{W_1}(\vx_2) = \vx_2 - \frac{\vv_1 \cdot \vx_2}{\vv_1 \cdot \vv_1} \vv_1 \\
\vv_3 &= \Perp_{W_2}(\vx_3) = \vx_3 - \frac{\vv_1 \cdot \vx_3}{\vv_1 \cdot \vv_1} \vv_1
- \frac{\vv_2 \cdot \vx_3}{\vv_2 \cdot \vv_2} \vv_2 \\
&\vdots \\
\vv_k &= \Perp_{W_{k-1}}(\vx_k) = \vx_k - \frac{\vv_1 \cdot \vx_k}{\vv_1 \cdot \vv_1} \vv_1
- \cdots - \frac{\vv_{k-1} \cdot \vx_k}{\vv_{k-1} \cdot \vv_{k-1}} \vv_{k-1}
\end{aligned}
$$
Then for each $i$, $\{ \vv_1, \ldots, \vv_i \}$ is an orthogonal basis for $W_i$.
In particular, $\{ \vv_1, \ldots, \vv_k \}$ is an orthogonal basis for $W = W_k$.

Explain verbally.

**Notes:** To compute $\Perp_{W_i}$ you have to use the *orthogonal*
basis of $\vv_j$'s that you have constructed already, not the original
basis of $\vx_j$'s.

The basis you get depends on the order of the vectors you start with. You should always do a question using the vectors in the order given, since that order will be chosen to minimize the arithmetic.

If you are asked to find an orthonormal basis, normalize each $\vv_j$ at the end. (It is correct to normalize earlier, but can be messier.)