How did we expand out E_LLS = ||Ax-b||^2 to get the line below it?
motoole2
Great question! This is described in a bit more detail on page 743 of the Szeliski textbook. It might be helpful to refer to the matrix cookbook, which contains a number of identities related to matrices and vectors.
For the first step, we should note that $||\mathbf{x}||^2 = \sum_{k=1}^{K} \mathbf{x}_k^2 = \mathbf{x}^T \mathbf{x}$. Using this fact, we can expand out the first term as follows: $||\mathbf{A}\mathbf{x} - \mathbf{b}||^2 = (\mathbf{A}\mathbf{x} - \mathbf{b})^T (\mathbf{A}\mathbf{x} - \mathbf{b}) = \mathbf{x}^T (\mathbf{A}^T \mathbf{A})\mathbf{x} - 2\mathbf{x}^T \mathbf{A}^T\mathbf{b} + \mathbf{b}^T \mathbf{b}$.
After that, the last few lines take the derivative with respect to $\mathbf{x}$, set the derivative to 0, and solve for $\mathbf{x}$. If you would like more details on how to do this (e.g., computing derivatives of matrices and vectors), I would recommend looking at Section 2.4 of the matrix cookbook.
How did we expand out E_LLS = ||Ax-b||^2 to get the line below it?
Great question! This is described in a bit more detail on page 743 of the Szeliski textbook. It might be helpful to refer to the matrix cookbook, which contains a number of identities related to matrices and vectors.
For the first step, we should note that $||\mathbf{x}||^2 = \sum_{k=1}^{K} \mathbf{x}_k^2 = \mathbf{x}^T \mathbf{x}$. Using this fact, we can expand out the first term as follows: $||\mathbf{A}\mathbf{x} - \mathbf{b}||^2 = (\mathbf{A}\mathbf{x} - \mathbf{b})^T (\mathbf{A}\mathbf{x} - \mathbf{b}) = \mathbf{x}^T (\mathbf{A}^T \mathbf{A})\mathbf{x} - 2\mathbf{x}^T \mathbf{A}^T\mathbf{b} + \mathbf{b}^T \mathbf{b}$.
After that, the last few lines take the derivative with respect to $\mathbf{x}$, set the derivative to 0, and solve for $\mathbf{x}$. If you would like more details on how to do this (e.g., computing derivatives of matrices and vectors), I would recommend looking at Section 2.4 of the matrix cookbook.