Least Squares - Linear Algebra

Card 0 of 14

Question

Let $y=(1 \ 2 \ 3 \ 4 \ 5 \10 )$ , and $x=(0 \ 1 \ 2 \3 \4 \5)$ , find the least squares solution for a linear line.

Answer

The equation for least squares solution for a linear fit looks as follows.

$y=\theta_1+\theta_2x_i$

Recall the formula for method of least squares.

$x=(A^T\cdot A)^{-1}A^T\cdot b$

Remember when setting up the A matrix, that we have to fill one column full of ones.

$A=\begin{bmatrix} 1 & 0 \ 1 & 1\ 1 & 2\ 1 & 3\ 1& 4\ 1& 5 \end{bmatrix}$

$B=\begin{bmatrix} 1 \ 2\ 3\ 4\ 5\10 \end{bmatrix}$

To make things simpler, lets make $C=A^T\cdot A$ , and $D=A^T\cdot b$

$C=\begin{bmatrix} 1 & 1&1 &1 &1 &1 \ 0 & 1& 2& 3& 4& 5 \end{bmatrix}.\begin{bmatrix} 1&0 \ 1& 1\ 1& 2\ 1& 3\ 1& 4\ 1& 5 \end{bmatrix}$

$C=\begin{bmatrix} 6 & 15 \ 15 & 55 \end{bmatrix}$

Now we need to solve for the inverse, we can do this simply by doing the following. We flip the sign on the off diagonal, and change the spots on the main diagonal, then we multiply by $\frac{1}{\det(C)}$ .

$\det(C)=55(6)-(15)(15)=105$

$C^{-1}=\frac{1}{105}\begin{bmatrix}55 & -15 \ -15 & 6 \end{bmatrix}$

$D=\begin{bmatrix} 1&1 &1 &1 &1 & 1\ 0&1 &2 &3 &4 &5 \end{bmatrix} . \begin{bmatrix} 1\ 2\ 3\ 4\ 5\ 10\ \end{bmatrix}$

$D=\begin{bmatrix} 25\ 90 \end{bmatrix}$

$x=C^{-1}\cdot D=\frac{1}{105}\begin{bmatrix} 55& -15\ -15& 6 \end{bmatrix} \cdot \begin{bmatrix} 25\ 90 \end{bmatrix}=\begin{bmatrix} \frac{5}{21}\ \ \frac{11}{7} \end{bmatrix}$

Compare your answer with the correct one above

Question

$\begin{align}&\text{Approximate }\begin{bmatrix}x\y\end{bmatrix}\&\text{for the system: }\begin{bmatrix}-12&9\13&14\-5&-16\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-100\-5\14\end{bmatrix}\end{align}$

Answer

$\begin{align}&\text{When the rows of a matrix outnumber the columns in an equation,}\&\text{there are a greater number of equations than variables. In order}\&\text{to approximate the values of these variables, a least-squares}\&\text{approach is appropriate. This is done by multiplying each side}\&\text{of the equation by the transpose of the equation matrix, and}\&\text{solving the resulting equation:}\&Ax=B\&A^{T}Ax=A^{T}B\&(A^{T}A)^{-1}(A^{T}A)x=(A^{T}A)^{-1}A^{T}B\&x=(A^{T}A)^{-1}A^{T}B\&\text{Using this, we can approximate our values:}\&\begin{bmatrix}-12&13&-5\9&14&-16\end{bmatrix}\begin{bmatrix}-12&9\13&14\-5&-16\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-12&13&-5\9&14&-16\end{bmatrix}\begin{bmatrix}-100\-5\14\end{bmatrix}\&\begin{bmatrix}338&154\154&533\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}1065\-1194\end{bmatrix}\&\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}4.80\-3.63\end{bmatrix}\end{align}$

Compare your answer with the correct one above

Question

$\begin{align}&\text{Approximate the values of }\begin{bmatrix}x\y\end{bmatrix}\&\text{for the function: }\begin{bmatrix}15&15\-7&-12\-3&9\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-120\-4\-180\end{bmatrix}\end{align}$

Answer

$\begin{align}&\text{When the rows of a matrix outnumber the columns in an equation,}\&\text{there are a greater number of equations than variables. In order}\&\text{to approximate the values of these variables, a least-squares}\&\text{approach is appropriate. This is done by multiplying each side}\&\text{of the equation by the transpose of the equation matrix, and}\&\text{solving the resulting equation:}\&Ax=B\&A^{T}Ax=A^{T}B\&(A^{T}A)^{-1}(A^{T}A)x=(A^{T}A)^{-1}A^{T}B\&x=(A^{T}A)^{-1}A^{T}B\&\text{Using this, we can approximate our values:}\&\begin{bmatrix}15&-7&-3\15&-12&9\end{bmatrix}\begin{bmatrix}15&15\-7&-12\-3&9\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}15&-7&-3\15&-12&9\end{bmatrix}\begin{bmatrix}-120\-4\-180\end{bmatrix}\&\begin{bmatrix}283&282\282&450\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-1232\-3372\end{bmatrix}\&\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}8.29\-12.69\end{bmatrix}\end{align}$

Compare your answer with the correct one above

Question

$\begin{align}&\text{Approximate }\begin{bmatrix}x\y\end{bmatrix}\&\text{for the system: }\begin{bmatrix}-13&13\9&9\11&-11\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-109\-168\46\end{bmatrix}\end{align}$

Answer

$\begin{align}&\text{When the rows of a matrix outnumber the columns in an equation,}\&\text{there are a greater number of equations than variables. In order}\&\text{to approximate the values of these variables, a least-squares}\&\text{approach is appropriate. This is done by multiplying each side}\&\text{of the equation by the transpose of the equation matrix, and}\&\text{solving the resulting equation:}\&Ax=B\&A^{T}Ax=A^{T}B\&(A^{T}A)^{-1}(A^{T}A)x=(A^{T}A)^{-1}A^{T}B\&x=(A^{T}A)^{-1}A^{T}B\&\text{Using this, we can approximate our values:}\&\begin{bmatrix}-13&9&11\13&9&-11\end{bmatrix}\begin{bmatrix}-13&13\9&9\11&-11\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-13&9&11\13&9&-11\end{bmatrix}\begin{bmatrix}-109\-168\46\end{bmatrix}\&\begin{bmatrix}371&-209\-209&371\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}411\-3435\end{bmatrix}\&\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-6.02\-12.65\end{bmatrix}\end{align}$

Compare your answer with the correct one above

Question

$\begin{align}&\text{Approximate the values of }\begin{bmatrix}x\y\end{bmatrix}\&\text{for the function: }\begin{bmatrix}20&-19\-9&-18\14&-10\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-8\11\37\end{bmatrix}\end{align}$

Answer

$\begin{align}&\text{When the rows of a matrix outnumber the columns in an equation,}\&\text{there are a greater number of equations than variables. In order}\&\text{to approximate the values of these variables, a least-squares}\&\text{approach is appropriate. This is done by multiplying each side}\&\text{of the equation by the transpose of the equation matrix, and}\&\text{solving the resulting equation:}\&Ax=B\&A^{T}Ax=A^{T}B\&(A^{T}A)^{-1}(A^{T}A)x=(A^{T}A)^{-1}A^{T}B\&x=(A^{T}A)^{-1}A^{T}B\&\text{Using this, we can approximate our values:}\&\begin{bmatrix}20&-9&14\-19&-18&-10\end{bmatrix}\begin{bmatrix}20&-19\-9&-18\14&-10\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}20&-9&14\-19&-18&-10\end{bmatrix}\begin{bmatrix}-8\11\37\end{bmatrix}\&\begin{bmatrix}677&-358\-358&785\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}259\-416\end{bmatrix}\&\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}0.13\-0.47\end{bmatrix}\end{align}$

Compare your answer with the correct one above

Question

$\begin{align}&\text{Approximate }\begin{bmatrix}x\y\end{bmatrix}\&\text{for the system: }\begin{bmatrix}14&-16\5&-18\5&-14\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-92\-165\-168\end{bmatrix}\end{align}$

Answer

$\begin{align}&\text{When the rows of a matrix outnumber the columns in an equation,}\&\text{there are a greater number of equations than variables. In order}\&\text{to approximate the values of these variables, a least-squares}\&\text{approach is appropriate. This is done by multiplying each side}\&\text{of the equation by the transpose of the equation matrix, and}\&\text{solving the resulting equation:}\&Ax=B\&A^{T}Ax=A^{T}B\&(A^{T}A)^{-1}(A^{T}A)x=(A^{T}A)^{-1}A^{T}B\&x=(A^{T}A)^{-1}A^{T}B\&\text{Using this, we can approximate our values:}\&\begin{bmatrix}14&5&5\-16&-18&-14\end{bmatrix}\begin{bmatrix}14&-16\5&-18\5&-14\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}14&5&5\-16&-18&-14\end{bmatrix}\begin{bmatrix}-92\-165\-168\end{bmatrix}\&\begin{bmatrix}246&-384\-384&776\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-2953\6794\end{bmatrix}\&\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}7.31\12.37\end{bmatrix}\end{align}$

Compare your answer with the correct one above

Question

$\begin{align}&\text{Using the method of least-squares, approximate x and y values for: }\&\begin{bmatrix}0&16\-2&-18\3&5\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-73\158\-35\end{bmatrix}\end{align}$

Answer

$\begin{align}&\text{When the rows of a matrix outnumber the columns in an equation,}\&\text{there are a greater number of equations than variables. In order}\&\text{to approximate the values of these variables, a least-squares}\&\text{approach is appropriate. This is done by multiplying each side}\&\text{of the equation by the transpose of the equation matrix, and}\&\text{solving the resulting equation:}\&Ax=B\&A^{T}Ax=A^{T}B\&(A^{T}A)^{-1}(A^{T}A)x=(A^{T}A)^{-1}A^{T}B\&x=(A^{T}A)^{-1}A^{T}B\&\text{Using this, we can approximate our values:}\&\begin{bmatrix}0&-2&3\16&-18&5\end{bmatrix}\begin{bmatrix}0&16\-2&-18\3&5\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}0&-2&3\16&-18&5\end{bmatrix}\begin{bmatrix}-73\158\-35\end{bmatrix}\&\begin{bmatrix}13&51\51&605\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-421\-4187\end{bmatrix}\&\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-7.82\-6.26\end{bmatrix}\end{align}$

Compare your answer with the correct one above

Question

$\begin{align}&\text{Using the method of least-squares, approximate x and y values for: }\&\begin{bmatrix}1&-17\-6&1\1&-13\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}188\149\19\end{bmatrix}\end{align}$

Answer

$\begin{align}&\text{When the rows of a matrix outnumber the columns in an equation,}\&\text{there are a greater number of equations than variables. In order}\&\text{to approximate the values of these variables, a least-squares}\&\text{approach is appropriate. This is done by multiplying each side}\&\text{of the equation by the transpose of the equation matrix, and}\&\text{solving the resulting equation:}\&Ax=B\&A^{T}Ax=A^{T}B\&(A^{T}A)^{-1}(A^{T}A)x=(A^{T}A)^{-1}A^{T}B\&x=(A^{T}A)^{-1}A^{T}B\&\text{Using this, we can approximate our values:}\&\begin{bmatrix}1&-6&1\-17&1&-13\end{bmatrix}\begin{bmatrix}1&-17\-6&1\1&-13\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}1&-6&1\-17&1&-13\end{bmatrix}\begin{bmatrix}188\149\19\end{bmatrix}\&\begin{bmatrix}38&-36\-36&459\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-687\-3294\end{bmatrix}\&\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-26.87\-9.28\end{bmatrix}\end{align}$

Compare your answer with the correct one above

Question

$\begin{align}&\text{Approximate the values of }\begin{bmatrix}x\y\end{bmatrix}\&\text{for the function: }\begin{bmatrix}14&-8\-4&-5\-9&-2\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-161\188\30\end{bmatrix}\end{align}$

Answer

$\begin{align}&\text{When the rows of a matrix outnumber the columns in an equation,}\&\text{there are a greater number of equations than variables. In order}\&\text{to approximate the values of these variables, a least-squares}\&\text{approach is appropriate. This is done by multiplying each side}\&\text{of the equation by the transpose of the equation matrix, and}\&\text{solving the resulting equation:}\&Ax=B\&A^{T}Ax=A^{T}B\&(A^{T}A)^{-1}(A^{T}A)x=(A^{T}A)^{-1}A^{T}B\&x=(A^{T}A)^{-1}A^{T}B\&\text{Using this, we can approximate our values:}\&\begin{bmatrix}14&-4&-9\-8&-5&-2\end{bmatrix}\begin{bmatrix}14&-8\-4&-5\-9&-2\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}14&-4&-9\-8&-5&-2\end{bmatrix}\begin{bmatrix}-161\188\30\end{bmatrix}\&\begin{bmatrix}293&-74\-74&93\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-3276\288\end{bmatrix}\&\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-13.01\-7.26\end{bmatrix}\end{align}$

Compare your answer with the correct one above

Question

$\begin{align}&\text{Approximate the values of }\begin{bmatrix}x\y\end{bmatrix}\&\text{for the function: }\begin{bmatrix}-13&12\-12&1\-2&19\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-103\-178\-16\end{bmatrix}\end{align}$

Answer

$\begin{align}&\text{When the rows of a matrix outnumber the columns in an equation,}\&\text{there are a greater number of equations than variables. In order}\&\text{to approximate the values of these variables, a least-squares}\&\text{approach is appropriate. This is done by multiplying each side}\&\text{of the equation by the transpose of the equation matrix, and}\&\text{solving the resulting equation:}\&Ax=B\&A^{T}Ax=A^{T}B\&(A^{T}A)^{-1}(A^{T}A)x=(A^{T}A)^{-1}A^{T}B\&x=(A^{T}A)^{-1}A^{T}B\&\text{Using this, we can approximate our values:}\&\begin{bmatrix}-13&-12&-2\12&1&19\end{bmatrix}\begin{bmatrix}-13&12\-12&1\-2&19\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-13&-12&-2\12&1&19\end{bmatrix}\begin{bmatrix}-103\-178\-16\end{bmatrix}\&\begin{bmatrix}317&-206\-206&506\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}3507\-1718\end{bmatrix}\&\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}12.04\1.51\end{bmatrix}\end{align}$

Compare your answer with the correct one above

Question

$\begin{align}&\text{Using the method of least-squares, approximate x and y values for: }\&\begin{bmatrix}14&-9\-10&-4\-8&-6\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-12\-198\-179\end{bmatrix}\end{align}$

Answer

$\begin{align}&\text{When the rows of a matrix outnumber the columns in an equation,}\&\text{there are a greater number of equations than variables. In order}\&\text{to approximate the values of these variables, a least-squares}\&\text{approach is appropriate. This is done by multiplying each side}\&\text{of the equation by the transpose of the equation matrix, and}\&\text{solving the resulting equation:}\&Ax=B\&A^{T}Ax=A^{T}B\&(A^{T}A)^{-1}(A^{T}A)x=(A^{T}A)^{-1}A^{T}B\&x=(A^{T}A)^{-1}A^{T}B\&\text{Using this, we can approximate our values:}\&\begin{bmatrix}14&-10&-8\-9&-4&-6\end{bmatrix}\begin{bmatrix}14&-9\-10&-4\-8&-6\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}14&-10&-8\-9&-4&-6\end{bmatrix}\begin{bmatrix}-12\-198\-179\end{bmatrix}\&\begin{bmatrix}360&-38\-38&133\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}3244\1974\end{bmatrix}\&\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}10.91\17.96\end{bmatrix}\end{align}$

Compare your answer with the correct one above

Question

$\begin{align}&\text{Approximate }\begin{bmatrix}x\y\end{bmatrix}\&\text{for the system: }\begin{bmatrix}11&4\-8&0\8&20\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-42\-66\-36\end{bmatrix}\end{align}$

Answer

$\begin{align}&\text{When the rows of a matrix outnumber the columns in an equation,}\&\text{there are a greater number of equations than variables. In order}\&\text{to approximate the values of these variables, a least-squares}\&\text{approach is appropriate. This is done by multiplying each side}\&\text{of the equation by the transpose of the equation matrix, and}\&\text{solving the resulting equation:}\&Ax=B\&A^{T}Ax=A^{T}B\&(A^{T}A)^{-1}(A^{T}A)x=(A^{T}A)^{-1}A^{T}B\&x=(A^{T}A)^{-1}A^{T}B\&\text{Using this, we can approximate our values:}\&\begin{bmatrix}11&-8&8\4&0&20\end{bmatrix}\begin{bmatrix}11&4\-8&0\8&20\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}11&-8&8\4&0&20\end{bmatrix}\begin{bmatrix}-42\-66\-36\end{bmatrix}\&\begin{bmatrix}249&204\204&416\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-222\-888\end{bmatrix}\&\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}1.43\-2.84\end{bmatrix}\end{align}$

Compare your answer with the correct one above

Question

$\begin{align}&\text{Using the method of least-squares, approximate x and y values for: }\&\begin{bmatrix}15&-4\19&-15\-2&19\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-195\-163\15\end{bmatrix}\end{align}$

Answer

$\begin{align}&\text{When the rows of a matrix outnumber the columns in an equation,}\&\text{there are a greater number of equations than variables. In order}\&\text{to approximate the values of these variables, a least-squares}\&\text{approach is appropriate. This is done by multiplying each side}\&\text{of the equation by the transpose of the equation matrix, and}\&\text{solving the resulting equation:}\&Ax=B\&A^{T}Ax=A^{T}B\&(A^{T}A)^{-1}(A^{T}A)x=(A^{T}A)^{-1}A^{T}B\&x=(A^{T}A)^{-1}A^{T}B\&\text{Using this, we can approximate our values:}\&\begin{bmatrix}15&19&-2\-4&-15&19\end{bmatrix}\begin{bmatrix}15&-4\19&-15\-2&19\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}15&19&-2\-4&-15&19\end{bmatrix}\begin{bmatrix}-195\-163\15\end{bmatrix}\&\begin{bmatrix}590&-383\-383&602\end{bmatrix}\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-6052\3510\end{bmatrix}\&\begin{bmatrix}x\y\end{bmatrix}=\begin{bmatrix}-11.03\-1.18\end{bmatrix}\end{align}$

Compare your answer with the correct one above

Question

It is recommended that you use a calculator with matrix arithmetic capability for this question.

Give the equation of the least squares regression line for the following data:

, , , .

Round your coefficients to three decimal digits, if applicable.

Answer

Form the matrices $X = \begin{bmatrix} 1 & x_{1} \ 1 & x_{2} \1 & x_{3} \1 & x_{4} \end{bmatrix}$ and $Y = \begin{bmatrix} y_{1} \ y _{2} \ y_{3} \ y_{4}\end{bmatrix}$

using the abscissas and ordinates of the four points:

$X = \begin{bmatrix} 1 & 0 \ 1 & 10 \1 & 20 \1 & 30 \end{bmatrix}$ and $Y = \begin{bmatrix} 128 \194\ 253 \ 330\end{bmatrix}$

The least squares regression line is the line of the equation ,

where $A = \begin{bmatrix} b \ m \end{bmatrix}$ can be found using the equation

$A =( X^{T}X)^{-1} X^{T}Y$ .

This can be calculated as follows:

$X^{T}X =\begin{bmatrix} 4 & 60 \ 60 & 1,400 \end{bmatrix}$

$(X^{T}X )^{-1}=\begin{bmatrix} 0.7 & -0.03 \ -0.03&0.002\end{bmatrix}$

$X^{T}Y = \begin{bmatrix} 905 \ 16,900 \end{bmatrix}$

$A =( X^{T}X)^{-1} X^{T}Y = \begin{bmatrix}126.5 \ 6.65 \end{bmatrix}$

The least squares regression line is the line of the equation

.

Compare your answer with the correct one above

Tap the card to reveal the answer