Linear Systems and Least Squares

Regression is the tool researchers use most, and interviewers test the linear algebra underneath it: "derive the least squares estimator", "what does the normal equation mean geometrically?", "why not just invert the matrix?". The answers follow from one picture. The fitted values are the closest point to the data that the model can reach, and the residual points straight away from everything the model can express.

Linear systems

A system $Ax = b$ with a square matrix $A$ has exactly one solution when $A$ has full rank. When $A$ is rank-deficient, the system has either no solution or infinitely many, depending on $b$ .

Regression is the other case: more equations than unknowns. With $n$ observations and $p$ coefficients, $n > p$ , the system $X\beta = y$ usually has no exact solution, because $n$ data points do not lie exactly on a $p$ -parameter model. So we choose the $\beta$ that makes the error as small as possible.

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Integer sit amet quam auctor, convallis ligula a, aliquet turpis. Vestibulum fringilla sollicitudin leo sit amet lobortis. Nullam accumsan finibus justo quis sollicitudin. Nam sed volutpat nisl, efficitur convallis purus. Curabitur scelerisque tincidunt turpis, vel ullamcorper magna faucibus a. Etiam non lobortis elit, vel egestas ex. Maecenas quis egestas velit. Pellentesque sit amet dolor laoreet turpis convallis elementum. Cras nec dui non turpis tincidunt maximus vitae commodo elit.

Vivamus maximus leo nec sem ultricies, placerat luctus odio scelerisque. Morbi fringilla est neque. Duis fermentum tempor justo ut laoreet. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Etiam feugiat ipsum in nisl fermentum commodo. Aenean viverra, nisl sed gravida facilisis, felis sem malesuada orci, non tincidunt velit risus id risus. Integer gravida fermentum massa, eu viverra felis iaculis non. Suspendisse sit amet hendrerit erat, eget accumsan libero.

Aenean ut quam efficitur felis pulvinar pulvinar quis in libero. In hac habitasse platea dictumst. Suspendisse potenti. Maecenas vel gravida sem. Aliquam aliquam pellentesque cursus. Sed malesuada lacus dignissim odio varius commodo. Vivamus tristique facilisis lorem.

The rest of this lesson is for subscribers

Unlock every lesson in Basics of Quantitative Finance, and every other premium course.

Subscribe to continue

Test your knowledge

Questions are only available to subscribers.

Continue

Keep reading Basics of Quantitative Finance

30 lessons in this course, and every other premium course, on one subscription.

Subscribe now

Every lesson in every course, with the worked examples and interactive simulators
Graded questions on every lesson, with explanations for the wrong answers as well as the right one
The trainers, timed assessments and brainteaser library that go with them