Overfitting, Misspecification, and Model Risk
Three failures, often conflated, with different causes and different remedies.
Overfitting
The model learned noise. It fits history closely and generalises badly.
Signature: excellent in-sample performance, poor out-of-sample.
Causes: too many parameters relative to data, and searching many specifications and reporting the winner. The second is the more insidious, because each individual choice feels reasonable.
Defences: hold out data you touch once, prefer simpler models, use regularisation, and record how many variants you tried. That last one converts an unquantified bias into something you can correct for.
Misspecification
The model has the wrong form. Not too complex, just wrong: an omitted variable, an inappropriate distribution, a linear model for a non-linear relationship.
Signature: structure in the residuals. Errors that are not random tell you what the model failed to capture, which is why residual plots are the first diagnostic.
The rest of this lesson is for subscribers
Unlock every lesson in Basics of Quantitative Finance, and every other premium course.
Subscribe to continueTest your knowledge
Keep reading Basics of Quantitative Finance
23 lessons in this course, and every other premium course, on one subscription.
- Every lesson in every course, with the worked examples and interactive simulators
- Graded questions on every lesson, with explanations for the wrong answers as well as the right one
- The trainers, timed assessments and brainteaser library that go with them