Overfitting and the Train-Test Gap
"What is overfitting, and how would you know if your model is doing it?" opens the machine learning round at many systematic funds. A definition alone is not enough. The interviewer wants the diagnosis, an explanation of why financial data makes overfitting so easy, and an example from your own work.
The definition
A model overfits when it fits the noise in its training sample as well as the signal, so that it performs worse on new data than its fit on the training data suggests. A more useful working version: a model overfits when a simpler model would do better out of sample.
Every sample contains both signal, which repeats in new data, and noise, which does not. A flexible enough model can fit both. The part of the fit that came from noise is lost the moment the model meets new data.
The rest of this lesson is for subscribers
Unlock every lesson in Advanced Topics in Probability and Statistics, and every other premium course.
Subscribe to continueTest your knowledge
Keep reading Advanced Topics in Probability and Statistics
47 lessons in this course, and every other premium course, on one subscription.
- Every lesson in every course, with the worked examples and interactive simulators
- Graded questions on every lesson, with explanations for the wrong answers as well as the right one
- The trainers, timed assessments and brainteaser library that go with them