Cross-Validation and the Choice of k
"How would you choose in k-fold cross-validation?" A complete answer treats the cross-validated error as an estimate with a bias and a variance of its own. It then says what that estimate measures, and when the method fails. This lesson covers each part.
Hold-out and k-fold
A hold-out split fits on 80% of the data and scores the other 20%. It costs one fit, but the score depends on which rows landed in the test set, and a fifth of the data never trains the model.
k-fold cross-validation tests every row once. Split the rows into folds of about equal size. For each fold , fit on the other folds and compute the error on fold . The estimate is the average of the fold errors:
The rest of this lesson is for subscribers
Unlock every lesson in Machine Learning for Quantitative Research, and every other premium course.
Subscribe to continueTest your knowledge
Keep reading Machine Learning for Quantitative Research
27 lessons in this course, and every other premium course, on one subscription.
- Every lesson in every course, with the worked examples and interactive simulators
- Graded questions on every lesson, with explanations for the wrong answers as well as the right one
- The trainers, timed assessments and brainteaser library that go with them