Decision Trees and Bagging
A typical interview question is: "Explain bagging, and why a random forest samples features." A follow-up asks when the method would mislead you on financial data. Both answers come from one idea. A single deep tree has low bias and high variance. Averaging many trees removes variance, but only as far as the trees disagree.
How a tree splits
A regression tree divides the data with a sequence of yes-or-no questions of the form "is feature below threshold ?". At each node it tries every feature and every threshold, and keeps the split that most reduces the squared error within the two resulting groups. A leaf predicts the mean of the training targets that reach it. Classification trees do the same with an impurity measure, such as Gini impurity or entropy, in place of squared error.
The rest of this lesson is for subscribers
Unlock every lesson in Machine Learning for Quantitative Research, and every other premium course.
Subscribe to continueTest your knowledge
Keep reading Machine Learning for Quantitative Research
27 lessons in this course, and every other premium course, on one subscription.
- Every lesson in every course, with the worked examples and interactive simulators
- Graded questions on every lesson, with explanations for the wrong answers as well as the right one
- The trainers, timed assessments and brainteaser library that go with them