Back to course

Forecast Metrics for Returns

"How would you evaluate a model that forecasts stock returns?" R2R^2 and the share of correct signs are a start, but neither means much alone. A full answer names the benchmark that R2R^2 is measured against. It measures the correlation across stocks on each date and tests its mean. It then turns that correlation into an expected Sharpe ratio and states the assumptions. This lesson builds that answer.

Out-of-sample R2R^2

Out-of-sample R2R^2 compares the squared error of the forecast r^t\hat{r}_t with that of a benchmark forecast btb_t, both made with information available before tt:

ROS2=1−∑t(rt−r^t)2∑t(rt−bt)2R^2_{\text{OS}} = 1 - \frac{\sum_t (r_t - \hat{r}_t)^2}{\sum_t (r_t - b_t)^2}

A positive value means the model beats the benchmark. The number depends on the benchmark, and there are two usual choices.

  • The historical mean, the average of past returns up to tt. For a single stock this estimate is very noisy, so it is a weak benchmark. A model can beat it simply by predicting values closer to zero.
  • Zero. For individual stocks, a forecast of zero is hard to beat, so it is the stricter test.

The rest of this lesson is for subscribers

Unlock every lesson in Machine Learning for Quantitative Research, and every other premium course.

Subscribe to continue

Test your knowledge

Questions are only available to subscribers.
Continue

Keep reading Machine Learning for Quantitative Research

27 lessons in this course, and every other premium course, on one subscription.

Subscribe now
  • Every lesson in every course, with the worked examples and interactive simulators
  • Graded questions on every lesson, with explanations for the wrong answers as well as the right one
  • The trainers, timed assessments and brainteaser library that go with them