Back to course

Calibration and Brier Scores

Everything so far assumed you can produce a probability worth trading on. That is a testable claim, and this market is unusual in handing you the test: every position is a forecast, every resolution is a grade. Professionals in event markets track their own calibration the way equity traders track P&L, and firms hiring for these desks explicitly like candidates who arrive with a scored forecasting record.

Calibration

A forecaster is calibrated if, among all the times they said 70%70\%, the event happened about 70%70\% of the time, and likewise at every other level. Calibration is checked by bucketing your forecasts, comparing each bucket's average forecast to its resolved frequency, and plotting one against the other: the calibration curve. Systematic deviations are personality flaws made visible, overconfidence bends the curve one way, timidity the other, and the longshot bias from the previous lesson is exactly such a bend in the market's own curve.

The rest of this lesson is for subscribers

Unlock every lesson in Prediction Markets Trading, and every other premium course.

Subscribe to continue

Test your knowledge

Questions are only available to subscribers.
Continue
NextMaking a Market in a Binary

Keep reading Prediction Markets Trading

14 lessons in this course, and every other premium course, on one subscription.

Subscribe now
  • Every lesson in all seven courses, with the worked examples and interactive simulators
  • Graded questions on every lesson, with explanations for the wrong answers as well as the right one
  • The trainers, timed assessments and brainteaser library that go with them