Back to course

Statistical vs Practical Significance

Statistical significance says an effect is distinguishable from noise. Practical significance says it is large enough to matter. They are independent, and confusing them wastes a lot of capital.

The link is sample size. The test statistic

t=δ^σ/nt = \frac{\hat{\delta}}{\sigma/\sqrt{n}}

grows with n\sqrt{n} for any non-zero δ^\hat{\delta}. So any true effect, however tiny, becomes statistically significant with enough data. Significance is a statement about your measurement precision as much as about the world.

The case that matters in trading

A strategy shows a mean edge of 0.008%0.008\% per trade over 500,000 trades. The standard error is minute, the p-value is essentially zero, and the edge is unambiguously real.

It is also worthless. Round-trip costs of 0.02%0.02\% turn a real +0.008%+0.008\% gross edge into a reliable −0.012%-0.012\% net loss. Statistically significant, economically negative, and the more you trade it the more certainly you lose.

The rest of this lesson is for subscribers

Unlock every lesson in Fundamentals of Probability and Statistics, and every other premium course.

Subscribe to continue

Test your knowledge

Questions are only available to subscribers.
Continue
NextAdvanced Frequentist Tools

Keep reading Fundamentals of Probability and Statistics

41 lessons in this course, and every other premium course, on one subscription.

Subscribe now
  • Every lesson in every course, with the worked examples and interactive simulators
  • Graded questions on every lesson, with explanations for the wrong answers as well as the right one
  • The trainers, timed assessments and brainteaser library that go with them