Hypothesis-First Research
The previous section established that the value of backtest evidence depends on the base rate of the idea and on how many things were tried. Both of those are set before any code runs, which makes research design the highest-leverage validation tool there is. The cheapest fix for a biased experiment is not to run it.
Mechanism before mining
Hypothesis-first research starts from a statement of why an edge should exist: who is on the other side, why they trade at a disadvantage, and why the effect has not been arbitraged away. Common mechanism families are risk premia (compensation for holding what others cannot), structural flows (rebalancing, hedging, index changes that must trade regardless of price) and constraints (mandates, leverage limits, benchmarks that force predictable behaviour).
The mechanism then makes predictions beyond the headline return: where the edge should be strongest, when it should weaken, what it should correlate with. Each confirmed side prediction is cheap additional evidence; a strategy whose only supporting fact is its own backtest has exactly one data point.
The rest of this lesson is for subscribers
Unlock every lesson in Research Validation and Backtesting, and every other premium course.
Subscribe to continueTest your knowledge
Keep reading Research Validation and Backtesting
21 lessons in this course, and every other premium course, on one subscription.
- Every lesson in every course, with the worked examples and interactive simulators
- Graded questions on every lesson, with explanations for the wrong answers as well as the right one
- The trainers, timed assessments and brainteaser library that go with them