Bias da data mining
Data mining bias is finding patterns in historical data that appear profitable only by chance, because many rules or parameters were tested until one worked.
Explained
The more variations tested, the more likely a spurious winner. Defences include economic logic, out-of-sample testing, fewer parameters and adjusting for multiple tests.
Example
A trader tests 500 indicator combinations and picks the best backtest.
In live trading the strategy performs no better than random.
Common mistakes
Not counting how many tests were run.
Skipping out-of-sample checks.
Building strategies without a rationale.
Questions traders ask
How do I avoid data mining bias?
Limit tests, require logic and validate out of sample.
Is it the same as overfitting?
Closely related: overfitting tunes one model too closely to the data.