Walk-forward optimization: avoiding overfitting
Walk-forward optimization (WFO) is the gold standard for validating trading strategies, but it's easy to overfit even with WFO.
Common pitfalls:
My rule of thumb: if you test more than 20 parameter combinations, your results are likely overfit.
59 Replies
For those wondering: yes, you can use external Python packages in submissions, but they must be in the approved list. No custom C extensions.
I've found that sector neutrality is a key factor in the scoring. Strategies that are long one sector and short another tend to underperform.
Has anyone tried using attention mechanisms for this? The temporal attention weights could tell you which historical periods are most relevant.
The competition scoring docs could definitely be clearer. I spent 2 hours debugging what turned out to be a normalization issue.
Be careful with the Kelly criterion for position sizing. Full Kelly is way too aggressive. I use quarter-Kelly in practice.
I think the platform should add a paper trading mode so we can test strategies in a more realistic setting between competitions.
Good point about overfitting. My rule of thumb: never trust a backtest with fewer than 500 observations in the out-of-sample period.
Good point about overfitting. My rule of thumb: never trust a backtest with fewer than 500 observations in the out-of-sample period.