Ask any algorithmic trader what killed their first strategy and you'll hear the same guilty confession: it looked incredible in backtests. Equity curve smoother than a fresh jar of Vegemite. Maximum drawdown practically invisible. Then live trading started and reality hit like a freight train. The question of why backtests lie — and how to catch them lying — is genuinely one of the most important questions in systematic trading.
The direct answer is this: over-optimisation, often called curve fitting, happens when you tune a strategy's parameters so precisely to historical data that you've essentially memorised the past rather than modelled it. Walk-forward analysis is the standard antidote. Instead of optimising once across all available data, you optimise on a rolling "in-sample" window, then test immediately on the "out-of-sample" period that follows — repeating this process forward through time. If performance degrades badly at each out-of-sample step, your parameters were fitted to noise, not signal.
Think of it like training for a pub trivia night using last week's exact questions. You'd score perfectly in practice. Show up to a different quiz and you'd be useless. Curve-fitted parameters are that trivia cheat — brilliant at the questions they've seen, hopeless at anything new. The strategy hasn't learned how markets behave; it's learned how these specific price bars moved, in this specific sequence, once.
The chart above illustrates the classic curve-fitting signature: in-sample returns stay high across every walk-forward window while out-of-sample returns steadily decay. A robust strategy shows both bars staying roughly proportional. When that gap widens over successive windows, the parameters are pathologically data-specific. Research published on walk-forward optimisation consistently shows that strategies with fewer, economically-justified parameters survive out-of-sample testing far better than over-parameterised alternatives. The practical rule many systematic traders follow is a minimum ratio: at least five in-sample bars for every one parameter being optimised. Fewer than that and you're effectively asking the optimiser to draw a straight line through two points and call it a trend. Monitoring overfitting in quantitative models is a discipline in itself, and one worth building into every strategy review process. Good reading on parameter robustness also lives in the backtesting fundamentals literature, which frames how historical simulation should — and shouldn't — inform forward expectations.
The practical takeaway you can use today: run your next strategy through at least six walk-forward windows and calculate the out-of-sample efficiency ratio — total out-of-sample return divided by total in-sample return. Anything below 0.5 deserves serious scrutiny before a single live dollar touches it.
A great backtest is a hypothesis, not a promise. Walk-forward analysis is how you find out whether you've built a strategy or just a very expensive history lesson.
This content is for educational purposes only and does not constitute financial product advice. Past performance is not indicative of future results. Profit Logic Ltd (ACN 688 669 936) accepts no responsibility for errors or omissions in this content or anywhere on this website. Always seek advice from a licensed financial adviser before making investment decisions.