Passing a holdout test is a reason to start trading a rule carefully, not a reason to trust it fully. The test was run on clean data with assumed costs, and you chose this rule because its numbers looked better than others you tried. The market it will now meet may differ from both test periods. This lesson covers the bridge between the two: how to start, what to measure and when to stop.
Expect live results to be weaker
Live results trail backtests for reasons that are predictable, even if their size is not:
- Costs. Spreads widen at the open and in volatile markets, orders fill a little worse than the assumed price, and occasionally a gap jumps a stop.
- Selection. You trade the rule that tested best among those you tried. Part of its result was luck (lesson 5), and luck does not repeat on demand.
- Regime. The test covered certain markets. A rule that thrived on steady trends may meet years of choppy ranges.
- Execution. A late order, a skipped signal or a position sized by feel rather than by formula changes the result, usually for the worse.
A sensible working assumption is that the live edge will be smaller than the tested one, perhaps much smaller. If the rule is worth trading only at its full backtested edge, it is probably not worth trading.
Start small
The first months of live trading test two things the backtest could not: whether the edge survives real fills, and whether you follow the rules. Neither needs full size. A common approach is to start at a quarter to a half of the planned risk per trade (for example 0.25% instead of 1% of equity) and to increase it in steps only after a set number of trades has been completed according to the rules.
Starting small also buys time, which you need, because live evidence accumulates slowly. A swing rule taking 40 trades a year needs a year to produce as many trades as a weak backtest; a trend rule taking 12 a year needs three. Size up because the rules have been followed and the numbers are inside the expected range, not because of a few good weeks.
Compare the journal with the backtest
A trading journal records every live trade: date, entry, stop, exit, size, the reason for the exit and the result in R (the multiple of the initial risk). Compared with the backtest, it answers two questions: am I trading the rule as tested, and is the rule behaving as tested?
| Compare | Backtest | Journal | What a gap suggests |
|---|---|---|---|
| Trades per month | From the test | Count | Skipped signals, or a different universe |
| Win rate, average win and loss in R | From the test | From R results | Execution differences, or a change in behaviour |
| Fill versus signal price | Assumed cost per side | Measured | Costs underestimated |
| Holding time, exposure | From the test | Measured | Exits not followed |
| Rules followed | Always | Yes or no per trade | Discipline, not the rule, is the problem |
The cumulative result needs particular care, because short samples are noisy. If a backtest shows an average of +0.25R per trade with a typical spread of 1.3R around it, the cumulative result after n trades is centred on 0.25 × n, give or take about 1.645 × 1.3 × √n in 90% of cases (assuming independent trades). After 40 trades that range runs from about −3.5R to +23.5R; after 100 trades, from about +3.6R to +46.4R.
Two lessons follow. A real edge can easily show a loss after 30 or 40 trades, so a short losing run is not evidence that the rule is broken. And a live path that falls below the bottom of the fan after a meaningful number of trades is evidence: either the edge was smaller than tested, or it has gone. The trading journal and trading expectancy articles explain the measures in more detail; Clear Tape’s trading journal records trades in R and groups the statistics by trading system.
When to drop a strategy
The decision to stop is easiest to make well before it is needed. Write the criteria next to the rule on the day you start, for example:
- Pause and review if the live drawdown exceeds the deepest backtest drawdown by a margin you fix in advance (many traders use 1.5 times), or if a statistic such as the win rate sits far outside the tested range after a set number of trades.
- Stop if the cumulative live result falls below the lower edge of the expected range after a meaningful sample, or if a repeat of the holdout test on newer data turns the verdict to No edge.
- Stop if the reason behind the rule no longer holds: a change in market structure, trading hours, costs or the instrument itself.
- Stop if you cannot follow the rule. A rule you keep overriding is not the rule being tested.
Equally, write down what is not a reason to stop: one losing month, a drawdown within the tested range, or another strategy doing better this quarter. Abandoning a sound rule in an ordinary drawdown, and adopting whatever has just worked, is one of the most reliable ways to buy high and sell low in strategies as well as stocks.
Regime change
A market regime is a stretch of time with a recognisable character: steady trend or choppy range, calm or volatile. Each kind of rule has regimes it likes and regimes it dislikes:
Regimes matter for live trading in two ways. First, a rule can spend a long time in a regime that does not suit it, producing a long, discouraging drawdown that is still entirely normal for that rule; the training and holdout periods should have shown you how long. Second, regimes can change abruptly. The pace of a decline in a volatile regime can exceed anything in the test: a rule whose worst tested drawdown built up over months may meet one that arrives in weeks. Sizing for a deeper drawdown than the test showed is the practical defence.
Trying to predict regimes and switch rules accordingly is itself a strategy, and a hard one; it needs its own rules and its own test. For most individual traders it is simpler to run a rule whose tested history already includes several regimes, size it modestly and judge it over a full cycle.
How to apply it
Entry
Start a tested rule at a quarter to a half of the planned risk per trade, with stop criteria written down beside it.
Stop
Pause when the live drawdown or a key statistic passes the limits you set in advance; stop when the cumulative result leaves the expected range after a meaningful sample or the reason behind the rule disappears.
Target
Increase size in steps after a set number of trades taken by the rules, with live statistics inside the backtest’s range.
Invalidation
If the journal shows the rules are not being followed, fix execution before judging the rule.
Common mistakes
- Starting at full size because the backtest looked convincing.
- Judging a rule on its first twenty trades.
- Not keeping a journal, so live results cannot be compared with the test.
- Stopping in an ordinary drawdown, then starting the strategy that has just had a good run.
- Changing the rules during a trade or a drawdown, rather than after a review.
Summary
Expect live results to be weaker than the backtest because of costs, selection, regime and execution. Start small, increase size only after the rules have been followed for a set number of trades, and keep a journal in R. Compare the journal with the backtest on frequency, win rate, payoff, costs and discipline, and judge the cumulative result against the range the backtest implies, which is wide for small samples. Write down when you will pause and when you will stop before you begin. Regimes change, sometimes quickly; size for a worse drawdown than the test showed. The final lesson puts the whole process into practice in Clear Tape.
For educational purposes only. Not investment advice.