A backtest answers a narrow question: if these exact rules had been followed on these past prices, with these costs, what would the account have looked like? It is a simulation, not a record, and every number it produces depends on its assumptions. This lesson walks through how the simulation works and how to read what comes out, so that you can tell a fair test from a flattering one.
What a backtest does
The engine moves through history one bar at a time. On each day it updates the indicators with that day’s data, checks whether any open position should be closed, checks for new entries, sizes them from the current equity, and records the result. Then it moves to the next day. The essential discipline is that on each day the engine may use only data up to that day: any glance at a later bar, even indirectly, is look-ahead bias (lesson 6).
The engine also has to make assumptions that the rules themselves do not state: the starting capital, whether idle cash earns interest, what happens when there is not enough cash for a new position, and whether margin is allowed. Clear Tape’s backtest, for example, starts each test from $100,000, risks a fixed share of equity per trade, caps any one stock at 20% of equity, uses no margin and skips a signal when cash runs out. Different assumptions give different numbers, so they belong next to the result.
Data and the benchmark
A test needs daily bars adjusted for splits, otherwise a 10-for-1 split looks like a 90% crash (lesson 6). It also needs enough history to include different kinds of market: rising, falling and sideways years. A rule tested only on 2017, one of the calmest years on record for US stocks, has not been tested on the conditions it will eventually meet.
A result means little on its own. “The rule made 9% a year” is impressive or disappointing depending on what the same money would have done otherwise. That alternative is the benchmark, and choosing it is a decision with consequences:
- SPY (the S&P 500 ETF) is the natural benchmark for a rule that trades large US stocks or the index itself.
- Buying and holding the same stock is the right comparison for a rule that times one stock.
- Buying and holding the same group of stocks in equal weights is the fairest comparison for a rule that picks from a list. If the list itself is biased (for example, it contains only today’s survivors), the benchmark shares the bias, so the comparison stays fair. Clear Tape uses this benchmark for its verdicts and shows SPY only as a reference line.
A benchmark that is too easy (cash, or an index the rule never trades) makes any rule look clever.
Costs and slippage
Every trade pays a commission (often zero now for US stocks), the bid–ask spread and slippage: the difference between the price the test assumes and the price you actually get. A common simple model charges a fixed number of basis points (hundredths of a percent) on each buy and each sell. At 10 basis points per side, a round trip costs 0.20% of the position. That sounds trivial until it is multiplied by turnover:
For liquid US stocks above $5, 10 basis points per side is a reasonably conservative default; for small, thinly traded stocks it is too low. A useful habit is to rerun every promising test at double the cost. An edge that disappears at 20 basis points per side was too thin to trade. A test run with zero costs should be labelled “gross” wherever its numbers appear.
The key numbers
CAGR (compound annual growth rate) turns a total gain into a yearly rate: CAGR = (ending value ÷ starting value)^(1 ÷ years) − 1. For example, an account that grows from $100,000 to $180,000 in 8 years has a total return of 80% but a CAGR of 7.62%. CAGR allows tests of different lengths to be compared.
The Sharpe ratio, introduced by William F. Sharpe in 1966, measures return per unit of volatility after subtracting what cash would have earned. On yearly figures: (return − risk-free rate) ÷ volatility. For example, a rule earning 9% a year with 15% volatility while Treasury bills pay 3% has a Sharpe ratio of (9 − 3) ÷ 15 = 0.40; a benchmark earning 8% with 18% volatility scores (8 − 3) ÷ 18 = 0.28. The rule earns more per unit of risk. In practice the ratio is computed from daily returns: the average daily excess return over the 3-month Treasury bill rate, divided by its standard deviation, multiplied by the square root of 252 trading days. Leaving out the risk-free rate overstates every Sharpe ratio, and most of all that of a calm, low-volatility strategy: in years when cash paid 5%, earning 6% with little risk is a small achievement, not a large one.
Maximum drawdown is the largest fall from a peak in equity to a later low, before a new peak. An account that goes 100 → 140 → 105 → 150 has a maximum drawdown of 25% (from 140 to 105), even though it ended higher. Drawdown is the number that tests whether you will keep following the rules; see maximum drawdown.
Exposure (shown in Clear Tape as time in the market) is the share of time the strategy holds positions: in the market on 130 of 252 days is 51.6% exposure. A rule with low exposure can have a lower CAGR than the benchmark and still a better Sharpe ratio, because it spends much of its time in cash.
Trade count says how much evidence stands behind the other numbers. Twenty trades can be explained by luck; several hundred are harder to dismiss. Win rate, average win and loss, and expectancy per trade (trading expectancy) complete the picture.
| Made-up example (10 years) | Timing rule | Buy and hold |
|---|---|---|
| CAGR | 8.8% | 10.5% |
| Sharpe ratio (cash at 2%) | 0.58 | 0.53 |
| Maximum drawdown | −23.0% | −31.0% |
| Volatility | 14.0% | 18.3% |
| Exposure | 64.4% | 100% |
| Trades | 30 | 1 |
Which is better depends on what you value. The rule lost on return and won on risk-adjusted return and drawdown: in lesson 4’s terms, that is a trade-off, not a clear win. Notice also that 30 trades over ten years is thin evidence.
Total return versus price return
A price chart ignores dividends. For a stock or ETF that pays them, the total return (price change plus reinvested dividends) is higher than the price return, and the gap compounds. At a 7% yearly price gain plus a 2% dividend yield, ten years turn 1.0 into 1.97 on price alone but 2.40 with dividends reinvested.
This matters in a backtest because the comparison must be like for like. If the benchmark is measured on price only while the rule collects dividends, or the other way round, the result is off by roughly the dividend yield every year, which for broad US indexes has recently been in the region of 1.5 to 2 percentage points, as large as many genuine edges. The same goes for cash: a rule that is out of the market half the time should earn the Treasury bill rate on that cash. Clear Tape’s holdout test counts dividends for both the strategy and the benchmark and pays the 3-month Treasury bill rate on idle cash.
How to apply it
Entry
Before running a test, write down the benchmark, the costs per side, the fill (next open by default), the starting capital and whether dividends and cash interest are included.
Stop
Rerun any promising result at double the costs and with next-open fills. Treat an edge that disappears as too thin to trade.
Target
Judge the result on CAGR and Sharpe ratio against the benchmark, the depth and length of the drawdowns, and the trade count, never on CAGR alone.
Invalidation
A result that relies on zero costs, signal-day fills, a price-only benchmark against a total-return strategy, or fewer than about 20 trades is not yet evidence.
Common mistakes
- Comparing a rule with cash, or with an index it never trades.
- Running tests at zero cost and quoting the result without the word “gross”.
- Computing a Sharpe ratio without the risk-free rate.
- Comparing total return with price return.
- Judging a rule on its best single number and ignoring the trade count behind it.
Summary
A backtest replays rules one day at a time using only data available on that day, under assumptions about capital, cash, costs and fills that should be stated with the result. Compare against a fair benchmark, ideally the same stocks bought and held. Charge realistic costs and slippage, since turnover multiplies them. Read CAGR and the Sharpe ratio together, with maximum drawdown, exposure and trade count beside them, and make sure dividends and cash are treated the same way on both sides. The next lesson explains why even a fair backtest can mislead, and how a holdout period helps.
For educational purposes only. Not investment advice.