The previous seven lessons described a process. This one runs it, step by step, in the Backtest tab of the Trading systems page. The aim is not to find a winning rule in one evening. It is to run one honest study from start to finish, so that every later study follows the same routine. Backtests are available to Tools and Pro members. The tab’s own “How to read a backtest” guide links back to the lessons of this course.
Step 1: Pick a strategy from the library
Open Start from the library. It collects published rule sets in two groups. Clear Tape presets are the preset trading systems, each with its origin, rulebook and known weaknesses. Classic rules are six rules from the trading literature, written in Clear Tape’s rule language:
- 200-day moving-average timing, after Mebane Faber’s 2007 study;
- a 50/200-day moving-average crossover, a widely used rule; the closest academic test is the 1992 study by William Brock, Josef Lakonishok and Blake LeBaron, which tested shorter pairs such as 1/200 and 2/200, with and without a 1% band, but not 50/200 itself;
- the Turtle 55/20 breakout and the Donchian 20/10 channel, after Richard Donchian’s channel rules as described by Curtis Faith;
- an RSI(2) dip with an exit above the 5-day average, after Larry Connors and Cesar Alvarez;
- a 50-day breakout on volume, after William O’Neil’s general principle of breakouts on heavy volume.
Every entry shows its source and a line headed Differs from the source: a month-end rule run on daily closes, a buffer the original did not have, an intraday breakout filled at the close. Read that line before you run anything. A library rule is a starting point for research, not a recommendation, and its published results were obtained under conditions different from yours.
For a first study, choose a simple classic rule and press Run holdout test on its card. That selects the rule, the long-history ETFs (index and sector ETFs such as SPY, QQQ and IWM) and the holdout test, and runs it. ETFs avoid two problems from earlier lessons at once: they carry far less survivorship bias than today’s stock lists (lesson 6), and they cover the full 2000–2014 training period, so the verdict will not be Not enough data for lack of history.
Step 2: Run the holdout test
You can also fill in the form yourself. Its settings, and a sensible place to start:
| Setting | Choices | Suggested start |
|---|---|---|
| System | Your systems, the presets or the library | A classic rule |
| Stocks | My watchlist, one ticker, the long-history ETFs, or the screener universe (the 500 most liquid stocks) | The long-history ETFs, or one of them |
| Test | Holdout test (2000–2014, then 2015 to now) or a recent window of 1, 3, 5 or 10 years | Holdout test |
| Entry fill | Next day’s open (conservative) or the signal day’s close | Next day’s open |
| Risk per trade | 0.1% to 5% of equity | 1% |
| Costs per side | 0 to 100 basis points, slippage included | 10 (the default) |
Leave the parameter sweep off for the first run. Setting costs to zero is allowed, but the verdict then carries a Gross label, as it should (lesson 3). A recent-window test still works as before, but it measures price returns only and gives no verdict; the holdout test is the one this course is about.
Step 3: Read the verdict
The result opens with the verdict, one of the four from lesson 4, each shown in the same size and place:
Read the result in this order:
- The verdict and its reasons. Below it, each reason is listed in plain words: which measure was won or lost in which period, or that a period had fewer than 20 closed trades. The reasons tell you why, which matters more than the label.
- Read with these results. The disclosures are always open: whether costs were deducted, that returns include dividends and idle cash earns the 3-month Treasury bill rate, that the benchmark is the same stocks bought and held in equal weights and rebalanced monthly (SPY is shown for reference only), the survivorship note (“Bias: high” for the screener universe), and that the results are hypothetical.
- Training and holdout. The two periods side by side: CAGR, Sharpe ratio, maximum drawdown, closed trades and time in the market, each against the benchmark, with “Beat the benchmark” or “Did not beat the benchmark” on the two measures that decide the verdict.
- Growth of $100,000, each period on its own. The system, the same stocks in equal weight and SPY, starting again from $100,000 in each period. Look at the shape as well as the end point: when did the rule fall behind, and did it recover?
- Every signal. The trade list with each exit reason, as in any backtest.
Whatever the verdict, it describes these rules on these prices under these assumptions. It is not a forecast and not a signal to buy.
Step 4: Run a sweep and read the heat map
If the rule has a number you chose, a sweep shows whether the result depends on it. Sweeps work on rule-language systems: your own, and the library’s classic rules (the presets are fixed). Turn on Parameter sweep (training period only), pick one or two parameters, such as the exit buffer of the 200-day timing rule or the lookbacks of a channel breakout, and enter 2 to 8 values for each, including the current one so that your own setting is part of the grid. A sweep allows up to two parameters and 50 stocks, with up to 20 combinations on Tools and 30 on Pro.
The heat map is coloured by the training period only: each cell shows how far its Sharpe ratio exceeded the benchmark’s in 2000–2014, cells with fewer than 20 trades are marked as such and not scored, and your own setting is outlined. Beside the grid, a line applies the plateau rule from lesson 5: Plateau, Isolated peak, no setting beating the benchmark, or too few neighbouring settings to judge, together with the number of variants tested. The holdout verdict is shown for your own settings only, so the holdout period stays a fair test.
A plateau that includes your settings supports the study. An isolated peak, or a plateau far from your settings, is a reason for doubt, not for switching to the brightest cell.
Step 5: Save it to your research journal
Select Save study and give it a short title. The study keeps the rules, the stocks (the full list, so it can be repeated exactly), the settings, the verdict and the summary numbers, and appears under Saved studies with its verdict and the date its data runs to. Save every study, including No edge ones: the journal is a record of what you tried, not a gallery of what worked. How many studies you can keep depends on your plan, and Download all (JSON) exports them.
Once a month, Clear Tape re-runs each saved study on the longer holdout period. Most of the time the verdict does not change, since a month adds little data. When it does, the study shows “Verdict changed” with the old and new verdicts; Pro members can also choose to be notified. If Clear Tape changes how a test is calculated, affected studies are marked “Method updated”, because verdicts before and after are not directly comparable. You can also re-check a study yourself or delete it at any time.
A worked session
Here is how one evening might go. The outcomes are hypothetical; run the test yourself to see the real ones. You open the library, choose 200-day moving-average timing and read that it differs from the paper in two ways: daily closes against a 200-day average instead of month-end closes against a 10-month one, and a 3% buffer on exit. You press Run holdout test. Suppose the verdict comes back as a Trade-off, and the reasons show the rule beat the benchmark’s Sharpe ratio in both periods but not its CAGR in the holdout. The growth charts would then show where the rule sat out declines and where it lagged in advances. You sweep the exit buffer from 0% to 6%; suppose the line reads Plateau, with your 3% inside it. You save the study as “200-day timing, 3% buffer, ETFs” and note whether smaller drawdowns are worth a lower return to you. That decision is yours; the study only makes it an informed one.
Had the verdict been No edge, the session would have been just as useful: one idea tested properly, recorded and set aside.
How to apply it
Entry
Start with a simple classic rule on the long-history ETFs, at next-day-open fills, 1% risk and 10 basis points per side, and read the “Differs from the source” line first.
Stop
Run the holdout test once per version of the rule. If you change the rule, save the new version as a separate study.
Target
A saved study with a verdict you understand, its reasons, a sweep around your settings and a note of what you would do next.
Invalidation
Treat a verdict as unusable if it rests on zero costs, on a stock list chosen with hindsight without reading the bias note, or on settings picked from the sweep after the fact.
Common mistakes
- Running the screener universe first and reading the verdict without the “Bias: high” note.
- Re-running the holdout after every small change until the verdict improves.
- Switching to the brightest heat-map cell and calling the new result a holdout test.
- Saving only the studies that passed.
- Treating a Passed holdout verdict as a reason to trade at full size straight away (lesson 7).
Summary
In Clear Tape the research loop runs in the Backtest tab: start from a library rule and read how it differs from its source, run the holdout test with conservative settings, read the verdict with its reasons, the disclosures, the training and holdout table and the growth charts, sweep the rule’s key numbers and judge the heat map by its shape, and save every study, whatever its verdict, so that it is re-checked monthly. Expect many studies to end in No edge or Trade-off; that is the process doing its job. Revisit the holdout lesson and the sweep lesson whenever a result seems too good.
For educational purposes only. Not investment advice.