How we test trading strategies

The short version: realistic costs, fills you could actually get, data the strategy never saw during development, and a script that refuses to publish any number the backtest didn't produce.

Most backtests you find online are built to look good. These rules are built to make a strategy look worse than it will probably be — because the expensive surprise is always the one in the other direction. Every article on this site follows them. When an article breaks one on purpose, it says so in the text.

The data is frozen, and every article names it

Costs are charged on every trade — on purpose, too many

Every backtest pays 0.05% per side: every entry and every exit. That is two to three times what a liquid ETF or futures trade really costs a retail trader. A strategy that only works at zero cost doesn't work; when a result depends on costs, the article shows the break-even.

Fills have to be possible

A buy stop above yesterday's high is filled at that price only if the market trades up through it. If the market opens above it, you get the open — worse. Our backtests fill at the open or the trigger, whichever is worse for you. It sounds like a detail. It killed an article: a narrow-range breakout strategy that looked solid lost half of its out-of-sample Sharpe ratio on SPY once its entries were filled where they really would have been. It was never published.

The same goes for the close: if a signal needs the closing price, the article checks whether you could actually act on it before the bell, and says what it costs if you can't.

Out-of-sample means used once

The most recent part of the history — usually a fifth to a third of it — is locked away while a strategy is built, and used once, at the end. If the strategy fails there, it fails; we don't go back and adjust it. Articles show in-sample and out-of-sample results side by side, so you can see whether the edge held up or faded.

No single best setting

A rule that works at exactly one value and not at its neighbours isn't an edge, it's a coincidence. Parameters are shifted around their value and the article shows the neighbourhood — a plateau is good news, a spike is a warning. When a test explores many variants, the article says how many, because the best of fifty variants is supposed to look good.

Tests are written down before they run

For new research, the hypothesis, the rules and what would count as a failure are written down and saved before the backtest runs. Rules for named strategies come from the original source — the book, the paper, or the author's own definition — and the article cites it. When a strategy needs interpretation (many discretionary concepts do), the article spells out the exact mechanical version that was tested and admits that other traders would read it differently.

Every number is checked by a script

Each article has its own Python script that produces every chart and a file of results. Before anything is published, a checker reads the article and refuses it if it finds:

Negative results are published too. The strategy graveyard exists because most of what gets tested fails, and a site that only shows the survivors is showing you a selection, not research.

How AI is used here

I use AI as a research assistant. It writes much of the backtest code, runs the tests and drafts articles under the rules on this page; the checks above exist so that nothing it writes reaches you unverified. That isn't a footnote to the story — it is the story. For years I traded strategies I had never tested. When AI finally made it quick to code and test them, I found out that most of them didn't work. Testing everything is the habit that came out of that.

What a backtest can't tell you

A backtest is a hypothetical result on past data. It can show that a rule was robust, cheap enough to trade and survivable through its drawdowns. It can't promise the future, and nothing on this site is investment advice. Read the risk disclaimer, and if you trade any of this, start small.

Who's behind this: Robin Eriksson. Found a mistake? Reply to any email from the list — corrections are made in the article and noted there.