The short version: of 150+ strategies put through an identical pre-registered test bar, the large majority failed on unseen data — including 17 of 17 classic book setups, 44 of 44 trend-following variants and every popular published backtest we re-tested. The survivors fit in one short list: eight families, and they're almost never chart patterns — they're structural and calendar flows, or regime-filtered versions of simple ideas. The single biggest killer, by a distance: in-sample survivorship bias — the famous curve was never re-tested on data it hadn't seen. The full body count is below.
The bar every strategy had to clear
None of this means anything unless the test is identical and decided before the results come in. Ours is, and it is deliberately strict — it's the bar we use before risking real money:
- In-sample Sharpe > 0.3 and out-of-sample Sharpe > 0.5, standalone
- Weekly correlation below 0.3 against everything already in the book
- Measurable portfolio uplift (> 0.05 OOS Sharpe at normal weight)
- A parameter plateau — neighboring settings must work too, or it's curve fitting
- Walk-forward: mean Sharpe > 0.3 across folds, fewer than half negative
- Bootstrap: at least 70% probability the uplift is real, not luck
In-sample window 2010–2018, out-of-sample 2019–2026, volatility-targeted, tier-based real transaction costs. Identical for a 1995 book pattern and for our own best idea. That last part matters: most of the corpses in this graveyard are ours — ideas we hoped would work and buried when they didn't. Here's what that bar did to the famous names.
The graveyard: famous setups, honest numbers
Classic book patterns — 17 tested, 0 passed. Setups from Raschke & Connors' Street Smarts (1995), DeMark (1994), Kaufman (1995) and Chande (1997). Thirty years of photocopied chart patterns, and not one cleared a bar you'd risk money on. Turtle Soup — the famous failed-breakout fade — failed on every market we tried (ES, NQ, GC, TLT), with walk-forward means between −0.04 and −0.40. Thirty years is a long time for the market to read the same book.
The in-sample mirage, exhibit A: Connors TPS on the Nasdaq. Strongest raw performer of its whole batch — Sharpe 0.96, eight positive years out of eight. Out-of-sample: 0.006. Not negative — just nothing. A decade of beautiful backtest compressed into a coin flip the moment the data was unseen. This single result is why we never publish an in-sample number without its OOS twin.
Seasonal lore. The Santa Claus rally on SPY: in-sample +0.31, out-of-sample −0.07 — arbitraged after 2018, presumably by everyone who read the same December articles. Halloween and window-dressing effects, same story.
Trend following in small doses — 44 variants, 0 passed. Clenow-style core trend on an 8-instrument panel: 0 of 24 configurations. Carver-style acceleration: 0 of 20. The honest finding isn't that trend following is dead — it's that single-instrument trend runs a Sharpe of 0.2–0.5 and needs 30–100 markets to become an account, which no retail-scale panel delivers.
Published strategy catalogs. Four popular published setups — DAX overnight drift, EuroStoxx loss-reversal, natural-gas short-summer, soybean seasonals — all published with pre-2019 backtests. All four failed out-of-sample. The pattern generalizes and it has a name: deflated Sharpe. Which brings us to what unseen data actually does to a backtest:
The survivors share one trait
Here's the full list of families that cleared the bar, with their out-of-sample Sharpes: Turnaround Tuesday (1.30) · IBS mean reversion (1.04) · Zarattini intraday momentum, long-only (1.18) · Friday gold close (1.69 net of costs) · a seasonal sleeve of gold-January plus pre-NFP drift (1.04) · month-end bond seasonality in 10-year futures (0.66) · regime-filtered metals skewness (0.98) · and the RSI(2) principle migrated to 2-hour/4-hour bars (1.18–1.25).
Read that list again and notice what's not on it: not one classic chart pattern. Every survivor is either a structural or calendar flow — somebody has to trade at the weekly gold close, at month-end bond rebalancing, before scheduled Fed meetings, regardless of what the chart looks like — or a simple idea wearing a regime filter. That's the trait. Flows recur because they're driven by obligations, not opinions. Chart patterns die because everyone can see them, and what everyone can see gets sold to everyone. If you want the arithmetic of what a few genuinely different survivors do together, that's the portfolio math — but first, the practical part.
Three questions that kill most backtests in five minutes
- "Was any of this data unseen?" If the article can't point to a held-out window, assume the curve is fitted. We built the perfect backtest once — 2,160 parameter combinations — and watched it die on new data.
- "What did costs do?" Every number in our log charges real transaction costs; most published curves charge zero. Short-hold strategies flip sign on this alone.
- "How many variants were tried?" One reported result from hundreds of attempts is multiple-testing bias, and it's the default in strategy publishing. A plateau of neighboring settings that all work is the antidote — a lone magic parameter is a confession. And even a real edge only shows you one path of many: its drawdown distribution is wider than its history.
Lab notes
The graveyard nearly claimed us too. In April a metals gap-up strategy tested at Sharpe 1.80 with an 81% win rate — good enough that it was one review away from deployment. The bug: the entry condition accidentally referenced the same day's close, information you can't have at the open. Fixed, the Sharpe fell to 0.72 and a sister variant went negative. The whole version was rolled back and the incident is logged next to lesson #38 and friends in the master log. Rule since then: any result that looks too clean gets hunted for lookahead before it gets admired.
One honest caveat, because this article is not a takedown: we're not claiming these strategies never worked. Several genuinely earned money in their era, and some still work in narrow regimes. The claim is narrower and more useful — re-tested on unseen data with realistic costs, most don't clear a bar you'd risk real money on. The graveyard isn't proof that trading is impossible. It's the reason the small list of survivors is worth anything at all.
Download a free strategy
Want the rules to one that survived? Simple and robust — 78% winners over the past three years, net of costs. The free PDF gives you the market, the timeframe and the exact rules in plain English, plus every single trade it has taken.
Instant delivery. One PDF, no spam. Unsubscribe anytime.
FAQ
Does Connors RSI(2) still work?
On daily bars, mostly arbitraged: the TPS version showed 0.96 in-sample and 0.006 out-of-sample. But the principle migrated timeframes — the same logic on 2h/4h Nasdaq bars survived at 1.18–1.25 OOS. Edges don't always die; sometimes they move.
Is the opening range breakout dead?
The naive version failed to beat a random-direction baseline. A 15-minute-formation ORB on regular-trading-hours data survived at 0.79–0.91 OOS. One implementation detail separates the corpse from the descendant.
What's the biggest reason published backtests fail re-testing?
In-sample survivorship: the curve was fitted or selected on the data it's shown on. Add zero costs and multiple-testing bias, and a book strategy can look spectacular for decades while earning nothing forward.
What do the survivors have in common?
They're structural or calendar flows (weekly gold close, month-end bond rebalancing, pre-FOMC drift) or regime-filtered simple ideas — almost never classic chart patterns. Flows recur because someone must transact regardless of price.
Robin Eriksson
Founder of EdgeLab. Five years of discretionary losses taught me to test everything — now I publish the strategies that survive. About me →
Related: Why your backtest passes — and your live account doesn't · Your worst drawdown is still ahead of you: 10,000 Monte Carlo reshuffles · Toby Crabel's NR7 breakout, tested on stocks, gold and bitcoin — 1,850 trades
