// How every idea is tested
Methods
Every idea goes through the same steps. The goal is to kill bad ideas cheaply and to avoid fooling ourselves with good-looking backtests.
1. Write the test down first
Before any code runs, the rules, the parameters and the data split are written down and frozen. Changing them after seeing results starts a new test with a new id.
2. Real costs from the first line
Fees and slippage go in before the first result is read. On Binance perps that is 0.13% per round trip as a base. Ideas that only work at zero cost are recorded as such.
3. Enough history
A backtest needs at least two years and has to include a bull market, a bear market and a sideways stretch. A strategy that only works in one of them is a bet on that regime.
4. In-sample and out-of-sample
The first 80% of the period is in-sample, the last 20% is out-of-sample with at least 20 trades. The split is fixed before any configuration is picked. A result has to pass on the full period and on both parts.
5. The pass bar
| Net return | above zero |
|---|---|
| Max drawdown | smaller than 25% |
| Profit factor | above 1.3 |
| Trades | at least 50 |
| Sharpe ratio | at least 0.8 |
| Free parameters | 4 to 5 at most |
Some tests add stricter checks: the same result at 1.5 times the costs, an untouched holdout, a second dataset, or removing the best few days.
6. Random twins
Newer tests compare each trade to random “twin” entries: the same instrument, the same year, a date with no event. If the strategy does not beat its random twins, the signal adds nothing over just being in the market.
7. Bury it with a reason
A failed idea gets a cause of death and a reopen condition: the specific evidence that would justify another test. That is what the Graveyard lists.
Older entries predate some of these steps. Each written-up page says which checks its test actually had.