Removing one coin from the backtest flipped the answer
I asked an agent to find a paper trading setup with fewer bots. Its winner looked great until a test that deleted SOL from the universe showed one sleeve earning 96% of its profit from that one coin.
I run a few Freqtrade bots in dry-run mode on the home cluster. They trade BTC, ETH, SOL and BNB futures on paper, 2x leverage, each bot with a $100 pretend wallet. Four bots, four strategies, one ensemble. It looked fine in backtests, but four bots is a lot of bots.
So on June 14 I wrote down what I wanted and let an agent chew on it overnight. More trades per month, the same or better profit, the same or lower drawdown, and fewer bots. Ideally one or two, three at most. The commits run from 23:07 to 06:38, and the agent's own notes call it "a multi-hour autonomous search".
The first answer
The search produced one new piece, a Donchian channel breakout on the 4h chart with the same BTC-above-EMA200 gate my other trend bots use. The obvious version, with default channel periods, chased noise and drew down 29.7%. Scaling the periods by about 1.5x, so the 4h bot looked back over the same calendar span as the proven 8h one, fixed that.
With that in hand it proposed Book A. Three bots: the new 4h Donchian, the existing daily Donchian, and a Bollinger squeeze bot on 8h. On the 2023 to 2026 backtest it did 40.4% a year with a 4.9% max drawdown. That beat my four bot setup on paper, with one bot fewer. Done, apparently.
Except the same run had already found a problem with the scoreboard. An adversarial audit pass showed the repo's way of combining bots into one equity curve inflated annual return by 5 to 6 points and understated drawdown, because it booked profit only when trades closed and never marked open positions to market. So a new evaluator got written, every number was recomputed, and Book A still came out ahead. That should have been the end.
Delete SOL and look again
The second audit asked a question I should have asked weeks ago. Where does the profit actually come from?
Per coin, the answer was mostly SOL, across every bot. The daily Donchian made 62% of its profit on SOL. The 4h one, 41%. The Bollinger squeeze bot made about 96% of its profit on SOL. Almost a single-coin strategy wearing a four-coin config.
The test that followed is simple and I recommend it to anyone backtesting anything with a small universe. Take the biggest contributor out and rerun. The evaluator got a guard so it would only pick up backtest results from the three-coin universe:
WANT_SOL = os.environ.get("WANT_SOL", "1") == "1"
...
has_sol = any("SOL" in p for p in pairs)
if WANT_SOL and pairs and not has_sol: continue
if (not WANT_SOL) and has_sol: continue
Here's what fell out, annual return over max drawdown:
| Book | With SOL | Without SOL |
|---|---|---|
| Old 4 bots | 39.0 / 4.8 | 20.4 / 6.5 |
| A: 4h + 1d + Bollinger squeeze | 40.4 / 4.9 | 14.7 / 13.3 |
| B: three Donchian bots | 45.4 | 5.1 / 19.9 |
| C: 4h + 1d + Supertrend | 42.0 / 4.1 | 23.3 / 8.1 |
Book A without SOL was worse than the thing it was supposed to replace. Book B, the headline winner at 45.4%, nearly vanished. The two bot option had its own problem: the daily bot carried 60% of that book's profit, and 78% of the daily bot's profit came from three trades. Three.
Book C swapped the Bollinger squeeze for Supertrend. It kept 23.3% a year without SOL, had the lowest drawdown, and was positive in 92% of rolling 180-day windows, against 87% for A and 77% for my old setup. At 02:33 the recommendation changed from A to C.
Then the bear market voted
Eight minutes later it changed again.
Everything so far covered 2023 to 2026, which is a kind period for trend following. The agent backfilled 4h and 8h data to October 2020 and ran the books through 2021 and 2022. None of that period had been used to pick anything, so it was the most honest out-of-sample test available. In 2022, BTC fell about 65%.
| Book | 2021 to 2026 return / DD | 2022 |
|---|---|---|
| Old 4 bots | 23.5 / 24.0 | -6% |
| A | 25.3 / 17.4 | -2% |
| C | 24.6 / 26.2 | -8% |
The Bollinger squeeze only fires when volatility expands, and that made A the best book through the bear. C, the one that survived losing SOL, was the worst of the three. The two stress tests pointed in opposite directions, and the notes say so plainly: pick A if you weight surviving the next bear, C if you weight not depending on SOL. The lean went back to A.
The bigger lesson from that table isn't A versus C at all. The real full-cycle drawdown was 17% to 26%, not the 4% to 5% the recent window showed. Every number I'd been looking at for weeks came from a period with no bull-to-bear transition in it.
The rest of the night
A few more checks landed before morning, and one of them caught the test harness lying. After the backfill, the evaluator's "pick the newest matching result" logic started grabbing the extended 2021 runs instead of the standard ones, and trades per month jumped from 8.9 to 13.9. Nobody's strategy had improved. It got a second guard keyed on the first trade date.
Then a Monte Carlo pass, 4,000 block-bootstrap paths over the full-cycle returns, put Book A's 95th-percentile drawdown at 18.7% and the 99th at 24.4%. A 31-pair walk-forward test checked whether adding altcoins would dilute the SOL problem. It made things worse: the majors alone went +6.0% out of sample with a 2.6% drawdown, the version with all the alts went -2.8% with 8.3%. The whitelist stayed at four coins.
At 13:35 Book A went live on the homelab, still in dry-run, with C written up in the notes as the alternative. I don't count that as a win for A. I picked which risk I'd rather hold, and I know exactly how exposed to SOL it is, which I hadn't bothered to check before.
None of this is financial advice. It's a hobby with a Docker Compose file and a lot of backtests, and the only money involved is pretend.
A footnote from September
On September 2 the dry-run wallet went from $100 to $250 across every bot. The commit message says the change was "to allow BTC entries". I had spent a whole night stress-testing what happens when the book loses one of its four coins, and a wallet setting had been quietly keeping the paper bots out of BTC.