← Back to the Ledger
Machine learningKilled

The exit-learning loop, run at maximum effort until every survivor confessed

27 agents, 0 survivors
two pre-locked specifications, 239 ledgered hypotheses, and every candidate that beat random was reading the future
In plain English

Our bot was bad at deciding when to leave a trade, so we asked whether a machine could learn better exits from seven months of its own history, and we ran the question the way a lab would: rules written down first, a final exam sealed in advance, and independent checkers ordered to rerun everything. Three learned strategies looked genuinely skilled. The checkers then showed all three were accidentally being told when each trade had ended, information nothing can have in real time, like acing a quiz with the answer sheet stapled to the back. Strip that away and the skill vanishes below a coin flip; the honest version was worse than the bad exits it was meant to fix. The exam was never opened. The only exit improvement that survived all of it: take your profit at a fixed nine percent, no learning required.

What was claimed
An AI trained on months of your own trades can learn when to take profit.
The bar it had to clear
recover at least 17.3% of the perfect-exit gap on out-of-sample trades (the break-even line), and beat what a random exit scores on the same trades
What the data said
4.6% on the first locked test; on the second, all three finalists were refuted by rerun ablations and the sealed test window was never opened
Why it's dead

Seven months of farmer trades leaked profit at the exit, so we asked, twice and under locked rules, whether the exit could be learned. The first attempt trained on 686 real trades and captured 4.6% of the oracle gap out of sample against a pre-committed 17.3% break-even bar. The operator then ordered a maximum-effort search: six independent design families, walk-forward validation, a physically locked test window, every hypothesis logged to a ledger, and adversarial auditors instructed to rerun code rather than argue with it. Three candidates beat the random bar in validation. The audits then established that all three were the same defect in different clothes: each policy had access to when the trade's window actually ended, which is future information. Remove it and the best model collapses below random; make the simplest variant implementable and it scores worse than the exit logic it was meant to replace. Our own evaluation harness had also flattered everything, twice, including the random benchmark itself, and the write-up retracts the seductive headline it briefly produced.

The detail

Attempt one: reconstruction validated against the trade tape at r=0.999, gradient model on 80,545 bars, train capture 5.8%, test 4.6% vs the 17.3% bar; a fixed take-profit baseline scored 0.3% on the same test trades. Attempt two: 6 design lenses (microstructure at 200ms book resolution, volatility/funding regime, survival hazard, parametric ratchets, gradient-boosted meta-labeling, cross-asset), 239 validation evaluations on the ledger, top three by dollar-weighted validation capture 0.289 / 0.258 / 0.215 against a random-exit bar of 0.202. Nine adversarial audits: the meta-label model's dominant feature was fraction-of-window-elapsed (permutation importance 0.066 vs under 0.005 for every market feature); zeroing that single column and rerunning the full cross-validation collapsed it to 0.056-0.075, below the random median. The fractional-window rules consumed the realized window length directly; the implementable ablation (fraction of the training-median hold) scored −0.053, worse than the incumbent. The audits also corrected the harness itself: the matched null on the pooled validation trades is p50 0.178 / p95 0.237, and best-of-16 selection should face a max-of-16 null (median 0.238, p95 0.279), under which every headline in the search sits at or below the random median. Verdict under the pre-locked rule: champion set empty, test window never unlocked, question closed. The one implementable exit improvement ever measured on this tape remains a dumb fixed take-profit worth about +$48 over five months.

Kill date
2026-09-01
Sample
686 real trades, 105,097 minute-bars, 6 model families, 239 ledgered validation evaluations, 9 audits
Method
Documented kill
Verdict
a null that knows the future flatters everyone

Tested on the record and published in full, with the real numbers, whatever the result.

See all kills