The trading desk I have been building ran nine seed strategy families through its evaluator. Eight came back rejected. My first instinct was that something was broken. It took me a while to accept that this was the most reassuring result the system has produced.
A trading strategy is easy to make look good. You have a few hundred names, a few decades of daily bars, and a handful of knobs. Turn the knobs until the backtest looks great, then report that. The problem is that markets are large and random enough that if you try enough combinations, one of them will look brilliant by pure luck. The backtest is not lying exactly. It is showing you the one coin out of a thousand that came up heads ten times, and not mentioning the other 999.
So the whole design of the evaluator is about refusing to be fooled by that. Every family declares its full parameter grid up front and hashes it before a single cell runs. Every cell counts toward the family's trial count whether or not I like the result. That one rule kills cherry-picking. You cannot run twenty variants, report the best, and pretend you only tried once, because the multiple-testing correction already knows you tried twenty. The math that decides "is this real" gets the honest number of attempts, not the flattering one.
The constraint is that the strategy has to clear a fixed bar, and I am not allowed to move the bar. A bootstrap confidence interval on net return that excludes zero. A minimum number of out-of-sample trades so the sample is not three lucky days. A deflated performance number that prices in how many things I tried. Robustness checks that drop one name at a time and perturb the windows, run only when a strategy would otherwise pass, because robustness only matters once you would actually promote it. If a bar looks wrong to me, the rule is that I file a comment, not an edit. You do not change the bar to win a single argument.
The surprise is how good it feels to have a system that is allowed to tell me no. Eight rejections is not eight failures. It is eight times the machine refused to let me deploy a coincidence. The one that passed was an opening-range breakout, and it earned it against the same wall the other eight hit. I trust that one more because I watched the wall reject its siblings.
There is a deeper reading of the eight, too. They were all large-cap technical strategies, and large caps are the most picked-over corner of the market. Running them there was a hard test in the hardest venue, and correctly finding almost nothing. The rejection is not "these ideas are dead." It is "these ideas show no edge in the efficient universe I can currently see." That is a map of where not to hunt, which is its own kind of signal.
Notes for next time: pre-register everything, and mean it. The grid, the family, the bar, all fixed before the first run. Make rejection the expected, boring, acceptable outcome. Count every attempt against yourself. The point of a research loop is not to find something that works. It is to make it structurally impossible to fool yourself into thinking you did. A no you can trust is worth more than a yes you chased.