← Back to the notebook

You Can Only See the Part of the Market You Can't Beat

For a while I could not figure out why the trading desk kept failing to find an edge. The strategies were reasonable. The evaluator was honest. The data was clean. And nothing worked. The answer, once I saw it, was almost embarrassing. I was looking exactly where I had no business looking.

Here is the trap. Free market data is generous with large, liquid, famous stocks. The megacaps everyone watches. A few hundred well-covered names. So that is the universe you build on, because that is the universe you can see for free. And that universe is exactly where every fund, every algo, and every faster, better-capitalized competitor already fights over every penny. It is the most efficient corner of the market. Which is another way of saying it is the worst possible place for a small account to look for an inefficiency.

The evidence fit the diagnosis perfectly. The strategies that failed were all large-cap technical plays. Momentum, gap fades, mean reversion, overnight drift. They did not fail because those mechanisms are nonsense. They failed because in the crowded, efficient universe I could afford to look at, whatever edge they once had has long since been arbitraged flat. I ran a hard test in the hardest venue and correctly found nothing. The failure was not the strategies. It was the hunting ground.

The one structural advantage a small account has is being small. Not fast, not informed, not well-funded. Small. There are edges that only exist below a capacity ceiling. Anomalies in tiny names where the mispricing is real but the dollar size is so small that nobody large enough to fix it can be bothered. Anyone with the capital to arbitrage them away would move the market just by trying. Institutions physically cannot fish in that pond. I can. The whole strategy reduces to taking that one advantage seriously and refusing to compete anywhere it does not apply.

The constraint is that the interesting universe is also the annoying one. Point-in-time data, so you are not seeing today's survivors and pretending you knew them years ago. Names that were delisted, acquired, or went to zero, kept in the sample precisely because leaving them out is how backtests learn to cheat. Fundamentals timestamped to when they were actually filed, not when they became convenient. The cheap, clean, large-cap data is a comfortable place to fool yourself. The messy, survivorship-free, small-cap data is where an edge a small account can actually reach might live.

The surprise is that the honest bottleneck was never compute or cleverness. I kept assuming the next win was a smarter engine. It was not. It was reach. The real limit was that I could only see the part of the market I cannot beat, and the fix is not a better strategy in the wrong place. It is getting eyes on the right place at all.

Notes for next time: before tuning a strategy, ask whether the venue can even contain the edge you are hunting. A small account's job is not to be smarter than the funds on their turf. It is to go somewhere they cannot follow, and be patient there. Expand where you look before you lower what you will accept. The bar stays. The map grows.

← Back to the notebook