We Ran 4,829 Backtests. Only 4.2% Cleared a Basic Quality Bar.

We keep every backtest anyone runs on TextToQuant. That is 4,829 completed runs carrying real results, covering 4,096 distinct strategies, executed between November 2025 and August 2026.

We went through them. This is what they say, including the parts that are unflattering to us.

Start with what this is not#

Before any number: this is an audit of our own corpus, not a survey of traders. It comes from around 30 accounts. About 92% of the runs are on crypto majors, and BTC alone is 68% of them. About 85% sit on the 4h or 1d timeframe. And every single one is an in-sample evaluation, so nothing here says anything about whether a strategy would have survived out of sample.

It is also survivorship-biased by construction. A backtest that fails writes no row, so the failures are structurally invisible to us. The real denominator is larger than 4,829 and we cannot recover it.

If you were hoping for "here is what works in markets", this is not that, and anyone publishing that from a sample like this would be selling you something. What it can tell you is what happens to a strategy between the moment someone believes in it and the moment a backtest finishes, and that turns out to be worth knowing.

48% were profitable. That number is almost meaningless.#

Of the 4,829 runs that actually took trades, 48.3% finished with a positive return.

Close to a coin flip, which already tells you something. But the number is inflated by runs that should never have counted. 12.2% of them took fewer than five trades. At that sample size a "result" is one lucky entry, and our biggest reported return in the whole corpus is a 5,000% figure produced by a single trade.

This is why TextToQuant puts honesty flags on every run instead of leading with the return. A backtest with four trades and no modelled costs is not a weak result. It is not a result.

Apply a minimum bar and it collapses#

Take only the runs with at least 30 trades and trading costs actually modelled, the floor below which we think a backtest says nothing. That leaves 1,713 runs.

Requiring both a Sharpe above 1 and a profit factor above 1.2, which is not a demanding bar and reads roughly as "this is worth a second look", leaves 72 runs out of 1,713. 4.2%.

And remember every one of those is in-sample, on a strategy chosen by someone who already liked it, tested on the window they chose. The out-of-sample number would be lower. We are showing you the flattering version.

The finding that survived: costs flip winners into losers#

Everything above is descriptive. It cannot tell you why, because the runs differ in a hundred ways at once.

So we looked for the runs where the same strategy, meaning identical entry conditions, identical exits, same asset, same timeframe and same date window, had been executed both with trading costs switched off and with costs switched on. That holds everything constant except the cost model.

There are 52 such pairs. The sample is small, and we would rather report it small than not report it:

• 41 of 52 were profitable with costs switched off
• 30 of 52 were profitable with costs switched on
• 11 of 52, better than one in five, flipped from a winner to a loser, purely because costs were modelled
• Median return fell by 10.2 percentage points

That is the same strategy, the same market, the same window. The only thing that changed was whether the backtest was allowed to pretend trading is free.

If you take one thing from this: a backtest with fees = 0 is not a slightly optimistic backtest. On this evidence it is wrong about the sign of the result about a fifth of the time.

What we checked and could not confirm#

We ran the same paired test on stop losses, holding the entry conditions, market and window fixed, with and without a stop. 64 pairs: 19 profitable without a stop, 22 with one.

That is noise at this sample size. We are not going to tell you stops improved returns, because our data does not show it. We are including this because a study that only reports the tests that worked is the exact failure mode this whole article is about.

Why we published this#

We sell a backtesting tool. The commercially convenient article says most strategies fail elsewhere and succeed here. The honest one says that on our own platform, on our own users' ideas, under our own most generous assumptions, about 4% of runs cleared a modest quality bar, and that a setting most tools default to hiding changes the answer for one strategy in five.

That is not an argument against backtesting. It is an argument for backtesting that tells you when it has nothing to say. Every run on TextToQuant reports its grade, its trade count and its honesty flags alongside the return, and refuses to annualise a figure the sample cannot support, precisely because of the numbers on this page.

Reproducing this#

Every figure here comes from one script over a read-only snapshot of the run table, and it re-runs nothing: reports/census.py in our strategy-miner directory, which emits the JSON that this article quotes. The caveats in the first section are encoded in its docstring so they travel with the numbers.

If you want to check any specific claim, or you want the underlying distribution for research, write to us and we will send it.