Backtest Pitfalls in China A-Share Research: Six Failure Modes, Ranked by How Hard They Are to Catch

A smooth equity curve is not evidence of anything until you know which mistakes the backtest is free of. Three of the six failure modes below are well documented and turn up in ordinary code review. The other three — benchmark choice, cost and fill conventions, and beta leakage — pass code review, pass a train/test split, and in one of our own runs passed a deflated Sharpe ratio penalty as well. This piece ranks the six by how hard they are to catch and spends most of its length on the last three. The numbers come from research runs on China A-share data, not from textbook examples.

First, the market rules a backtest has to respect

China A-share microstructure differs from US equities in ways that quietly invalidate imported backtest code. If you are reading this from outside the market, the table below is the minimum vocabulary. Each row exists because it has broken a backtest we ran.

The six pitfalls, ordered by detection difficulty

Pitfalls one through three: the ones review catches

Lookahead means the signal consumes information that did not exist at decision time. The A-share versions are specific: deciding at 14:35 using the day's closing price, or backfilling with adjustment factors that were only published after a later dividend. A useful test is to replay the pipeline bar by bar with future rows physically deleted rather than masked. Masked rows leak through joins.

Survivorship bias comes from universe files built out of currently listed tickers. Delisted and ST names disappear, and with them most of the losing tails. Check the universe count per month; a series that only grows is a symptom.

Overfitting is more common than it looks because published numbers are usually champion numbers. We re-ran one of our own live models. Its nominal hit rate of 61.6% was measured on a chosen parameter set and a chosen sample; on full-market forward-adjusted data across roughly 2.5 years it produced 56.6%. Be careful with the attribution: those five points are the total distance between "champion parameters on one sample" and "full market, forward-adjusted, 2.5 years". Four things changed at once, and we never ran the experiment that isolates the parameter search on its own — that would mean holding the sample fixed and varying only the parameters, then the reverse. Treat five points as an upper bound on the search effect, not a measurement of it.

Four: the benchmark decides whether the strategy exists

That same model, after real fees and the take-profit and stop-loss levels actually in use were applied, had negative expectancy. Then the benchmark changed. Measured against CSI 500 and CSI 1000 — indices whose market-cap profile actually matches what the rule was picking — out-of-sample excess return went to roughly −0.3% to −0.5%. The reading is unambiguous: the positive raw return was small-cap market beta, and the benchmark had been hiding it. If a strategy tilts toward smaller names, comparing it to a large-cap index is not a comparison, it is a compliment you paid yourself.

One boundary we owe the reader, since this article argues against forward-adjusted data: that reproduction was itself run on forward-adjusted prices, so by our own standard the exact figures are not strictly reproducible. We keep them because the conclusions — a gap of about five points, negative expectancy after costs, excess turning negative against a size-matched benchmark — hold at the order-of-magnitude level regardless of adjustment method. The decimals should not be re-quoted.

Five: the measurement convention nobody wrote down

Take one batch of signals from 2026-07-30, where every name gapped down at the next open. Measured next-open to next-close, the batch averaged +0.56%. Measured from the signal-day close to the average of the next day's high and close, it averaged −0.28%. Same signals, same price data, 0.84 percentage points apart, and the sign flips. The first convention discards the overnight gap entirely, which is exactly where the loss lived. Neither number is dishonest on its own. Reporting one without naming the convention is.

Fill assumptions belong in the same category. Under T+1, an exit rule that sells on the signal day is not conservative, it is impossible. On a one-word board there is no tradable range, so any assumed fill is fabricated. Commission, stamp duty on the sell side, and transfer fees are small individually and decisive at high turnover.

Six: beta leakage, or trading the index in disguise

The hardest case we have run. The target was an intraday signal on a single large-cap name, two years of 5-minute bars, tested in two formulations: mean-reversion and momentum. The setup included a deflated Sharpe ratio penalty for the number of configurations tried and an adversarial validation check for train/test drift. Mean-reversion failed outright. Momentum passed every gate we had built. Then we correlated the signal series with the index over the same window and got +0.97. It was not predicting the stock; it was restating the market trend in a more expensive form. Removing 45 event-driven trading days still left it losing, and buy-and-hold over the identical window was positive.

What the deflated Sharpe ratio does and does not tell you

DSR deflates an observed Sharpe by the number of trials, plus sample length, skew, and kurtosis. It answers one question well: did I find this by searching too hard? It does not answer whether the surviving signal is a market factor wearing a costume. Those are different failure modes and need different tests. Pair DSR with a correlation check against the relevant index and, better, with a regression of strategy returns on benchmark returns — then ask whether the residual alone still pays for costs.

The check we ran too late

The correlation test that ended the intraday project is one line of code. We ran it after the DSR and adversarial validation because those were already scripted and felt more rigorous. That ordering cost weeks. Cheap falsification tests belong first, before anything expensive, regardless of how sophisticated the expensive ones look.

A second constraint worth stating: forward-adjusted history is recomputed from the present backwards, so every dividend or split rewrites the entire series. The same window pulled six months apart returns different numbers, and old names with many stock dividends can compute to negative prices far enough back. Backtests on forward-adjusted data are not reproducible in the strict sense. For research runs meant to be rerun, backward-adjusted or raw-plus-factor storage is the safer choice.

A backtest is a measurement. Every measurement has units, and most reported hit rates ship without them.

Eight checks before trusting a backtest

1. 1State the entry price, the exit price, and the adjustment method in one sentence. If you cannot, the number has no units.
2. 2Rerun with future rows deleted rather than masked, and confirm the result changes as expected.
3. 3Count universe members per month and confirm delisted and ST names are present.
4. 4Report the parameter set's neighbors, not only the winner. A peak with a cliff on both sides is a fitting artifact.
5. 5Regress strategy returns on the benchmark most similar in market-cap profile, and look at the residual.
6. 6Correlate the signal series with that benchmark. There is no clean threshold — the higher it runs, the less is left to attribute to the rule. Ours came back at +0.97, which is not a borderline case. The decisive test is the regression above: whether the residual survives costs.
7. 7Apply real commission, stamp duty, and slippage, then re-decide whether the edge survives.
8. 8Delete fills that occur at a price limit or on a no-range session, and see what is left.

Boundaries

Every figure above comes from our own runs on specific windows and specific universes; none of it generalizes to other strategies or other periods, and none of it is a claim that any particular approach works or fails in general. The checks are diagnostic, not sufficient — a strategy can clear all eight and still be wrong for reasons this list does not cover, including regime change after the test window ends. Nothing here is investment advice.