Walk-forward Leakage-safe Real market data 20 equities · 2009–2025

The random walk is not beaten.

Five models — naive, moving average, Ridge, XGBoost, LSTM — benchmarked on 79,249 rows of real daily equity data across 6 expanding walk-forward folds. None of them lower next-day RMSE below tomorrow = today. That is the expected, correct result for near-efficient prices, and reporting it is the point.

See the numbers ↓ Read the code

79,249

Rows scored

20 equities, 2009–2025, from yfinance and FRED.

Walk-forward folds

Training data is always strictly before the scored days — enforced by a test.

Models compared

Two honest baselines before any learned model is allowed to claim a win.

51.9%

Best directional accuracy

Above chance — where the predictive value actually shows up.

The headline finding is a negative result. A tuned number that will not reproduce is worth less than a benchmark that says plainly which models fail and why. The linear model matches naive on RMSE while reaching above-chance directional accuracy — that gap is the interesting part, not a leaderboard win.

Walk-forward results (averaged across folds)

Every model, scored on data it never saw during training. RMSE vs naive is the only column that matters: positive means the model beat the random walk.

Model	RMSE	MAE	SMAPE%	Dir. acc	RMSE vs naive
naive	2.410	1.355	1.20	0.0%	+0.00%
linear	2.414	1.355	1.20	51.9%	-0.17%
lstm	2.493	1.404	1.24	48.7%	-3.46%
xgboost	2.693	1.582	1.38	51.3%	-11.75%
moving_avg	3.509	2.047	1.80	50.6%	-45.63%

By volatility regime (RMSE vs naive)

The same comparison split by market regime. Learned models degrade faster than the benchmark when volatility rises.

Model	Low-vol	High-vol
naive	+0.00%	+0.00%
linear	-0.13%	-0.30%
lstm	-4.36%	-6.15%
xgboost	-0.66%	-32.25%
moving_avg	-33.73%	-54.96%

Feature ablation (best learned model, RMSE change vs full set)

Each feature group removed in turn. The deltas are within noise — no group is carrying the model.

Variant	RMSE	Δ vs full
full	2.414	+0.00%
no_macro	2.413	-0.04%
no_volatility	2.412	-0.08%
no_momentum	2.412	-0.06%

Why it is built this way

No leakage. Expanding-window walk-forward; training data is always strictly before the scored days, and a test asserts it.
Honest baselines first. Naive and moving-average are evaluated before any learned model — you cannot claim a win without beating them.
A stationary target. Models forecast next-day return and reconstruct the price level, so RMSE is directly comparable to the random walk.
Diagnostics, not just a leaderboard. Directional accuracy, a volatility regime breakdown, and a feature ablation.
Reproducible. This page is generated from reports/summary.md, which is written by the benchmark itself.

Run it yourself

Two commands. The synthetic mode needs no network.

pip install -r requirements.txt python -m src.forecast.run --synthetic pytest -q

Drop --synthetic to refetch the real panel from yfinance and FRED.