LibraryAI in quantitative finance
Short answer. We trained one machine-learning model on US stocks, the NIFTY 50, EUR/USD and Bitcoin, and tested it five ways. Tested honestly, it called the next week's direction 52.7% of the time on average, about a coin flip, with a Sharpe ratio of 0.16. Three mistakes each made it look like a winner:
A backtest pretends to trade in the past. Look-ahead bias is any way the pretend trader sees something it could not have known on that day. Leakage is the machine-learning version: information about the test answers slips into the training data or the inputs. Both make a model look skilled when it is not, and both are silent: nothing crashes.
López de Prado lists cross-validation leakage among the ten reasons most machine-learning funds fail[1]. Every leak below took one line of code.
| Market | A honest | B shuffled | C centred | Buy-and-hold Sharpe |
|---|---|---|---|---|
| US stocks (SPY) | 55.3% / 0.33 | 60.8% / 0.89 | 81.8% / 4.93 | 0.54 |
| India (NIFTY 50) | 53.6% / 0.09 | 61.0% / 0.97 | 83.2% / 5.58 | 0.55 |
| Forex (EUR/USD) | 50.4% / -0.16 | 59.0% / 0.68 | 82.8% / 4.67 | -0.12 |
| Crypto (Bitcoin) | 51.7% / 0.38 | 59.2% / 1.19 | 81.3% / 6.48 | 0.67 |
Train only on the past, test on the next year, then move forward a year and repeat. We also drop the last 5 days before each test year from training, because their 5-day answers reach into the test year. This drop is called purging.
for year in test_years:
start = first_day_of(year)
train = rows_before(start)[:-HORIZON] # purge labels that reach into the test year
model.fit(X[train], y[train])
predictions[year] = model.predict(X[year])Result: accuracy between 50.4% and 55.3%, and a Sharpe ratio from -0.16 to 0.38. It is the number to believe, because it was scored the way you would trade it.
Standard machine-learning code splits rows at random. In a time series that is a leak. Our target covers the next 5 days, so rows a day apart share 4 of the same 5 days. With shuffled folds, a test row's near twin sits in the training set, and the forest remembers it.
# Leaks: rows next to each other in time land on both sides
KFold(5, shuffle=True).split(X)
# Fix: train on the past only, and purge overlapping labels (as in A)Result: accuracy 60.0% against 52.7%, and a Sharpe ratio of 0.93 against 0.16. Bailey and co-authors show more generally that a hold-out sample used many times stops protecting you[2].
A centred average, rolling(21, center=True) in pandas, puts today in the middle of the window, so it uses the next 10 days. It is a normal choice for smoothing a chart. It is a leak in a model.
# Leaks: the window runs 10 days past today
gap = close / close.rolling(21, center=True).mean() - 1
# Fix: a trailing window ends today
gap = close / close.rolling(21).mean() - 1Result: accuracy 82.3% and a Sharpe ratio of 5.41, against 52.7% and 0.16 honest. One input out of eight did that. The same thing happens with any statistic computed over the whole sample, and with data stamped by the period it covers and not the day it was published.
Take a plain rule: long when the price is above its 20-day average, short when below. The signal uses today's close. If the backtest credits that signal with today's return, it trades on a price it only knew at the close.
signal = np.sign(close - close.rolling(20).mean())
ret = np.log(close).diff()
pnl_wrong = signal * ret # leaks: earns the move that made the signal
pnl_right = signal.shift(1) * ret # honest: decided at yesterday's close| Market | Next day (honest) | Same day (leak) |
|---|---|---|
| US stocks (SPY) | -0.12 | 3.80 |
| India (NIFTY 50) | 0.17 | 4.42 |
| Forex (EUR/USD) | -0.36 | 3.83 |
| Crypto (Bitcoin) | 0.71 | 4.98 |
The honest rule's Sharpe ratio averages 0.10. The leak lifts it to 4.26. This is the bug to look for first in any backtest written by someone else, including a chatbot.
Fitting a scaler on all the data before splitting lets the test period's mean and spread into training. It is a real leak, so fit the scaler on training rows only. In our test, with a logistic regression, it barely moved the result. We show it so you can see the size: not every leak is large, and the dangerous ones are those that copy the answer.
| Market | Scaler on training rows | Scaler on all rows |
|---|---|---|
| US stocks (SPY) | 59.5% / 0.77 | 59.5% / 0.80 |
| India (NIFTY 50) | 55.6% / 0.30 | 55.5% / 0.30 |
| Forex (EUR/USD) | 47.6% / -0.18 | 47.6% / -0.20 |
| Crypto (Bitcoin) | 52.3% / 0.15 | 52.4% / 0.18 |
Many tutorials predict tomorrow's price and show a chart where the forecast hugs the real line. That proves nothing. A forecast of "tomorrow equals today" scores almost perfectly on price levels, and below zero on returns, which are what you trade.
| Market | R² on prices | R² on returns | Direction right |
|---|---|---|---|
| US stocks (SPY) | 0.9997 | -1.21 | 49.3% |
| India (NIFTY 50) | 0.9997 | -0.93 | 53.0% |
| Forex (EUR/USD) | 0.9950 | -1.36 | 48.2% |
| Crypto (Bitcoin) | 0.9988 | -1.05 | 48.0% |
R² is the share of the variation that a forecast explains, 1 being perfect. Measure a model on returns, or on direction after costs, and compare it with this do-nothing forecast. Gu, Kelly and Xiu, using machine learning on US stocks, report monthly stock-level R² of only 0.33% to 0.40% for their best models, which still had real economic value[4]. Honest numbers in finance are small.
Cut the data at a date, recompute, and compare. If the signal on that date changes when later prices are removed, the code uses the future. We ran this test on all 24 strategies a chatbot wrote for us (results).
def uses_the_future(positions, df, checks=18):
full = positions(df)
n = len(df)
for k in range(3, checks + 3):
cut = n * k // (checks + 3)
part = positions(df.iloc[:cut + 1])
a, b = part.iloc[-1], full.iloc[cut]
if not (np.isnan(a) and np.isnan(b)) and not abs(a - b) <= 1e-9:
return True # this date's signal changed when the future was removed
return FalseIt catches inputs that see the future, such as a centred window. It does not catch same-bar trading, which lives in the backtest, not the signal, so check the shift there by eye. And it cannot catch a leak in the data itself, such as a figure revised after the fact.
The free backtest check asks these questions and computes the deflated Sharpe ratio for you.
Thuztra Learn teaches the maths of quant finance from zero, in short lessons, for readers who are unsure of it. Three questions in the app build your quant roadmap, free. Membership is a paid subscription. See what Thuztra Learn is and what it costs, or see the whole Core quant curriculum.
Get the app and answer the three questions:
Download on the Microsoft Store Download for Android