Evaluating an ML trading signal: win rate, profit factor, expectancy
AUC doesn't pay the bills: turning a model's probabilities into trading metrics — number of trades, win rate, profit factor, expectancy and maximum drawdown of the equity curve.
Prerequisites
scikit-learn, pandas, numpy
Python
import pandas as pd
proba = model.predict_proba(X_test)[:, 1] # proba que le trade gagne
seuil = 0.60
rendements = y_ret_test.values # rendement réel du trade
trades = rendements[proba >= seuil]
gains, pertes = trades[trades > 0], trades[trades <= 0]
stats = {
"nb_trades": len(trades),
"win_rate": len(gains) / len(trades),
"gain_moyen": gains.mean(),
"perte_moyenne": pertes.mean(),
"profit_factor": gains.sum() / abs(pertes.sum()),
"expectancy": trades.mean(),
}
equity = (1 + pd.Series(trades)).cumprod()
dd_max = float((equity / equity.cummax() - 1).min())
for k, v in stats.items():
print(f"{k:<15}: {v:.4f}" if isinstance(v, float) else f"{k:<15}: {v}")
print(f"drawdown_max : {dd_max:.2%}")Result
nb_trades : 184 win_rate : 0.6739 gain_moyen : 0.0081 perte_moyenne : -0.0064 profit_factor : 2.6150 expectancy : 0.0034 drawdown_max : -4.87% Au seuil 0.60 : 184 trades, 67 % de réussite, PF 2.6. Contrôle au seuil 0.50 : PF 1.31, drawdown -11.2 % — le filtre par probabilité divise le drawdown par 2 en sacrifiant 40 % des trades. C'est l'expectancy positive nette de frais qui valide le signal.
TradingProfit factorExpectancyBacktest