Developer/operator lane only. Regular users should run onboarding, select profile 1-5, pull approved model revisions from R2, and run trading.
Training priority
1. Local GPU — auto-detected (CUDA / AMD ROCm)
2. Local CPU — fallback (Xeon Platinum 8272CL / 4-core / 32 GB takes ~2.5 h)
scripts/retrain.py for automatic local GPU → CPU fallback dispatch.
Signal and position models are always trained together. They share the same StandardScaler. Training one without the other breaks scaler dimensions and causes wrong predictions or crashes at inference.
Common commands
# Standard retrain — signal + position models
python train_ml_model.py --ensemble --position
# Full retrain including risk multiplier model
python train_ml_model.py --ensemble --position --risk
# Fresh data from broker + full retrain
python train_ml_model.py --ensemble --position --risk --refresh
# Full reset (no warmstart — train from scratch)
python train_ml_model.py --ensemble --position --risk --no-warmstart
# Full pipeline: fresh data + retrain + SHAP analysis + push to R2
python train_ml_model.py --ensemble --position --risk --refresh --shap --push-to-hub
# SHAP feature importance only (no retrain — uses existing pkl files)
python train_ml_model.py --shap-only
Warmstart (incremental learning)
By default, if models/ensemble_rf.pkl exists, training adds boosting rounds on top of existing weights. The model accumulates knowledge across retrains without forgetting previous patterns.
# Default: warmstart ON when models exist
python train_ml_model.py --ensemble --position
# Force full retrain (discard existing weights)
python train_ml_model.py --ensemble --position --no-warmstart
--no-warmstart after:
- Optuna hyperparameter tuning
- Feature count changes
- A major market regime shift
Optuna hyperparameter tuning
# Tune signal model (50 trials, ~30–60 min on CPU)
python ml/tune/hyperparams.py
# Tune position model
python ml/tune/position.py
# Retrain with new params (always --no-warmstart after tuning)
python train_ml_model.py --ensemble --position --no-warmstart --shap
Risk multiplier model
The risk model is a LightGBM regression that outputs a multiplier applied to base_risk_pct before lot sizing:
effective_risk_pct = base_risk_pct × multiplier
[0.10, 1.25]. When the model files are missing, it falls back to 1.0 — no effect on trading.
How it’s trained:
The trainer runs the OOS backtest internally to get the ordered trade sequence, then replays it across multiple starting balances ($500–$5,000). For each equity state, it tests 9 candidate multipliers (0.15–1.20) over the next 20 trades and labels each state with the multiplier that maximises WR × PF / √MaxDD without breaching the active profile’s max_total_drawdown_pct.
The 7 equity-state features (separate from the 55 signal features):
| Feature | What it measures |
|---|
equity_ratio | Current equity / starting equity |
drawdown_from_peak | (Peak − current) / peak |
rolling_wr_10 | Win rate over last 10 trades |
rolling_wr_20 | Win rate over last 20 trades |
rolling_pf_10 | Profit factor over last 10 trades |
consecutive_losses | Current consecutive loss streak |
atr_norm | ATR at entry / entry price |
max_total_drawdown_pct as the hard ceiling. Profile 1 (20% halt) produces conservative multipliers. Profile 5 (65% halt) allows the multiplier to reach 1.20 during strong runs. When you switch profiles and retrain, the risk model relearns for the new tolerance.
Three-File Rule for the risk model:
ml/risk_trainer.py RISK_FEATURES == ml/risk_predictor.py RISK_FEATURES == ml_config.json → risk_model.features
# Train risk model only (signal + position must already be on disk)
python train_ml_model.py --risk
# Disable without deleting files
# Set "enabled": false in config.json → risk_model
models/risk_lgb.txt, models/risk_scaler.pkl, models/risk_metadata.json.
After every retrain
# 1. Verify the compatibility manifest was written correctly
python3 -c "
import json
mc = json.load(open('models/model_compat.json'))
ml = json.load(open('ml_config.json'))
assert mc['feature_count'] == len(ml['features']), 'MISMATCH'
print('OK:', mc['feature_count'], 'features — trained', mc['trained_at'][:10])
"
# 2. Verify risk model metadata if --risk was used
python3 -c "
import json, os
if os.path.exists('models/risk_metadata.json'):
m = json.load(open('models/risk_metadata.json'))
print('Risk model trained:', m['trained_at'][:10], '| val MAE:', m['val_mae'])
else:
print('Risk model not yet trained — run: python train_ml_model.py --risk')
"
# 3. True OOS backtest
python backtest_config.py \
--balance 500 --no-swap --leverage 500 \
--spread 16.95 --oos-only --no-chart
# 4. Push to R2
python ml/r2_hub.py --push
# 5. Update strategy_params.json and TODO.md
Labeling system
atr_aware (active — always use this):
- BUY (1): Long TP hit before SL within 48 M15 bars. TP = ATR×1.5, SL = ATR×0.8.
- SELL (0): Short TP hit before SL.
- HOLD (2): Neither hit — chop or no edge.
- News override:
is_news_near == 1 AND volume_surge > 3 → force HOLD.
- Label distribution: ~38% SELL / ~38% BUY / ~24% HOLD.
sltp_aware (deprecated — do NOT use):
Fixed TP=0.3%, SL=0.25%. Caused live WR to collapse from 78% to 49% because fixed labels do not match ATR-based live execution. Never use with dynamic_sltp.enabled = true.
labeling.method in ml_config.json must always be atr_aware when config.json → dynamic_sltp.enabled = true. Mixing them collapses live win rate from ~78% to ~49%.
Training data
datasets/training_data_btcusd.csv — 70,031 bars, 2-year BTCUSD M15 OHLCV
datasets/news_2015_2026.csv — Historical Forex Factory events
--refresh or the scraper utilities in scraper/.
Model compat manifest
After every unified train, models/model_compat.json is written:
{
"feature_count": 55,
"feature_hash": "abc123...",
"trained_at": "2026-03-05T16:00:00Z",
"models": ["rf", "xgb", "lgb"],
"position_model": true
}
