Skip to main content

​
Overview

scripts/weekly_optimize.py runs every Sunday at 2 am UTC via cron. It takes approximately 2–3 hours on a 4-core VM.

​
The 13 phases

1

Pre-flight and baseline

Loads current models, runs a quick backtest on recent OOS data, and records the baseline score. This score is the bar that the new run must beat.
2

MT5 connectivity check and data refresh

Verifies the MT5 API is reachable, then fetches the latest BTCUSD M15 candles to extend the training dataset.
3

SHAP analysis

ml/shap_analysis.py computes SHAP feature importances across all three signal models. The output informs which features are contributing and flags anomalies.SHAP results do not automatically drop features. Protected session/news features are never dropped regardless of SHAP values.
4

Optuna hyperparameter tuning

ml/tune/hyperparams.py runs an Optuna study to find the best hyperparameters for the signal ensemble. The objective function is a custom metric combining OOS precision, recall, and drawdown.Typical search: 50–100 trials. Results are saved to results/optuna_study.pkl.
5

Retrain — Signal model

Trains RF, XGB, and LGB signal classifiers with the tuned hyperparameters. Runs calibrate_models() after training to produce calibrated pkl files.
6

Retrain — Position model

Trains the position model using ml/tune/position.py hyperparameters. Requires the signal model to be already saved.
7

Retrain — SL/TP model

Trains the LightGBM SL/TP regressors. Requires signal and position models to be saved.
8

Retrain — Risk model

Trains the LightGBM risk multiplier model. Requires SL/TP models because training runs a full backtest to generate account-state labels.
9

Config sweep (scoped to risk profile)

scripts/sweep.py sweeps a grid of config parameters: confidence_threshold, risk_percent, sl_multiplier, tp_multiplier, and session filters. The sweep is scoped to your active risk profile — it only tests parameter combinations within the profile’s bounds.The sweep runs on the first 70% of the OOS window only.
10

Apply best config

The parameter combination with the highest score is written to config.json.
11

Full OOS evaluation

A clean backtest runs on the full OOS window including the 30% holdout that the sweep never saw. This is the definitive score.
12

Commit or rollback

If new_score >= baseline × 1.02: push models to HF Hub, commit config.json, and send a Telegram success report.Otherwise: restore models and config from backup, send a Telegram failure report. The bot continues with the previous version.
13

Cleanup and cron reschedule

Cleans up temporary files and confirms the next Sunday cron run is scheduled.

​
Running the pipeline

# Full run with risk questionnaire (first time)
python scripts/weekly_optimize.py --balance 500

# Skip questionnaire — use stored profile
python scripts/weekly_optimize.py --profile balanced

# Skip retrain — sweep and OOS only (~45 min)
python scripts/weekly_optimize.py --skip-retrain

# Resume from a specific phase after a crash
python scripts/weekly_optimize.py --from-phase 5

# Dry run — shows what would run without executing
python scripts/weekly_optimize.py --dry-run

​
Manual retrain path

To retrain outside of the weekly pipeline: 
# Minimal retrain — all 4 models in correct order
python train_ml_model.py --ensemble --position --sltp --risk

# With SHAP analysis and data refresh
python train_ml_model.py --ensemble --position --sltp --risk --refresh --shap

# Retrain risk model only (e.g. after config change)
python train_ml_model.py --risk

# Validate and push
python backtest.py --balance 500 --no-swap --leverage 500 --spread 16.95 --oos-only --no-chart
python ml/hf_hub.py --push
Never train models out of order. The Risk model depends on a backtest that requires SLTP models. The Position model depends on signal model outputs. Always use: Signal → Position → SLTP → Risk.

​
Optuna tuning

Each model has its own Optuna study:
StudyFileTrialsObjective
Signal hyperparamsml/tune/hyperparams.py50–100OOS F1 × (1 − max_drawdown)
Position hyperparamsml/tune/position.py30–50OOS accuracy × exit precision
Config sweepscripts/sweep.pyGrid searchScore formula
To run Optuna tuning independently: 
python ml/tune/hyperparams.py          # signal model tuning
python ml/tune/position.py             # position model tuning
python scripts/sweep.py --profile 3   # config sweep for profile 3

​
Backtesting

backtest.py is a config-faithful bar-by-bar backtester in backtest/run.py. It replicates the live trading loop exactly — same feature engineering, same model inference, same risk sizing, same circuit breakers. 
# Standard OOS validation
python backtest.py \
  --balance 500 \
  --no-swap \
  --leverage 500 \
  --spread 16.95 \
  --oos-only \
  --no-chart

# Full window (IS + OOS)
python backtest.py --balance 500 --no-swap --leverage 500 --spread 16.95

# With chart output
python backtest.py --balance 500 --no-swap --leverage 500 --spread 16.95 --oos-only
--oos-only runs only on the held-out OOS segment. Always prefer this for realistic validation. Generic lookback backtests can be contaminated by in-sample data.