> ## Documentation Index
> Fetch the complete documentation index at: https://docs.novosky.app/llms.txt
> Use this file to discover all available pages before exploring further.

# How it works

> A complete walkthrough of the NOVOSKY trading loop — from raw candles to executed order to position close.

## Trading loop overview

`trading.py` runs in a \~60-second loop. On every iteration it:

1. Fetches the latest BTCUSD M15 candles from the MT5 REST API
2. Engineers 62 features from the raw OHLCV data
3. Scores the bar through the signal ensemble
4. Applies filters and gates
5. Sizes the position with the dynamic SL/TP and risk multiplier models
6. Executes via the MT5 REST API
7. Manages open positions through the position model

```mermaid theme={null}
flowchart TD
    A[BTCUSD M15 candles\nMT5 REST API] --> B[Feature engineering\n62 features]
    B --> C{Signal model\nRF + XGB + LGB}
    C -->|confidence >= threshold| D[Signal: BUY / SELL]
    C -->|below threshold| E[HOLD — skip]
    D --> F[Filters\nATR floor · circuit breaker\nweekly DD pause · hard halt]
    F -->|pass| S{Dynamic SL/TP model\nLightGBM regression}
    S -->|SL/TP multipliers| G{Risk multiplier model\nLightGBM regression\n7 equity-state features}
    G -->|risk multiplier 0.10–1.25| H[Position sizing\nbase_risk% × multiplier × equity ÷ SL]
    H --> I[Execute order\nMT5 REST API]
    I --> J{Position model\nRF + XGB + LGB}
    J -->|HOLD| K[Wait for TP or SL]
    J -->|EXIT| L[Early close]
    K & L --> M[Log trade\nTelegram · Supabase]
```

***

## Stage 1 — Feature engineering

`ml/feature_engineering.py` transforms raw OHLCV candles into the 62-feature vector that all four models consume.

Feature families:

* **Price structure:** Bollinger Bands, ATR, range position, high/low ratios
* **Trend:** EMA crossovers, MACD signal, trend strength, price vs EMA200
* **Momentum:** RSI, RSI divergence, momentum over 5 and 10 bars
* **Volatility:** Normalized volatility, ATR-to-price ratio
* **Volume:** Volume ratio vs rolling average, volume trend
* **Session/news:** London, NY, Asian session flags; news proximity and risk window
* **Account state:** Current drawdown %, equity ratio, recent win rate, consecutive losses

The feature vector is identical at training time and inference time. The order is fixed — any change requires full retraining.

***

## Stage 2 — Signal model

`ml/ensemble_predictor.py` runs a **majority-vote ensemble** of Random Forest, XGBoost, and LightGBM classifiers.

Each model outputs a probability for BUY, SELL, or HOLD. The ensemble aggregates by majority vote, then applies a confidence threshold. Trades below `confidence_threshold` (default 0.633) are discarded as HOLD.

```python theme={null}
# simplified inference
rf_pred = rf_model.predict_proba(features)
xgb_pred = xgb_model.predict_proba(features)
lgb_pred = lgb_model.predict_proba(features)
signal, confidence = majority_vote([rf_pred, xgb_pred, lgb_pred])
if confidence < threshold:
    return HOLD
```

Model inference order: **calibrated pkl → ONNX → raw pkl**. CalibratedClassifierCV cannot be converted to ONNX, so the ONNX fallback uses the uncalibrated model.

***

## Stage 3 — Filters and circuit breakers

Before any order is placed, the signal passes through a chain of gates:

| Gate            | Triggers when                                   | Action                                   |
| --------------- | ----------------------------------------------- | ---------------------------------------- |
| ATR floor       | Current ATR \< `min_atr`                        | Skip — market too quiet                  |
| Circuit breaker | Consecutive losses ≥ `max_consecutive_losses`   | Pause trading                            |
| Weekly drawdown | Weekly loss ≥ `max_weekly_drawdown_pct`         | Pause for the week                       |
| Hard halt       | Total equity loss ≥ `hard_halt_pct`             | `sys.exit(99)` — manual restart required |
| News block      | `is_news_near` = 1 and `news_block_minutes` > 0 | Skip — high-impact news window           |

The hard halt never restarts automatically. Before restarting after a halt, update `starting_balance_usd` in `config.json` to your current equity.

***

## Stage 4 — Dynamic SL/TP

`ml/sltp_predictor.py` uses a LightGBM **regression** model to predict optimal SL and TP multipliers relative to ATR.

```python theme={null}
sl_pts = sl_multiplier × ATR × config.atr_multiplier
tp_pts = tp_multiplier × ATR × config.atr_multiplier
# tp_pts must always be > sl_pts (Phase 16 fix)
```

The equity-aware SL cap then applies:

```python theme={null}
cap_pts = (max_risk_pct × equity) / (lot × pip_value)
sl_pts = min(sl_pts, cap_pts)
```

This ensures that even if the model predicts a wide SL, actual risk never exceeds your profile's limit.

***

## Stage 5 — Risk multiplier

`ml/risk_predictor.py` runs a LightGBM model that takes 7 equity-state features and outputs a risk multiplier in `[0.10, 1.25]`.

Features: `drawdown_pct`, `equity_ratio`, `win_rate_recent`, `consecutive_losses`, `volatility_20`, `atr_14`, `hour`

The multiplier scales the base risk percentage:

```python theme={null}
effective_risk = base_risk_percent × risk_multiplier
```

When the account is healthy and recent trades are profitable, the multiplier trends toward 1.25 (Kelly-style upscaling). During drawdown or after losses, it reduces toward 0.10.

***

## Stage 6 — Position sizing

Lot size is computed from the effective risk and the SL distance:

```python theme={null}
risk_usd = equity × effective_risk_pct / 100
lot = risk_usd / (sl_pts × pip_value)
lot = max(min_lot, min(lot, max_lot))
```

For cent accounts (`pip_value = 100 USC/lot/point`), `risk_usd` is in raw USC.

***

## Stage 7 — Position management

After order execution, the position model (`ml/position_predictor.py`) evaluates every open position on each loop iteration.

Same 62-feature vector. Outputs:

* **HOLD** — keep the position, wait for TP or SL
* **EXIT** — close early (strong reversal signal)
* **ADD** — increase position size (strong continuation signal; currently conservative)

The position model reduces the number of trades stopped out at SL by closing earlier when momentum reverses.

***

## Weekly optimization pipeline

Every Sunday at 2 am UTC, `scripts/weekly_optimize.py` runs autonomously.

```mermaid theme={null}
flowchart LR
    P0([Pre-flight\nbaseline]) --> P1([MT5 check\ndata refresh])
    P1 --> P2([SHAP\nanalysis])
    P2 --> P3([Optuna\ntuning])
    P3 --> P4([Retrain\nSignal · Position\nSLTP · Risk])
    P4 --> P5([Sweep\nscoped to\nrisk profile])
    P5 --> P6([Apply\nbest config])
    P6 --> P7([OOS eval\nfull window])
    P7 --> P8{Score\nimproved\n≥ 2%?}
    P8 -->|Yes| P9([Push HF\ncommit\nnotify ✓])
    P8 -->|No| P10([Rollback\nnotify ✗])
```

Key invariants:

* The sweep uses only the first 70% of the OOS window — the last 30% is a true holdout
* Models are always trained in order: Signal → Position → SLTP → Risk
* If the new score doesn't beat baseline by ≥ 2%, everything rolls back

See [Optimization pipeline](/ml/pipeline) for the full 13-phase breakdown.