TD3 (Twin Delayed DDPG) with deterministic policy and target-policy noise smoothing.
Critic loss. Standard TD3 twin-Q TD with Huber loss (F.smooth_l1_loss). Backup target r + γⁿ × min(Q1', Q2'). Per-symbol and global components are summed.
Actor loss. −E[min(Q1, Q2)] × N_symbols (per-symbol only). The global Q is detached from the gradient.
Polyak target update for every target module: encoder + mixer + global pool + Q heads + actor head.
Random-k clustering. KMeans with randomized n_clusters per cycle (range [30, 1200]) plus per-cycle KMeans seed jitter — anti systematic-sampling-bias.
Sign-balanced PER sampler. UP / DOWN / FLAT quotas over pnl_signed clusters.
Reward normalizers. Two independent z-score EMAs — one on the vec stream (PnL), one on the shape stream (drawdown_pen).
State-dict guard. Fail-fast on legacy v2.1 keys (actor_adapter.*, ResidualSkipHead.*, DiverseCritic.*, …) — clean restart only.