Actor pathway (per-symbol policy).
obs → actor_encoder (SnapshotEncoder, d_model = 128)
→ actor_symbol_mixer (TransformerEncoder × 2, 4 heads)
→ ObsContextProjector (residual context)
→ actor_mean_head [CrossSampleCenter → Linear → GELU → Dropout → Linear]
→ clamp(±3) → tanh → action ∈ [−1, +1]Critic pathway (twin Q-ensemble).
obs → critic_encoder (separate from actor — v2.3 split)
→ critic_symbol_mixer (TransformerEncoder × 2, 4 heads)
→ ObsContextProjector
→ critic_feature_csc (shared anti-bias)
→ concat(action) → DiverseCritic × 2 (16-quantile distributional Q)Concept innovations.
1. CrossSampleCenter (E1 anti-bias). The headline architectural change of v2.4.18. A module between the feature stack and the policy head that strips per-feature batch mean. Goal: remove the DC component accumulating in encoder / mixer residual streams, which otherwise nudges the head into learned bias compensation through weight alignment. Applied symmetrically on both sides (actor head + critic features) — the actor / critic architectural-symmetry principle: changing the feature representation on one side mirrors on the other, otherwise gradient flow through ∂Q/∂a breaks. Train-mode uses the current batch mean; eval-mode uses an EMA running mean (momentum 0.01). Target networks sync running_mean through the Polyak update.
2. Per-symbol distributional Q. Each twin critic predicts a 16-quantile distribution of Q for every symbol independently. CVaR-aware penalty in the actor objective favours actions with a tighter left tail.
3. World model + Planner. Parallel TD-MPC contour — latent dynamics + reward predictor, CEM planner for multi-step lookahead. Activated periodically (every 80 cycles) for verification and at difficult decision points.
4. Pattern Memory. State clustering in latent space (MiniBatchKMeans, K randomized per cycle) for structured replay sampling. Sign-balanced quotas (UP / DOWN / FLAT) prevent bias toward dominant regimes in the buffer.
5. Adaptive supplement. Parallel-contour generation of synthetic trajectories — actor-based exploration + TD-MPC rollouts + isolated contour workers — for diversity.