Two orthogonal axes. nstep_horizons = [4, 8, 12, 16] → H = 4 horizon HEADS. Each head h is a separate estimator of the same optimal Q*, trained on its own n-step target (4 / 8 / 12 / 16 steps of bootstrap). Heads differ in their bias / variance profile — short horizons are low-variance / higher-bias, long horizons the reverse. num_quantiles = 32 → the granularity of the distribution INSIDE each head. Every head is a full distributional QR-TD3 critic: it emits not a scalar Q but 32 quantiles of the return distribution.
Critic output. A single DiverseCritic now produces a (B, N, H, nq) = (B, N, 4, 32) tensor.
┌─ head n=4 → 32 quantiles ─┐
obs (sa) ──> shared trunk ──> ├─ head n=8 → 32 quantiles ─┤ ──> (B,N,4,32)
(input_proj + blocks) ├─ head n=12 → 32 quantiles ─┤
└─ head n=16 → 32 quantiles ─┘Code shape. A shared trunk plus per-horizon heads: head_norms = ModuleList([LayerNorm(hidden) for _ in range(n_horizons)]) — 4× heads = ModuleList([Linear(hidden, num_quantiles) for _ in range(n_horizons)]) — 4 × Linear(hidden, 32)
num_quantiles is read from critic_ensemble_specs[*].num_quantiles; the top-level "num_quantiles": 32 is now only a fallback — the global Q heads (q1g / q2g) it used to feed were removed back in v2.4.4.