← Alpha-v2.1AI AGENTS / ALPHA-V2.1 / METHODOLOGY / ARCHITECTURE
Methodology · § 02 / 05

Architecture

Alpha-v2.1 (model_id = 2) · by Quant Lab · Independent encoders + cross-symbol attention mixer

Inherits the shared cross-symbol encoder + per-task readout adapters of v2. Three architectural changes:

Adapter rank-collapse fix. SwiGLU FFN inside ResidualMLPBlock was replaced with a classical Linear → GELU → Linear. Effective adapter rank (PR_eff) at D=256 jumped from ~1.6 to ~33 — the readout no longer collapses to a single direction and uses ≥12% of the latent dimensionality.

Bias accumulation control. The terminal LayerNorm in LatentReadoutAdapter is now elementwise_affine=False. Mean-norm ‖mean‖₂ dropped from 16.4 to ~8.6, removing the accumulated bias that was breaking Q-head calibration over long retraining cycles.

Actor head — ResidualSkipHead (Phase 13). The 1-layer linear head is replaced with a residual skip variant: skip = Linear(LayerNorm(feat)) (Phase-1 baseline behaviour, guaranteed) plus a non-linear MLP(GELU) branch with a learnable gate, init zero. At gate ≈ 0 the head is mathematically identical to the linear baseline; the gate may open during training to add non-linear capacity. Migration logic preserves trained weights when converting linear → skip checkpoints.

← PREVIOUS
Overview
NEXT →
Training
All sections of Alpha-v2.1
01
Overview
02
Architecture
03
Training
04
Regulators / Objectives
05
Changelog
Architecture — Alpha-v2.1 — PROGMATIC