The actor loss is decomposed into separate, coefficient-controlled terms — each is logged as a raw addend so the contribution of every regulator to the final action is explicit.
Q-driven force. Per-symbol Q + global Q + CVaR penalty (the primary signal that pulls the actor toward profitable positions while penalising tail risk).
action_l2. Magnitude penalty on |action|.
portfolio_budget. Penalty for exceeding gross-exposure cap across symbols.
inaction. Anti-collapse — penalty for zero / near-zero positions.
diversity. Penalty for low std of actions across the batch (prevents the actor from outputting identical positions).
dir_imbalance. Penalty for direction skew — keeps long / short share balanced.