Volume-Decrease Bonus

Strategies for achieving higher returns with smaller trading volume.

Introduction: the principle of trading efficiency

KEY IDEA

The volume decrease bonus aims to achieve higher returns with fewer trading operations, which improves the efficiency of capital usage.

In algorithmic trading there is a fundamental problem: how to achieve maximum return with minimum trading volume. Traditional approaches often produce strategies with excessive trade frequency, which increases commission costs and reduces net profit.

Why it is better to start without the volume bonus

Enabling the volume decrease bonus from the very start of training can lead to undesirable consequences. Consider the main reasons for staged deployment.

Reasons for delayed activation

1. Action exploration — a turnover-reduction bonus immediately "presses" on position size. The model more often gets a negative gradient for any attempt to scale lots, which narrows exploration of the strategy space.
2. Signal balance — at the start of training the "PNL minus commission" signal is already noisy. Adding another (turnover) one with strong weight increases gradient variance and the risk of falling into a local minimum.
3. Ease of debugging — without the volume bonus it is easy to identify the cause of a reward drop: market, position, or code error. With the bonus the cause-and-effect relationship is less obvious.

Issues of early activation

Limited exploration: the agent avoids large positions and does not study their potential
Local minimum: the strategy "trade little -> get a steady bonus"
Diagnosis complexity: hard to pinpoint the source of reward issues

Staged bonus rollout

A three-stage strategy is recommended for rolling out the volume decrease bonus, allowing the agent to adapt to new conditions without abrupt changes in behavior.

A. "Warm-up" — switch criterion: n_env_steps < N_warmup or average daily turnover < N x deposit — parameters: use_volume_decrease_bonus = False
B. "Growth" — switch criterion: agent steadily generates PnL but turnover grows faster than deposit — parameters: enable bonus with small coefficient lambda ~ 0.1-0.2
C. "Efficiency tuning" — switch criterion: model knows how to earn but Sharpe / commission load needs improvement — parameters: raise coefficient to 0.3-0.5; if needed, exponential schedule

Implementation details

The volume decrease bonus is implemented as an additional component of the reward function that rewards the agent for efficient capital use.

# Volume decrease bonus
growth = max(prelim_balance - old_peak, 0.0)
volume_bonus = (growth if self.use_volume_decrease_bonus else 0.0) \
    * (1 / (1 + total_trade_volume / operating_volume))

# Integration into total reward
total_reward = reward_after_bonus + volume_bonus
norm_total_reward = total_reward / operating_volume

HOW IT WORKS

The bonus is proportional to balance growth relative to the peak; high turnover relative to operating volume is penalized; the coefficient can be tuned depending on the training stage.

Comparison with prod_score

LIMITATIONS OF USE_VOLUME_DECREASE_BONUS

Does not guarantee the final checkpoint is economical — the bonus can be overridden by random profitable high-volume trades. Operates only during training; for deferred validation no bonus is awarded.

It is important to distinguish the volume decrease bonus (use_volume_decrease_bonus) and the combined score (prod_score), since they operate at different levels and solve different problems.

Step-reward (on-policy) — use_volume_decrease_bonus — At each step adds an extra bonus for reducing positions, so the agent gets a "do not inflate volume" signal during training
Model-checkpoint (off-policy) — prod_score — Decides which checkpoint to save after a whole series of episodes. Looks at final PNL, DD and turnover — that is "what happened", not "what was attempted"

Practical example

RESULT ANALYSIS

train_idx says A is better (0.113 > 0.100). But B has 2.5x higher PNL/Turnover. Applying prod_score, B wins (0.085 > 0.083) — slightly less profitable, but 3x more economical in turnover.

Consider a comparison of two models that demonstrates the benefits of accounting for turnover in quality evaluation:

Model A — PNL $1,200 — Max DD 6% — Turnover $600,000 — train_idx 0.113 — PNL/Turnover 0.0020 — prod_score 0.083
Model B — PNL $1,050 — Max DD 4% — Turnover $200,000 — train_idx 0.100 — PNL/Turnover 0.0053 — prod_score 0.085

Optimal rollout strategy

A combined approach is recommended that pairs staged bonus activation with comprehensive model quality evaluation.

How it works

KEY PRINCIPLE

The volume decrease bonus and prod_score do not replace each other but complement each other: the first helps train more "carefully", the second selects what is actually more profitable in production.

During training — bonus-shaping (use_volume_decrease_bonus) gently nudges gradients toward "less volume", steering exploration into the desired region
During model selection — the combined prod_score looks at the fact: "How much did the strategy actually earn and at what cost in turnover"

Conclusion: the value of efficient capital use

RESULT

Staged rollout of volume decrease bonuses combined with comprehensive model quality evaluation produces strategies that use capital optimally and minimize trading costs.

Adopting volume decrease bonuses delivers the following advantages:

Higher efficiency — achieving greater return with smaller trading volume
Lower costs — minimizing commission expenses and market impact
Better scalability — strategies become more suitable for large amounts of capital
Greater robustness — less dependence on market liquidity and commissions