Combined Optimisation Metrics

Advanced approaches to evaluating trading strategy quality with consideration of turnover and commissions.

Comparison of strategy evaluation metrics

When training trading strategies with reinforcement learning, properly evaluating model quality is essential. Traditional metrics have limitations that may lead to suboptimal decisions in real trading.

Main evaluation metrics:

train_idx — dpnl / (1+DD) — "Return-on-risk": how much profit remains after accounting for maximum drawdown
pnl / turnover — profit divided by total trade volume — Trading efficiency: how many dollars are earned per dollar traded (a Sharpe/IRR proxy on "capital-turnover")

Limitations of train_idx

ISSUES AT EQUAL TRAIN_IDX

Commissions and slippage will erode net profit more heavily; larger margin/balance is required for the same positions; liquidity may become a bottleneck.

The train_idx metric shows risk-adjusted return but ignores the amount of money "pushed" through the market. This can lead to serious problems when transitioning to real trading.

Combined score (prod_score)

To address these problems, a combined approach is proposed that considers both risk-adjusted return quality and trading efficiency.

prod_score = train_idx * 0.7 + (pnl / turnover) * 0.3

70% — RETURN QUALITY

Our usual train_idx, which captures risk-adjusted return.

30% — TRADING EFFICIENCY

Profit per dollar of turnover, accounting for commissions and slippage.

IMPORTANT

The 0.3 weight is conventional and can be adjusted to priorities. The model should not sacrifice turnover for a token improvement in the index.

Volume decrease bonus (use_volume_decrease_bonus)

TWO CAVEATS

1) It does not guarantee that the final checkpoint is comparatively economical — the bonus may have been overshadowed by random profitable high-volume trades. 2) It only operates during training. For deferred validation (test-chunk) the bonus is not awarded, and we may observe the opposite effect.

use_volume_decrease_bonus does indeed push the policy toward lower turnover, but it has important limitations:

How it works

During training — bonus-shaping (use_volume_decrease_bonus) gently nudges gradients toward "less volume"
During model selection — the combined prod_score looks at the fact: "How much did the strategy actually earn and at what cost in turnover"

Practical example

Consider a comparison of two models with hypothetical numbers:

Model A — PNL $1,200 — Max DD 6% — Turnover $600,000 — train_idx 0.113 — PNL/Turnover 0.0020 — prod_score 0.083
Model B — PNL $1,050 — Max DD 4% — Turnover $200,000 — train_idx 0.100 — PNL/Turnover 0.0053 — prod_score 0.085

RESULT ANALYSIS

train_idx says A is better (0.113 > 0.100). But B has 2.5x higher PNL/Turnover. Applying prod_score, B wins (0.085 > 0.083) — slightly less profitable but 3x more economical in turnover.

Implementation

In the save block (_train_ppo → _ppo_iteration), after obtaining train_metrics:

idx = train_metrics["index"]
pnl = train_metrics["pnl"]
turnover = max(train_metrics["turnover"], 1e-8)  # protection against /0

efficiency = pnl / turnover  # $ per $ of turnover
prod_score = idx * 0.7 + efficiency * 0.3

We store the best pair (best_prod_score, best_checkpoint) and compare against it:

if prod_score > best_prod_score:
    best_prod_score = prod_score
    save_checkpoint(...)

Optionally, both metrics can be logged to track the trade-off.

Summary: complementary mechanisms

KEY PRINCIPLE

They do not replace but complement each other: one helps train more "carefully", the other selects what is genuinely more profitable in production.

The two mechanisms operate at different levels and complement each other:

use_volume_decrease_bonus — an on-policy switch that forces the agent to remember the "cost" of turnover; works during training, steering gradients toward smaller volume
prod_score — an off-policy filter that picks the final model by the PNL / volume ratio; evaluates the already-trained model in terms of real trading efficiency