Modern reinforcement learning (RL)-based trading systems can implement either centralized or distributed architectures. The question of choosing between a single-agent and a multi-agent strategy is not just an architectural decision but a fundamental choice about how to represent, delegate, and coordinate actions under conditions of limited shared capital and multiple trading instruments.
In this article we examine the key differences between these approaches, their advantages, limitations, and implementation specifics under real market conditions.
Single-agent behavior: centralized decision-making
Within a single-agent strategy:
- One agent analyzes all available information across all assets;
- Makes decisions simultaneously across all positions;
- Combines risk and return into a single objective space;
- Accounts for capital constraints when forming a unified strategy.
Such an agent can, for example, output a vector of N buy/sell signals for each of N instruments and evaluate the objective function taking into account joint effects: diversification, correlations, capital allocation, etc.
Single-agent advantages
- A unified worldview and consistency of decisions;
- Built-in accounting for risk and capital management;
- Possibility of training on a global metric: total PnL, portfolio drawdown, etc.
Single-agent disadvantages
- Growing policy complexity as the number of instruments increases;
- Scaling problem: a single agent must learn behavior across all symbols;
- Constraints on the model structure (e.g., a shared encoder-decoder scheme with memory limitations).
Multi-agent behavior: independent agents per symbol
In a multi-agent architecture:
- A separate agent is allocated to each symbol (or group of symbols);
- Each agent is trained independently, having access only to its own data;
- Decisions are formed in a decentralized manner, but require external coordination to handle shared capital constraints.
Multi-agent advantages
- Simplicity of training: each agent solves a limited task;
- Scalability: adding a new asset only requires adding a new agent;
- Possibility of specializing agents for different market regimes and instruments.
Multi-agent disadvantages
- Lack of a unified worldview;
- Need for an external capital management mechanism (an allocator);
- Risk of conflicting decisions: several agents may push for aggressive allocation under limited capital.
Capital allocation in a multi-agent system
In a multi-agent architecture, the need arises to coordinate agents through a shared limited resource — capital.
Here a meta-agent or capital manager task arises, which:
- Observes the forecasts or signals from the agents;
- Evaluates each agent's confidence, expected return, risk, or utility;
- Allocates capital among the agents, limiting total exposure;
- Can dynamically reallocate capital among agents as the market changes.
Implementations of the meta-agent
Such a meta-agent can be implemented as:
- a heuristic system based on rules and statistics;
- a learnable model (e.g., RL or a Bayesian optimizer) maximizing total reward;
- an attention layer over the agents, with soft allocation.
Implementation examples in trading
Single-agent model:
- A direct action vector a = [a1, a2, ..., an] from a unified model;
- Scaling of signals taking the total available balance into account;
- Reward: total equity change, Sharpe ratio, drawdown.
Multi-agent model
- n agents: each emits an individual signal and confidence;
- Meta-model (allocation): w_i = f(signal_i, confidence_i, risk_i);
- Reward: the meta-reward can take into account not only return, but also regularization of capital deployment, switching frequency, etc.
Hybrid approaches
Intermediate architectures are also possible:
- A shared encoder with separate policy heads;
- Hierarchical agents: one agent sets the strategy, others execute the tactics;
- Role distribution: some agents are active, others observe and learn.
Conclusion
The choice between a single-agent and a multi-agent strategy is a balance between:
- centralized optimization and modular scalability;
- complete information and agent independence;
- complexity of the global model and flexibility of distributed execution.
The key engineering task is not just to teach agents to act, but to teach them to allocate capital under competition between agents with differing predictive ability. This opens the path to building next-generation trading systems: adaptive, scalable, and interacting like a complex organism.