Agentic RL Has a Scaling Law — And It Changes How You Should Spend Your Budget

Beyond Token Counts

For years, pretraining scaling laws have treated data as a single number: how many tokens you feed the model. But agentic reinforcement learning (RL) — where an AI agent learns by interacting with environments, generating tasks, and collecting trajectories — has blown that scalar abstraction apart ¹. Modern synthesis pipelines now create tasks, environments, and trajectories independently and at wildly different costs, turning training data into a structured design space the old laws never had to navigate .

The question that nobody knew how to answer is now getting an answer: when you have a fixed synthesis budget, which axis should you scale up? ¹.

Three Axes, One Clear Winner

Researchers including Bowei He, Yankai Chen, Xiaokun Zhang, and colleagues at Alibaba and DeepSeek decomposed agentic data into three cardinality axes — task diversity, environment diversity, and trajectory count — plus a reward-density control variable ¹. They then fitted per-axis scaling relations across different large language models (LLMs), across math-and-code, software-engineering, web-navigation, and UI/tool-application domains .

The result was unambiguous and consistent: across all model sizes, model families, domains, and both PPO and GRPO RL algorithms they measured, the scaling exponents ranked environment > task > trajectory ¹. When they further split environment diversity into world diversity (the range of settings, actors, and scenarios) versus environment-structure diversity (the complexity of the task scaffolding), the advantage came almost entirely from world diversity, while structure diversity sat roughly on par with task diversity .

In plain terms: making your agent see more different kinds of worlds matters far more than making it solve more problems or rehearse more steps ¹.

Pricing Synthesis Methods

The researchers then used those fitted laws as a standard measuring stick to price synthesis methods, capturing each method's efficiency as a per-axis discount factor and identifying critical synthetic-to-real ratios at which collapse begins ¹. Those collapse ratios differed by an order of magnitude across axes, meaning blind scaling on any single dimension can be far more dangerous than the numbers suggest .

A Chinchilla-Style Recipe for Agentic RL

Finally, they cast budget allocation as a constrained optimisation under their laws and discount factors, deriving a Chinchilla-style compute-optimal recipe and validating it at a held-out scale ¹. Following that recipe yielded up to 1.7× compute efficiency over the dominant industry practice of simply scaling the number of trajectories .

The paper, How Data Scales in Agentic Reinforcement Learning: Laws and Synthesis Strategies, has been accepted by NeurIPS 2026 ¹. As agentic RL systems move from research demos into production, the old instinct to "just throw more data at it" is being replaced by a much more precise intuition: spend your budget on worlds, not steps.

This story reports on a trend for information only. It is not advice or encouragement to do any of it.

Corrections and updates

Corrected on 23 October 2026. An earlier version said the researchers compared AI image generation techniques; the paper is about agentic reinforcement learning, not image generation.

Key facts

Agentic RL scaling exponents order: environment > task > trajectory, consistent across models and domains
World diversity within environments drives the advantage; structure diversity is comparable to task diversity
Synthetic-to-real collapse ratios differ by an order of magnitude across axes
A Chinchilla-style compute-optimal recipe yields up to 1.7× efficiency over trajectory-scaling practice
Paper accepted at NeurIPS 2026