Teaching AI Agents to Know When to Stop

The Problem With Tool-Using Agents

Qwen3-4B and other large language model agents are increasingly being given access to external tools, which are software programs or APIs they can call to perform tasks. These agents can search the web, run code, click buttons in a browser, and perform many other actions. But having tools is only useful if the agent knows when to actually use them.

At any moment, an agent faces three choices: act on its own, collect more information, or escalate the task to a human. Each decision involves a trade-off. Acting too quickly can lead to irreversible errors. Gathering too much information wastes time and money. Asking for human help too often defeats the purpose of having an autonomous agent in the first place.

The challenge is that traditional training only shows agents the outcome of actions they actually took, leaving all alternative choices unobserved ¹. It is like learning to drive by only seeing what happened when you turned left at every intersection, never knowing what would have happened if you had gone straight or turned right .

A New Way to Train Agents

Researchers including Qisai Liu, Qingyue Jiao, and colleagues from various institutions have introduced a method that treats these decision points differently. They built a dataset of over three thousand outcomes from one thousand thirty-six distinct moments where an agent had to choose between available actions, recorded in a platform called ToolSandbox ¹.

At each decision point, every legal action was actually executed, and its downstream consequences were evaluated based on task completion, interaction cost, and the risk of irreversible errors ¹. This approach creates a counterfactual dataset, meaning the system records what would have happened with every possible choice, not just the one the agent originally made .

SURE-Act Improves Agent Decisions

Using this dataset, the team trained a controller called SURE-Act that operates on top of a frozen backbone agent. Rather than retraining the entire model, SURE-Act simply learns to rank the alternative actions available at each decision point and selects the best one at inference time ¹.

In trials, teaching models to rank different possible actions raised offline action-selection utility by 0.0309 compared with a baseline that relied only on regression ¹. The researchers also demonstrate that incorporating SURE-Act delivers cumulative improvements of as much as 0.0575 on ToolSandbox's Overall Score when paired with the Qwen3-4B agent .

The system also outperformed adapted decision rules from two other approaches: SAGE-Agent and Calibrate-Then-Act, by margins of 0.1506 and 0.1438 respectively ¹. The study introduces the first counterfactual dataset built for ToolSandbox and offers practical guidance on enabling agents to take better decisions when facing uncertainty .

This story reports on research for information only. It is not advice or encouragement to implement these techniques.

Key facts

• Researchers built a counterfactual dataset with 3,115 outcomes from 1,036 decision points in ToolSandbox
• SURE-Act controller ranks alternative actions to improve agent decision-making at inference time
• Adding SURE-Act to Qwen3-4B improved end-to-end scores by up to 0.0575 in ToolSandbox
• SURE-Act surpassed adapted rules from SAGE-Agent and Calibrate-Then-Act by margins of 0.1506 and 0.1438 respectively

Sources

1. Agents' Last Doubt: Knowing When to Act and When to Step Back in Tool-Using Agents