AI image generators improve mostly by a fixed nudge, not new tricks
A study finds that making Diffusion Transformer images look better after training hardly needs complex AI — just one constant adjustment works almost as well.

What you could do with it
- AI tool builders could slash training costs and time by swapping heavy reinforcement-learning weight updates for simple fixed shifts applied to frozen models 1.
- Researchers could speed up experiments on diffusion transformers by testing whether their own models' improvements also boil down to one constant adjustment 1.
- Anyone deploying AI image or video tools could serve higher-quality outputs faster since the fixed-shift trick needs far less computing power than full fine-tuning 1.

Figure: Meral et al., arXiv:2611.26627, CC BY 4.0
What the researchers found
SD3.5-M, a popular open AI image generator, was the test subject. Researchers gave it reinforcement learning post-training — a process that refines a already-trained model using reward signals 1. Usually, this involves updating many internal weights and requires a large amount of computing power.
But when they looked closely at what actually changed during this refinement, they found something surprising: most of the improvement came from a single constant shift applied to the model's block outputs — its internal processing stages — regardless of the prompt, the specific word, or the randomness seed used when generating images 1.
This means the improved behavior does not come from the model learning new prompt-dependent skills. Instead, the model is simply nudged toward behaviors it already had the ability to perform.
How much does a fixed shift really replace?
To test this idea, the researchers built a simplified version of the trained model. They kept the original Diffusion Transformer completely frozen and added only a small set of steering vectors — fixed numerical adjustments added to each block's output 1. These vectors were identical for every image request.
The result was striking: these simple vectors recovered between 90.0% and 96.3% of the quality gains achieved by six different RL-tuned versions of the same model 1. The approach used only 0.29 million parameters compared with 18.8 million used by standard RL fine-tuning — roughly 65 times fewer .
On two RL-tuned video generation models, the fixed-shift method captured 89.8% and 111.9% of the original reward improvement, suggesting it can sometimes over-correct in ways that still land on high-quality results 1.
When trained from scratch with online reinforcement learning, these same vectors alone exceeded the text-rendering and human-preference scores of full weight-based training on SD3.5-M 1.
What this means in plain terms
Imagine you have a talented musician who already knows every song but plays them all too softly. Rather than teaching them new songs, you simply turn up the volume — everything sounds better because the skill was already there 1. That is essentially what happens here: reinforcement learning appears to amplify existing capabilities rather than create entirely new ones.
For developers and creators, the implication is practical. Heavy RL fine-tuning remains useful, but a much simpler fixed adjustment can capture nearly all of its benefits at a fraction of the computational cost 1. This could lower barriers for smaller teams and individuals wanting to customize image generators without expensive cloud resources.
Limitations to keep in mind
The study was conducted on one specific model architecture and benchmark suite. Researchers themselves note that while the fixed-shift explanation captures the majority of observed gains, some prompt-dependent improvements may still require more complex modeling 1. The work opens a new direction rather than closing the book on reinforcement learning for generative AI.
The full paper is available on Hugging Face under the identifier arXiv:2611.26627 1.
How good is it
On the SD3.5-M image generator, adding one fixed set of steering vectors to a frozen base model captured 90.0% to 96.3% of the quality gains achieved by six different reinforcement-learning-tuned versions 1. That tiny adjustment used just 0.29 million parameters versus 18.8 million required by standard RL fine-tuning — about 65 times fewer weights to manage . For two RL-tuned video models, the same approach recovered 89.8% and 111.9% of the original reward improvement; the 111.9% figure means the fixed-shift approach can exceed the reward gains of the full RL-tuned version . Starting from zero values and trained using online reinforcement learning, just those vectors by themselves deliver stronger reward improvements on SD3.5-M than conventional weight-based reinforcement learning does, as measured by OCR performance for text rendering and PickScore for human preference.

Figure: Meral et al., arXiv:2611.26627, CC BY 4.0

Figure: Meral et al., arXiv:2611.26627, CC BY 4.0

Figure: Meral et al., arXiv:2611.26627, CC BY 4.0
Key facts
- Fixed steering vectors recovered 90.0–96.3% of RL reward gains on SD3.5-M using only 0.29M parameters vs 18.8M
- The shift is identical for every prompt, token, and sampling seed
- 65× fewer parameters than standard RL-tuned LoRA on the same model
- Method also worked on video generation models, capturing up to 111.9% of original gains
- Most improvement comes from amplifying existing abilities, not learning new ones
Sources
If this story concerns you or your company, or uses something that belongs to you, ask for a correction or for it to be taken down. How requests work
Keep reading
How it worksSpecWM turns AI video verification into improvement, speeding generationHerald of AI
SpecWM turns AI video verification into improvement, speeding generation
A new method makes AI-generated videos faster and higher quality by fixing mistakes instead of starting over.
Agentic RL Has a Scaling Law — And It Changes How You Should Spend Your Budget
A new study finds that in agentic reinforcement learning, diversifying the worlds your agent trains in beats grinding out more trajectories — and the right mix can yield up to 1.7× compute efficiency.
How it worksAI scans 4,000 medical studies — most don't share useful codeHerald of AI
AI scans 4,000 medical studies — most don't share useful code
Researchers used AI to find out whether doctors and scientists can actually rebuild results from published studies, and the answer was mostly no.