AIGenerated
EN · IT · ES

Training AIs on images makes them think more spatially even when you give them only words

A new study shows that giving language models practice with pictures changes how they solve text-only puzzles — and why this matters for the AI tools you use every day.

How it worksTraining AIs on images makes them think more spatially even when you give them only wordsHerald of AI

What you could do with it

  • AI tool designers could use these findings to build smarter assistants by understanding how prior multimodal training shapes a model's behaviour and where it might make odd mistakes 1.
  • Researchers could test different multimodal configurations across model families to see whether vision-shaped thinking emerges in text-only reasoning 1.
  • Engineers working on dual-path unified models like BAGEL or Janus-Pro could pinpoint exactly which pathway carries spatial thinking during generation and adjust design accordingly 1.

What the researchers did

A team including Qi Zhang, Bohan Wang, Qianru Sun and Hanwang Zhang wrote a paper published on the pre-print site arXiv on 11 November 2026 1. They asked a simple but surprising question: if you train an AI on both text and images, does it keep using that visual experience when someone later gives it only words?

To find out, they tested ten different multimodal setups across six different AI families, comparing them against matching text-only versions of the same models 1. They used two kinds of puzzles. One set required the model to imagine scenes in space — things like mental rotation, paper folding or imagining how a view looks from another person's perspective. The other set was purely symbolic, like basic logic problems that can be solved by following rules without any pictures .

They then "ablated" or blocked the part of the model that carries vision-shaped thinking and watched what happened to its answers 1.

What changed when vision-thinking was turned off

In the Chameleon model, which has about 7 billion parameters, blocking that vision-related pathway dropped scores on a tough reasoning test called BIG-Bench Hard (BBH) navigate by 27.6 percentage points, while leaving boolean-expression questions almost unchanged, even improving them slightly by 5.6 percentage points 1. That gap of 33.2 percentage points between the two tasks is what researchers call a double dissociation — it proves the two skills live in different parts of the model .

The effect was also very specific: it depended on the direction of the signal and appeared mainly in certain layers of the network 1. It was not caused by the tasks simply being harder, needing more reasoning steps or involving longer sequences .

When the researchers tested larger models around 32 billion parameters, the effect spread beyond pure spatial thinking to include binding together different visual features like colour and shape 1.

How different model designs handled it

In special dual-path models called BAGEL and Janus-Pro, the vision-shaped thinking sat neatly inside the generation path as a thin one-dimensional carrier, while a separate understanding path stored spatial information in a richer form but wasn't the main driver of performance 1. This helped the team separate what a model can decode from what it actually uses to reason .

The study suggests that multimodal training doesn't just improve how well an AI handles pictures — it permanently reshapes the geometry of how it reasons with words alone 1. That means every chatbot or helper tool built on these foundations may carry hidden visual habits shaped by earlier image training, whether or not anyone feeding it text ever intended to use them .

This story reports on a trend for information only. It is not advice or encouragement to do any of it.

How good is it

In the Chameleon-7B model, removing the vision-shaped thinking pathway caused accuracy on the BIG-Bench Hard (BBH) navigate task to drop by 27.6 percentage points while boolean-expression accuracy stayed essentially the same, even improving slightly by 5.6 percentage points, creating a 33.2 percentage-point gap that researchers call a double dissociation 1. When tested on larger 32-billion-parameter models, the effect broadened from purely spatial reasoning to include binding together visual features like colour and shape . The study covered ten multimodal configurations across six model families compared against matched text-only versions, conducted by Qi Zhang, Bohan Wang, Qianru Sun, and Hanwang Zhang . Higher scores on BBH navigate indicate stronger spatial-reasoning ability, so the sharp drop shows that vision-shaped thinking plays a causal role in those tasks .

Key facts

  • Blocking vision-shaped thinking in the 7B Chameleon model dropped BBH navigate accuracy by 27.6 percentage points while boolean expressions improved by 5.6 pp
  • The effect was direction-specific, layer-localized, and not explained by task difficulty or reasoning depth
  • Larger 32B models showed the effect broadening to non-spatial visual attribute binding
  • In dual-path models BAGEL and Janus-Pro, causal vision-shaping localised to the generation pathway

Sources

  1. Emergent Vision-Shaped Thinking in Text-Only Reasoning through Multimodal Training

If this story concerns you or your company, or uses something that belongs to you, ask for a correction or for it to be taken down. How requests work

Keep reading