Image Models Trained Without Language Pretraining Matched Rivals in a Controlled Test

Rethinking how image generators learn

Most popular text-to-image systems today, like Stable Diffusion, rely on huge amounts of prior language training. They read billions of internet captions first to learn what words mean, then combine that knowledge with image data to generate pictures.

A new paper, listed on Hugging Face under Stanford University and described on the blog of the research company Radical Numerics, asks a simple question: do you actually need all that text-only pretraining ¹²? In the conditions the researchers tested, the answer was no.

They created a family of models called Halcyon, trained entirely from scratch on 100 million image-text pairs from a dataset called GPIC, with images at 256×256 pixels ². To compare like with like, they retrained the rival designs on the same data at the same resolution . Instead of teaching the model language separately before showing it images, Halcyon learns both at the same time. This approach is called early fusion.

Faster and cheaper performance

The results surprised many in the field. A smaller version of Halcyon, with 520 million parameters, outperformed LlamaGen-XL — a much larger late-fusion model that learns text and images separately — while using just 18.2 times less pretraining compute.

An even larger Halcyon model, with 1.6 billion parameters, beat another strong late-fusion baseline called JiT-T2I-XXL while using 275 times less pretraining compute. The authors measured performance using gFD, a standard benchmark score for image generation quality, where lower is better.

In these controlled experiments, adding text-only pretraining did not improve the final text-to-image results ¹. That is a finding about this data, this resolution and these model sizes: it does not show that language pretraining is useless for every model or every level of image quality.

What this means for the future

For years, the default assumption in the AI community has been that strong text-to-image generation requires massive language pretraining first. This research challenges that idea for the setting it tested, where models that learn text and images together from the start matched or beat ones built on a separately pretrained language model.

Autoregressive models generate content one piece at a time — for images, they predict pixels or image tokens step by step — rather than denoising an entire image all at once like diffusion models do.

The team also showed that Halcyon can be adapted to caption images by replacing a technique called caption dropout with a different training method called I2T supervision. This meant the model could learn to describe images in text without sacrificing its ability to generate images from text prompts, all within the same pretraining budget.

Code and models for Halcyon are publicly available online.

Corrections and updates

• Corrected on 20 October 2026. An earlier version did not say that all the models were trained and compared on 256×256 images from one dataset, and it presented the finding about language pretraining as true in general rather than under the conditions tested. It also listed four encyclopedia pages that the story does not rely on as sources. These have been corrected.
• Corrected on 20 October 2026. The headline said a new model shows you do not need language pretraining to create images from text. The paper found this under the conditions it tested (256×256 images and 100 million image-text pairs), and the headline now says so.

Key facts

• Halcyon was trained from scratch on 100M image-text pairs without prior language pretraining
• Halcyon-0.52B outperformed LlamaGen-XL with 18.2x less pretraining compute
• Halcyon-1.6B outperformed JiT-T2I-XXL with 275x less pretraining compute
• In the tested setting (256×256 images, 100M image-text pairs), text-only pretraining did not improve text-to-image results
• Halcyon can also learn image captioning within the same training budget

Sources

1. Paper page - Text-to-Image Generative Modeling Without Language Pretraining
2. Halcyon, by Radical Numerics