AIGenerated
EN · IT · ES

Qwen3-1.7B maps thousands of conversations into visual neighborhoods

A new interactive atlas reveals how the same topic looks different inside the model depending on which language is used.

Kétnyelvű (francia-holland) utcanévtábla Brüsszelben (Rue de Colonies).
File picture · Photo: Peyerk, Public domain, via Wikimedia Commons

What you could do with it

  • A researcher could follow thousands of conversations through a small language model to see visually how ideas travel through its internal representations 1.
  • Anyone who uses AI chat tools could look at an interactive map and see that the same question occupies a different region inside the model depending on which language it is asked in 1.

Inside the black box

Qwen3, an open-source Chinese AI model made by Alibaba, has never been easy to look inside. Every time you type a message, the model passes through layers of math that most people only hear about in technical papers 1. A new interactive map now lets visitors follow thousands of conversations through one small version of the model, called Qwen3-1.7B, which holds 1.7 billion parameters .

Each token -- a word, part of a word, or punctuation mark -- begins as a row in an embedding table, a learned list of numbers 1. That row is just a starting point . As the model reads further, the residual stream, its central shared workspace, keeps adding new numbers at each position . In Qwen3-1.7B each position carries a 2,048-number vector . Calling those numbers the model's mind is metaphor; the numbers are real and measurable .

The English sheet

The project began with 900 English question-and-answer pairs spread across 100 everyday topics 1. For each pair the author recorded the final state at every token position, then flattened all of those vectors together with the 151,936 input embeddings into a single principal-component frame . The result shows one orange cloud for every vocabulary token and hundreds of colored lines that trace each conversation forward in text order .

When the camera turns, the English lines collapse almost to a sheet 1. A best-fitting plane contains roughly 99.3 percent of the variation in the 3D projection . That does not mean the model thinks simply; it means the particular 100-topic sample, projected onto three axes, occupies a thin slice of the full 2,048-dimensional space . Connecting successive token states in reading order draws each path .

Language changes the neighborhood

Adding Chinese versions of the same topics produced pink paths that sat visibly next to, but offset from, the blue English ones 1. Both sets use the same fixed global PCA frame; the answers were independently generated by the model rather than copied translations . By separating them, we can see one abstract concept in practice: the model keeps both what is being talked about and the language used for that discussion encoded within its vectors.

German and Croatian were then layered in as additional comparisons 1. The experiment itself cannot untangle whether the shift comes from the writing system, the vocabulary, the grammar, the tokenization, or the way questions are phrased . The author notes that expecting a representation to carry pure meaning, cleanly detached from the words used to express it, may have been the wrong assumption all along . Rotating the embedding table already reveals clusters among the raw vocabulary points before any conversation starts .

Corrections and updates

  • Corrected on 15 November 2026. An earlier version claimed the atlas mapped 1,500+ paths; the source instead says thousands without giving that specific number. An earlier version also omitted tokenization from the list of indistinguishable factors; the source explicitly includes it alongside writing system, vocabulary, wording, and answer quality.

Key facts

  • - Maps thousands of conversation paths through Qwen3-1.7B using residual-stream vectors 1
  • English Q&A across 100 topics collapses onto a plane holding ~99.3% of variation 1
  • Chinese responses appear in a separate neighboring region from English 1
  • Model uses 2,048-number vectors per token position in the residual stream 1
  • Interactive atlas compares English, Chinese, German and Croatian trajectories 1

Sources

  1. What thousands of conversations look like inside a language model — operator0x.com

If this story concerns you or your company, or uses something that belongs to you, ask for a correction or for it to be taken down. How requests work

Keep reading