Before a model answers, it thinks in concepts

I gave Qwen3.5-4B a riddle: "The currency used in the country shaped like a boot is called the ___." The word Italy is nowhere in that sentence. But twelve layers into the network, on the token "boot," the model is already holding "Italy." I can read the probability off it, and it's a bit over a quarter. By the time the model writes anything, the riddle is solved. The word just hasn't come out yet.

That gap, between reading the prompt and choosing the next word, is where a language model does the part we'd call thinking, and it's normally closed to inspection. Anthropic's global-workspace paper pried it open and released enough of the tooling to run on a model that fits on a laptop. I wrapped that tooling in an instrument I could aim at any prompt, called J-Scope. This is what it shows.

The lens is a change of basis, not a model

A transformer doesn't jump from prompt to answer in one step. It carries a running representation, the residual stream, up through its layers: one stack of numbers per token, refined at each level. Those numbers aren't words. At each layer every token is a 2,560-dimensional vector, and on its own it's unreadable.

A lens is a translator from that vector back into vocabulary. The crude version, the logit lens, hands a mid-network vector straight to the model's output layer and asks what word it predicts. It mostly returns noise, because the middle of the network isn't formatted for the output layer yet. The Jacobian lens is more careful: for each layer it fits a matrix that rotates the vector into the frame the output layer reads cleanly, and then reads it. Anthropic fit those matrices; Neuronpedia published a set for Qwen3.5-4B, estimated from a thousand Wikipedia passages.

The word "fit" makes it sound like a model. It isn't. The lens doesn't run the network or learn anything; it's a fixed rotation. Its vocabulary isn't the thousand passages it was calibrated on, it's the model's entire 248,320-token output space, and the passages only pin down the rotation. So when the lens reports "Italy" at layer 12, it isn't inferring Italy from context. It's reading an axis the model itself computed, in the model's own words. A window, not a mind.

Watching a concept form

Once you can read that axis, you can watch a concept arrive. Tracking "Italy" on the boot riddle, the probability sits at the floor through the early layers, lifts off around layer 8, peaks near 0.27 at layer 12 on the "boot" token, then falls away as the model hands the resolved fact forward. Climb mid-stack, then hand off: that shape is what a concept being worked out looks like. The model settles "boot-shaped country is Italy" in the middle of the network and moves on.

Not every glow is a thought, and the tool wears that caveat on its face. Track a concept on a word like "in" or "country" and the grid lights up faintly across many countries at once, because a preposition holds the type of thing the slot expects and the lens sees a spread of candidates. That's grammar, not retrieval. A concept that's genuinely been resolved spikes on one token and leaves its rivals at zero. Reading the workspace is mostly learning to tell those two apart, which is why the tool shows the whole grid instead of the one number you were hoping for.

Writing a word the prompt never said

Reading is half of it. If the lens gives you the axis a word lives on, you can add along that axis and write a concept into the stream that the prompt never mentioned. I took the direction for "France," added it at the single token the answer hangs on in "The capital of Kevin's home country is the city of ___," and the model answered "Paris." The probability of Paris rose from 0.019 to 0.136 and became the top token. France appears nowhere in the prompt, and the probability of the model saying "France" itself stayed at zero. It didn't echo the word I pushed in. It used it.

That gap between using and echoing is the whole game, because the lazy version fails loudly. Smear the same France direction across every token instead of aiming it, and the model stops reasoning and starts chanting: the probability of "France" itself climbs to 0.98, Paris falls back to zero, and the output distribution moves a KL divergence of nine from where it started. You've handed the model a tic, not a thought. The workspace has a write interface, but its calling convention is narrow: the right token, at an amplitude near what the concept carries when it arrives honestly in text. Push harder than that and the reading circuits treat the signal as damage. The tool marks the natural amplitude on the slider, so you can see how little force the real thing uses.

Erasing, and the shadow of a concept

If you can add a concept you can also subtract one. Erasing is a causal test rather than a readout: project "Italy" out of the "boot" column and the model's currency answer, "euro," collapses.

The mechanism rests on one geometric idea, a concept's shadow. A concept is a single direction, a unit vector, in the model's 2,560-dimensional space. How much of it is present at a given cell is how far the state leans along that direction, which is the dot product of the two, and that projection is the shadow. Reading a concept is measuring its shadow; the lens probability is a dressed-up version of that one number.

Erasing subtracts the shadow and nothing else. Take the state's component along the concept's direction and subtract exactly that: what remains has zero shadow on the concept and is untouched on every other axis. It's a rank-1 edit, one direction moved and the other 2,559 left where they were. Erasing Italy from the "boot" cell at layer 11 strips about 16% of that state's length and turns it by roughly nine degrees, a cosine similarity of 0.987 to the original. Sixteen percent is a lot to rest on one direction; a random axis would hold about two. Nothing is zeroed or flattened. The state is rotated a hair, along the single axis that spells Italy.

One honest asterisk comes with the readout. The lens reads a concept through the very direction erasing removes, so the shadow dropping to zero after an erase is partly the tool grading its own homework. Trust the next-token probabilities, not the grid, for whether the concept is really gone.

Trading one concept for another

That one move is the whole instrument. Adding pushes the shadow up instead of down, and the mixer sets it to any value you choose. Pick the cells where Italy lives and you get one slider per concept, each dialing that concept's shadow in multiples of its natural strength: zero is an erase, one is as loud as the word arriving in real text, anything between is a partial dose. Drag Italy to zero and Japan to one, and the answer walks. "euro" drops from 0.143 to zero, "yen" climbs from zero to 0.283, and the top token flips from euro to yen. Same prompt, every word unchanged, one vector turned at a handful of cells, and the model now believes the boot-shaped country spends yen.

Try it yourself

Everything above is a frozen slice of the real tool, baked so it runs in your browser with no model behind it. Pick a preset, watch the grid light up, read the concept trajectory, then drag the mixer and take the currency answer from euro to yen yourself. Controls that would feed the model a new input — editing the prompt, the injection bench, the α sweep — are marked “needs model” and switched off; to use those, clone the repo and run it on a laptop.

What it is, and what it isn't

None of this says anything about what the model "really" understands, and it's deliberately small: one 4B model, one lens estimated from a thousand passages, completion-style prompts. It carries no claim about Claude, about larger models, or about the consciousness questions the source paper is careful to bracket. What it does give you is a second way to relate to a model. The prompt is one interface to the workspace, the slow and indirect one, filtered through words. The lens is another, reading and writing the same concepts the prompt does, only without the words.

The reading side surprised me more than the writing. That a model can be steered isn't news. That you can watch it silently solve a riddle, name the concept it's holding, and point at the exact layer where it worked the answer out, changed how I read what a model says. When it finally answers, it's reporting a conclusion it reached a dozen layers earlier.

J-Scope runs on a Mac or a single GPU, and it ships with the frozen build embedded above. The code, the demo, and the 72-second tour are on GitHub. The harder result this grew out of, feeding a model facts through its workspace instead of its prompt, is the companion piece.