# What Coconut and a small RLT carry through their loops

Completed September 14, 2026. [Read the article](https://www.phenx.ai/research/what-travels-around-the-loop). [Registered Coconut protocol](coconut-jlens-protocol-2026-09-14.md), [mechanics scope and training amendment](mechanics-scope-2026-09-14.md), [aggregate results](supporting-data.json), and [figure data](chart-data.json).

This public supplement includes the report, protocols, aggregate intervention results and figure data. Full model checkpoints, fitted tensors and per-problem run archives are retained separately and are not included in this download.

**We found staged calculation signals and a causal information pathway in Coconut, and an editable running register in the small RLT.** These findings come from different tasks and training histories. They explain behavior in the tested models; they are not a controlled comparison showing an inherent architectural difference.

The study asked where useful information lives, when it changes, and what happens when that information disagrees with another channel. It used the existing frozen GSM Coconut checkpoint, two newly trained small RLT seeds, and a GRU training control. Accuracy served as a capability check and an intervention outcome.

**Coconut: the latent sequence can determine the answer despite a conflicting question.** We generated problems of the form `(a+b)×c`, with known intermediate sum and final product. Calibration, validation and test triples were disjoint, and test wording differed from calibration wording. The model answered 191/192 test problems correctly; 185 recipient/donor pairs had both baseline answers correct. Donors changed both the sum and multiplier, so the donor answer differed from the intended hybrid calculation `donor_sum × recipient_multiplier`.

| Channel changed | Recipient answer retained | Donor answer produced | Hybrid calculation produced |
|---|---:|---:|---:|
| Identity control | 185/185 | 0/185 | 0/185 |
| All six feedback vectors | 1/185 | **158/185 (85.4%)** | 14/185 |
| Question KV used during latent updates | 2/185 | **133/185 (71.9%)** | 20/185 |
| Question embeddings during answer replay | **158/185 (85.4%)** | 2/185 | 0/185 |
| Feedback vectors and answer-replay question | 0/185 | 185/185 | 0/185 |

Other outputs account for the remaining cases. Unconditional results over all 192 pairs and binomial intervals are retained in the analysis JSON.

![Coconut channel conflict outcomes](methods-figures/coconut-channel-conflicts.png)

One actual test pair makes the intervention concrete. The recipient asked about `(4+9)×8 = 104`; the donor asked about `(2+7)×6 = 54`. Supplying the donor's six latent vectors while keeping the recipient question produced **54**. Keeping the recipient latents while switching the answer-stage question to the donor produced **104**. The hybrid result would have been **72**.

This localizes an important information path: the feedback sequence carries information that the answer computation uses. It does **not** establish that it stores only a completed answer. The vectors could carry intermediate calculations, a representation of the problem, or a mixture. The question-KV intervention also shows that later latent updates can acquire different problem information after the initial feedback vector has been formed. It does not establish that attention is dispensable.

**Coconut also shows a temporal trace of its reference calculations.** The registered natural-domain readout found the first annotated intermediate value near the top of the vocabulary at an early slot. That prompted a clearly labelled post-test audit of the first three reference calculations. We used direct unembedding, without fitting a new model, and compared each value with 32 wrong-value draws from the same step's observed label pool. Eligible values were single tokens, absent from the question, and different from the final answer.

| Reference calculation | Eligible test problems | Slot with lowest median rank | Correct-value median rank | Wrong-value median rank at that slot |
|---|---:|---:|---:|---:|
| First | 210 | **2** | **3** | 282.5 |
| Second | 143 | **4** | **18** | 263 |
| Third | 70 | **6** | **21** | 181 |

Lower rank means more readable. The same 2/4/6 pattern appears on the 51 problems where all three intermediate values meet the eligibility rule: median ranks 3, 12 and 25 at those slots. This controls for the different problem subsets in the main table.

![Native Coconut calculation trace](methods-figures/coconut-native-step-trace.png)

This pattern is consistent with the checkpoint's training recipe: **two latent slots per removed text reasoning step**. It gives evidence of staged calculation signals, rather than a generic increase in confidence across repetitions. Slots here refer to vectors immediately before they enter the feedback edge; they are not labels for completed transformer passes. The audit is observational and exploratory. It does not prove that each value occupies a separately editable register or that every problem follows this sequence correctly.

**J-lens did not provide a complete account of those calculations.** We fitted separate 768×768 mean Jacobians at six feedback sites, using a target representation just before the first answer token after `###`. Both the native-GSM and task-conditioned fits expanded from 32 to 128 calibration prompts under the registered stability rule. Their half-sample matrices still disagreed substantially, especially at early sites. This is an answer-targeted adaptation, not the original all-position/all-future-target estimator.

On the controlled arithmetic test, the strongest intermediate-sum J readout reached 15.6% candidate accuracy across 17 possible sums; the labelled state probe reached 38.5% at slot 6. At slot 2, direct unembedding achieved 29.2%, versus 15.6% for the GSM J-lens. The native GSM audit likewise favored direct readout for the early reference value. Vocabulary rank and candidate accuracy are different measures; neither proves causal use.

![Coconut intermediate readout comparison](methods-figures/coconut-intermediate-readouts.png)

After choosing sites and strengths using validation only, the GSM J-derived intermediate edit produced the predicted hybrid answer in **6/185** jointly correct test pairs. The task-conditioned J edit and ordinary embedding edit produced **0/185**; an explicit final-answer J edit produced **4/185**. The GSM-J versus embedding difference was 3.24 percentage points, with a paired bootstrap 95% interval of 1.08–5.95 points. This small effect is exploratory across several arms, not a dependable arithmetic editing interface. Of 25 answers changed by that J edit, only six were the predicted hybrid. Its low success rate and the answer-steering control prevent a strong claim that we isolated the sum operation.

The useful conclusion is that weak J readouts do not establish empty latent state: the natural-state swaps and direct reference-value readouts provide independent evidence of useful content. Conversely, the readable calculation trace is not sufficient to establish a causal step-by-step algorithm.

**Small RLT: the recurrent output behaves as a writable running register.** We independently implemented a 133,248-parameter instance with two shared encoder/decoder cores, width 64, four attention heads, separate normalization, causal encoder memory, decoder cross-attention, a four-position local KV window, and output feedback across every token. Full gradients pass through output state, decoder KV and encoder memory. This is a mechanism demonstration, not the author's released checkpoint or a reproduction of the preliminary benchmark.

The task has a register initially equal to zero. Each token adds 1–4 modulo five or negates the register modulo five. Training supervises only the final value. The first RLT and GRU runs, trained directly on lengths 4–16, stayed near chance. We preserved those failures and registered a separate short-to-long curriculum. Both RLT seeds then passed the capability gate; the curriculum-matched GRU also learned the task. Curriculum was therefore a material part of the successful recipe.

The registered validation stopping rule ended seed 0 at step 2,700, after reaching training lengths up to 16, and seed 1 at step 1,500, while its curriculum maximum was still 8. The second seed's longer-sequence results therefore also test length extrapolation; differences between these runs are not attributable to initialization alone.

On independently drawn test programs, we replaced state at the midpoint with a natural donor state representing a different register value, then continued the original suffix. Because every operation is invertible, the predicted new final answer differs from the original. The following results concern length 16 and the subset where both original and donor-prefix continuations were correctly solved.

| Midpoint intervention | Seed 0: predicted new answer | Seed 1: predicted new answer |
|---|---:|---:|
| Output state only | **512/512 (100%)** | **500/501 (99.8%)** |
| Decoder KV only | 0/512 | 0/501 |
| Output state + decoder KV | 512/512 | 500/501 |
| Encoder memory only | 0/512 | 1/501 |
| All future encoder information | 0/512 | 1/501 |
| All state and encoder channels | 512/512 | 501/501 |

![RLT causal pathways](methods-figures/rlt-causal-paths.png)

“All encoder information” includes recomputing future encoded-token representations on the donor prefix, not just swapping cross-attention keys. Thus the result is not explained by accidentally leaving that second encoder route unchanged. The current operations in the suffix remain the recipient's operations in every arm.

A concrete trace started with register **3** after operation 8. Replacing only the recurrent vector with a donor vector representing **0** changed the decoded subsequent register sequence to the sequence predicted from zero, and changed the final answer from **2** to **4**. The suffix and encoder context stayed fixed. This is stronger than biasing an answer token: the intermediate state followed the predicted alternative computation through the remaining operations.

![RLT state propagation after a swap](methods-figures/rlt-state-propagation.png)

Linear probes independently recovered the register from the recurrent output with near-perfect accuracy at length 16, while encoder-state probes were much weaker. The causal vector replacement shows that this readable state is also used. At length 32, output-state swaps followed the predicted alternative in 100% and 98.7% of jointly correct pairs. The second seed's unconditional success was lower, 96.3%, alongside 96.5% baseline accuracy; the conditional figure must not hide those failures.

For this trained task, the recurrent output is the dominant route carrying the register between tokens. The decoder applies the next token's operation to that state. The cache and encoder can still support computation; the swaps establish that they do not normally overwrite the substituted register with the old prefix's register in these tests. This does not imply that a language-trained RLT would use its channels in the same proportions.

**Validation and scope.** Coconut reference generations matched, feedback embeddings matched exactly, maximum tested answer-logit disagreement was 2.29e-5, and batched/serial Jacobian relative disagreement was 5.61e-7. Finer finite differences converged after larger perturbations exposed nonlinearity. RLT prefix consistency, all three gradient paths, identity swaps, and full donor-prefix replay were checked. Entire-state/encoder swaps reproduced the corresponding natural mixed-program logits within 2e-4 in the completed runs. Data hashes, disjoint synthetic triples, counterfactual answer separation and paired denominators were verified. Results include all failures and all test arms.

The study completed 12,768 Coconut validation interventions, 1,920 held-out interventions, 960 channel-conflict trials, and 24,576 RLT intervention trials over two seeds and three lengths. The two-model training/task mismatch prevents attributing their different observed mechanisms to architecture alone. The RLT mechanism uses known-state probes, not a fitted RLT J-lens. Coconut's scalar-step audit remains exploratory. No language-scale RLT, universal reasoning claim, or complete neuron-level circuit has been established.

The next discriminating experiment is a matched task requiring both a carried register and retrieval from encoder memory. For Coconut, a separate confirmation should target the observed 2/4/6 calculation trace with matched natural intermediate-state interventions. Those extensions should test whether the identified information routes remain stable, rather than assume the current traces explain all latent reasoning.

Implementation references: [Coconut](https://github.com/facebookresearch/coconut), [RLT proposal](https://github.com/yifanzhang-pro/recurrent-looped-tranformer), [J-lens](https://github.com/anthropics/jacobian-lens), [prior looped-model J-lens study](https://arxiv.org/abs/2609.01924).