# Mechanics objective and RLT extension

User clarification during protocol implementation: the objective is to explain how Coconut and RLT use their loops, rather than demonstrate another accuracy improvement. No lenses or synthetic validation scores had been fitted/collected at this clarification. The registered Coconut data and counterfactual controls remain in place.

We will distinguish three hypotheses: an intermediate is carried in the feedback vector; it travels through decoder attention/cache; or it is reconstructed from access to the original input. Readability alone does not distinguish these. Interventions must predict a specific continuation, and recovery must be followed over subsequent transitions. Final-answer accuracy is a capability check and intervention outcome, not the primary architectural claim.

Before causal evaluation, add a Coconut channel-conflict diagnostic: swap all six feedback vectors; swap the question/start KV used during latent updates while retaining the recipient's initial feedback; swap only the question embeddings during answer replay; and swap both feedback vectors and answer-replay context. Use matched-length recipient/donor pairs and identical-context controls. Record original, donor and hybrid answers separately. These conflicting-channel interventions test where the answer comes from; they do not by themselves isolate an arithmetic variable. Coconut's generation replays all accumulated embeddings after the first output token, so that answer-stage access must be measured explicitly.

RLT has no author checkpoint in the reviewed release. Implement a small, explicitly independent instance with a parallel causal encoder, encoder/decoder sharing of compatible attention and feed-forward weights, decoder cross-attention to causal encoder memory, a sliding local decoder KV cache, and output feedback between all input tokens. Use full backpropagation through all state paths. Validate batched encoder versus incremental replay and prefix-state consistency. The small implementation and task are a mechanism demonstration, not a reproduction of the author's numerical results or a direct scale-matched comparison with pretrained Coconut.

Train on a known-state program task. A single register starts at zero; tokens add 1..4 modulo 5 or negate the register modulo 5. Train only the final state label, so intermediate states are not directly supervised. Use random program lengths 4..16, then inspect held-out programs at lengths 8, 16 and 32. Keep exact state labels at every step for external probes. Include a GRU capability baseline. Initially use two RLT layers, width 64, four attention heads, decoder window 4, and a learned concatenation merge. Training budget: at most 3,000 optimizer steps per model, stopping after validation accuracy exceeds 98% twice consecutively. Assess all results only for the trained seed; add a second seed if it is needed to resolve a mechanistic positive within the four-GPU-hour pilot budget.

At a midpoint in a held-out recipient program, take a donor program of the same prefix length but a different true register state. Continue the recipient suffix after swapping (a) output state only, (b) decoder KV only, (c) both, and (d) encoder memory only. Score the exact counterfactual produced by the donor register followed by the recipient suffix, separately from the recipient's original answer. Keep donor states natural and match positions. Apply sustained state/cache changes only as separately labelled interventions. Track task-state probe readouts along the suffix; use identity swaps and matched-norm random replacements as controls.

Also test replacement of all future encoder information by recomputing the encoder on donor-prefix plus recipient-suffix tokens. This includes the current encoded-token representation as well as global cross-attention memory; a memory-only swap leaves the current encoded-token route intact. Keep the decoder's recipient history fixed at the intervention boundary for this arm.

This task's operations are bijections, so a changed midpoint register predicts a distinct final answer. Encoder-memory swapping can alter several information channels, and donor-prefix caches contain their own computations; report path effects without asserting that a particular scalar coordinate is the whole algorithm. A final-state head can also read intermediate states only if validated; use a separately trained linear probe with a held-out calibration partition as the main diagnostic.

Stopping interpretation: if the model does not reach adequate task capability, describe the implementation/capability limitation and do not infer a universal mechanism. If a vector swap is repaired, inspect whether recovery tracks preserved context/cache. A causal change that follows the donor-state counterfactual provides stronger evidence of transported state than a readable label alone. Do not infer that tiny RLT and pretrained Coconut share a mechanism just because both have nonzero Jacobians.

Capability-recovery amendment, made before any RLT mechanics evaluation: the first RLT seed stayed at chance after its 3,000-step final-only run (18.9% validation at length 16; five classes). Preserve that run. Evaluate the originally specified GRU baseline, and run a separate RLT curriculum with five equal phases whose maximum lengths are 2, 4, 8, 12 and 16; sample lengths uniformly from 1 to the phase maximum. Keep the same final-state-only loss, model, optimizer, 3,000-step cap and validation rule. No intermediate labels enter training. If successful, replicate with seed 1 before attributing a mechanism to the trained variant. Report curriculum findings under their own checkpoint names. This is a training-protocol revision, not confirmation of the initial recipe.