# Coconut J-lens pilot: registered protocol

Registered September 14, 2026, before lens fitting and synthetic capability evaluation. User authorized planning and execution, including the GPU server. This study uses the existing frozen GSM Coconut checkpoint 22 (GPT-2, six feedback slots). The RLT extension remains conditional on a validated RLT implementation and a useful instrument; it is not part of this first checkpoint experiment.

Question: can a mean-Jacobian lens recover a known intermediate value and support a specific downstream arithmetic intervention, beyond ordinary token directions and natural-state replacement?

**Data and independence.** Generate simple two-stage word problems with answer `(a+b)*c`, a,b in 1..9 and c in 2..9. Shuffle the 648 distinct ordered triples with seed 20260914. Use 256 triples for probe/lens calibration, the next 96 for validation, and the next 192 for final test. The remaining 104 are unused. Calibration uses wording templates 0/1, validation template 2, and test templates 3/4. All sums 2..18 are candidate intermediate labels. Templates and generator are frozen in source before evaluation. Model pretraining overlap is unknown; these are held-out examples for the instrument, not claims about the model's training distribution.

Fit a primary lens on 32 unique GSM training questions, sampled with the same seed, excluding exact validation/test question overlap and limiting questions to 128 tokens without truncation. Fit a second, explicitly task-conditioned lens on the first 32 synthetic calibration cases. Neither fit uses numeric labels. Evaluate natural-domain readouts on 128 GSM validation examples and 256 GSM test examples selected before scores are computed. Gold intermediate labels, where analyzed, come from the first annotated calculation; restrict numeric vocabulary metrics to values represented by one leading-space token, and report coverage.

**Instrument definition.** Sources are the six post-final-normalization vectors immediately before they enter the feedback edge. The target is the post-final-normalization representation predicting the first answer token after the fixed `###` format prefix. Recompute later latent values and retain all downstream cache/embedding dependencies. Hold upstream prefix computation and discrete prefix-token decisions fixed. Average full 768×768 Jacobians separately at each site. This is an answer-targeted adaptation of J-lens, not the reference all-source/all-future-position reduction.

Use float32 eager attention, frozen weights and a batched-output-dimension estimator. Before fitting, compare no-intervention embeddings, answer logits and greedy generations with vendored Coconut; verify batched Jacobian rows against serial rows and finite differences. Numerical tolerance: maximum logit disagreement 2e-4, relative row disagreement 1e-3, and a finite-difference epsilon sweep. Stop to repair instrumentation if these fail. Record all failures and revisions.

Start with 32 prompts per lens. Compare the two disjoint 16-prompt half means at every site. If minimum flattened-matrix cosine is below 0.90, expand that same calibration corpus to 128 prompts, preserving the initial 32. Report the stability result without treating matrix agreement as proof of semantic validity. Use the final configured fit on held-out examples; no test-driven refitting.

**Readout baselines.** Compare direct unembedding, the GSM-fitted J-lens, the task-conditioned J-lens, and a wrong-coordinate J control made with a fixed permutation of source coordinates. Fit a labeled ridge classifier for the intermediate sum using all 256 synthetic calibration cases. Choose its regularization from {0.01, 0.1, 1, 10, 100} using validation only; use a shuffled-calibration-label probe as a negative control. Record per-site candidate accuracy, correct-label rank and full-vocabulary rank where applicable. Separately score the final-answer token. Intermediate and final-answer results must never be merged.

**Capability gate.** Measure greedy, unconstrained baseline answer accuracy before interventions, with a 12-token output limit. Require at least 60% validation accuracy to treat this synthetic task as a suitable reasoning assay. If it falls below that, still report the planned readout evaluation and bounded intervention diagnostics, but mark the reasoning conclusion inconclusive due to task competence. Report causal results both over all test cases and over the subset where the recipient and donor baselines are correct; do not silently filter the denominator.

**Counterfactual design.** Each recipient gets a deterministic donor with a different sum and a different multiplier, using the same wording. The primary predicted result is `donor_sum * recipient_multiplier`. It must differ from both original answers. Donor choice uses arithmetic only, never model outputs. Evaluate changes at one feedback site at a time; later latent values evolve normally.

Arms: no-op; natural donor-vector replacement; J-derived intermediate-token difference; task-J intermediate difference; ordinary embedding intermediate difference; wrong-coordinate J intermediate difference; random direction; labeled-probe intermediate difference; and a J-derived final-answer direction as an explicit answer-steering control. GPT-2 ties input and output embeddings, so embedding/unembedding directions are one baseline. Directional edits have norm {0.03, 0.10, 0.30} times the recipient vector norm. The same random direction is used across strengths for a case/site.

Choose one site/strength per directional arm by validation counterfactual success on the joint-baseline-correct subset (ties: lower strength, then earlier site); choose donor site similarly. Freeze choices before final-test interventions. If fewer than 24 validation pairs are jointly correct, select on all validation cases and flag the inadequate capability stratum. Record baseline-answer retention, hybrid-answer success, donor-answer copying, other answers, valid prefix rate and next-token KL. Test a wrong-target control by permuting donor sums across cases; interpret template transfer and ordinary-direction controls jointly.

For selected sites, run an additional descriptive readout after the edited slot and at later slots to distinguish a transient change from continued propagation. This alone does not distinguish all cache and attention pathways. Do not claim the feedback vector is the model's complete memory.

**Analysis and stopping.** Report paired bootstrap 95% intervals over recipient test cases (2,000 draws, seed 20260914) for differences versus baseline and the ordinary embedding arm. This is a pilot with one trained checkpoint, exploratory arm/site validation and no model-training replication. Claim evidence for controlled intermediate computation only if the task capability gate passes, held-out intermediate readout beats shuffled control, and the selected intermediate edit raises the predicted hybrid answer beyond no-op and embedding controls while not predominantly copying the donor answer. A successful final-answer steering control alone does not pass this gate. A failed J-lens alongside a successful supervised probe or donor intervention is an instrument boundary, not evidence that the model has no intermediate state.

Initial operational budget: existing checkpoint inference and lens fitting only, up to four GPU-hours across the two idle cards. Measure throughput and revise the execution batch size if needed, preserving the estimator. Stop after the registered pilot and write a complete result even if all semantic gates fail. Model retraining, a broad architecture sweep and RLT training require a subsequent concrete protocol; the user's authorization is sufficient to continue useful implementation work, but this pilot should first determine its direction.

Primary references: [J-lens implementation](https://github.com/anthropics/jacobian-lens), [looped-model adaptation](https://arxiv.org/abs/2609.01924), [execution validation and completed results](methods.html).