# Structured-bias paper: ICLR reassessment Date: 2026-08-06. This assessment deliberately gives no novelty credit to the clean KP scaling results. It asks whether the Harnett-specific operation alone can carry an ICLR paper: ```text r = a - P(h) Delta W = eta * r * eligibility ``` ## Verdict The topic is strong enough for ICLR, but the current evidence is not. With the clean scaling story removed, the present package is approximately **5/10 (borderline reject)**. A coherent structured-bias paper can reach **6/10** if it establishes a natural source of bias, predicts its learning dynamics, and shows that the unchanged two-line SDIL rule removes it more effectively than bias-specific alternatives at matched observation cost. It can reach a solid **7/10** with a measured physical bias source or a second substantially different system and a useful-scale confirmation. This should not be sold as another general alternative to backpropagation. The paper-level question is: > When a local teaching channel mixes task instruction with a systematic, > state-predictable component, what failure does that component cause, and can > a locally learned neutral prediction remove it without removing instruction? ## What the repository currently proves Positive evidence: - In the controlled MNIST intervention, raw and norm-matched raw signals fall to about 10.3% at the strongest soma-predictable traffic, while innovation retains 97.35% over five seeds. Norm matching rules out a pure scale effect. - In five paired CIFAR-10 ResNet-20 seeds, dynamic neutral projection remains within 0.2 points of clean KP under a four-times-RMS traffic intervention. The same mechanism remains finite through ResNet-56. - The implementation records instruction-off observations, checks that no task loss or clean teaching signal enters the neutral fit, and retains failed gates and untouched endpoints. Evidence that limits the claim: - The strongest controlled bias is generated from the same somatic statistic made available to the predictor. Recovery is therefore a clean necessity test, but not evidence that important real bias normally has this form. - The five-seed endogenous MNIST gate failed one of four traffic realizations. The FashionMNIST recovery found large gains but failed the no-traffic-gap requirement for top-down traffic. The necessary shared-context task produced only a 0.195-point innovation gain. These failures show that high predictor R-squared is not sufficient to identify instruction. - A fixed predictor did not stabilize the standard ResNet intervention. The successful variant needs an instruction-off observation on every example and a fast affine projection. This is useful evidence, but it weakens the claim that the original two-line rule alone solves the problem. - The clean ResNet-20/32/56 scaling belongs to reciprocal Kolen--Pollack credit. It establishes that the substrate scales, not that residualization causes scaling. - The current theory proves a standard conditional-expectation projection identity and a generic smooth-loss descent condition. It does not yet predict a bias-induced error floor, limit cycle, instability, or depth law. ## Prior-work pressure The novelty boundary is narrow: - Laborieux et al. show that finite one-sided nudging gives EP a first-order estimator bias. Randomly reversing beta cancels that term in expectation at two-phase cost, although their CIFAR-10 runs have high variance; centered EP is stable but adds a phase. - Holomorphic EP obtains exact finite-amplitude gradients through an oscillating complex nudge and reports ImageNet32 performance comparable to BP. - Jacobian homeostasis separately analyzes finite-nudge and weight-asymmetry bias and repairs the latter on ImageNet32. This is direct precedent that a bias-mechanism paper can be accepted, but it sets a high bar for theory and scale. - Dillavou et al. measure deterministic update bias in a physical contrastive resistor network, connect it to error floors, null-space drift and limit cycles, and introduce overclamping. Any physical contrastive experiment must compare to overclamping rather than presenting bias robustness as unclaimed territory. - Conditional-mean subtraction itself is regression/control-variate machinery. The contribution cannot be the algebraic subtraction alone; it must be the locally identifiable bias model, its learning consequence, and causal evidence that the removed component is nuisance rather than instruction. Primary sources: - Harnett/Francioni et al., Nature 2026: https://www.nature.com/articles/s41586-026-10190-7 - Finite-nudge EP bias, Frontiers 2021: https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2021.633674/full - Holomorphic EP, NeurIPS 2022: https://proceedings.neurips.cc/paper_files/paper/2022/hash/545a114e655f9d25ba0d56ea9a01fc6e-Abstract-Conference.html - Jacobian homeostasis, ICLR 2024: https://proceedings.iclr.cc/paper_files/paper/2024/hash/6a55f024db3f771194bdadc8f3a35381-Abstract-Conference.html - Physical imperfection and overclamping, arXiv v2 2026: https://arxiv.org/html/2505.22887 ## The minimum complete mechanism Use one explicit model throughout: ```text a_t = s_t + b(z_t) + epsilon_t r_t = a_t - P(z_t) ``` Here `s_t` is task instruction, `b(z_t)` is deterministic or slowly drifting bias predictable from locally available neutral state `z_t`, and `epsilon_t` is unpredictable zero-mean noise. The identification assumptions must be stated, tested and violated deliberately: 1. the neutral observation contains no instruction; 2. the neutral and task periods share the same bias law; 3. the predictor uses information available before instruction; 4. the predictor class can represent enough of the bias; 5. predictor learning is slow or cross-fitted enough not to absorb instruction. The theory must go beyond `P` minimizing mean-squared prediction error. At minimum it should derive, for a tractable linear or locally linear system: - the raw learner's displaced fixed point, error floor or limit cycle as a function of bias, curvature and learning rate; - how coherent bias accumulates with steps, depth or flat directions, compared with zero-mean noise of the same RMS; - an optimization/stability bound in terms of residual predictor error and neutral-to-task distribution shift; - a failure theorem or counterexample showing when subtraction removes part of the instruction. These statements generate quantitative curves before the large experiment. Fitting a line after observing the endpoint is not sufficient. ## Best experimental systems ### 1. Physical coupled-learning imperfection -- primary candidate The closest natural match is the resistor-network setting of Dillavou et al. Its per-edge measurement/application bias persists when the correct contrastive update shrinks, producing drift along flat directions and multi-task limit cycles. An instruction-off or zero-error observation gives a natural local baseline. The unchanged SDIL predictor should be tested against: raw coupled learning, overclamping, a per-edge constant-offset calibration, random zero-mean noise matched in RMS, and an oracle bias subtraction. The decisive result is not only lower final error. SDIL must correctly predict and reduce cycle span, null-space drift and forgetting using the bias measured before task training. Real experimental traces or a bias distribution measured from hardware would make this much stronger than a synthetic injection. ### 2. Deep contrastive network with hardware-shaped differential offsets Use an author DCHN or Dual Propagation implementation and introduce offsets at the actual local free/nudged measurement boundary, with distributions fixed from a physical system rather than chosen as `P(h)` by construction. The task ladder should include FashionMNIST, SVHN and CIFAR-10, with depth or relaxation length varied. Compare the same two-line SDIL rule to raw learning, ordinary constant calibration, overclamping where applicable, and clean/oracle paths. This supplies useful-scale evidence, but by itself it remains simulation. A five-seed artificial differential-bias result in Dual Propagation is supporting evidence, not the central natural-bias claim. ### 3. Finite-nudge EP bias -- diagnostic, not primary Random-sign beta is the correct same-cost baseline because it cancels the first-order term in expectation. Centered EP is the lower-variance expensive reference. Unless the original SDIL rule beats random-sign EP at matched phase and observation cost without calibration stages or extra mechanisms, this setting should be reported as a boundary where randomization already solves the mean bias. It should not drive method design. Weight-asymmetry EP is also unattractive as the first target because Jacobian homeostasis is a specialized, accepted and large-scale solution. ## Causal experiment matrix One compact matrix can distinguish the claimed mechanism: | factor | required values | purpose | |---|---|---| | corruption | none, zero-mean noise, fixed bias, state-dependent bias, slow drift | separates noise robustness from bias removal | | predictability | same RMS with low to high neutral R-squared | tests the proposed sufficient variable | | signal rule | raw, norm-matched raw, static calibration, SDIL, oracle | isolates subtraction from scale and ordinary calibration | | neutral validity | matched, shifted, instruction leakage | tests identification and failure boundaries | | scale | task/depth/relaxation or number of alternating tasks | tests coherent accumulation | The primary plots should be predetermined theory-to-data plots: error floor or cycle span versus bias; degradation versus scale at fixed bias; recovery versus neutral predictability; and accuracy/error versus measured observations or wall time. Endpoint accuracy alone is not mechanism evidence. ## Score gates **Current: 5/10.** Strong auditing and a large controlled effect, but the natural importance, unique mechanism and theory are incomplete. **Minimal accept, 6/10:** - one natural or independently measured bias source; - a theorem predicting the observed failure and residual dependence; - unchanged two-line SDIL, without a new calibration stage; - matched static-calibration and bias-specific baselines; - three or more seeds on one useful-scale task plus causal negative controls; - explicit neutral-shift and instruction-leakage failures. **Solid accept, 7/10:** add either real physical-network evidence or a second distinct learning system, and show the same frozen mechanism predicts both. **Stop the ICLR push** if any of the following occurs: - gains appear only when bias is constructed directly from the predictor's input; - a constant offset/EMA calibration, overclamping, or random-sign beta matches SDIL at lower cost; - useful-scale recovery requires per-example oracle-like neutral projection or another compound algorithm rather than the stated two-line rule; - measured bias does not worsen with any meaningful scale variable; - neutral observations absorb instruction under realistic distribution shift. ## Immediate decision Continue only as a short, gated reassessment. First reproduce the physical bias dynamics in the coupled-learning system and test the unmodified SDIL predictor against overclamping and constant calibration. In parallel, finish the already-frozen Dual Propagation differential-bias confirmation as supporting evidence. Do not spend full CIFAR-scale compute until the simple rule wins this mechanism gate. A failed gate should close the ICLR structured- bias paper rather than trigger another compound recovery method.