# BabyAI shared-feedback protocol ## Question The clean scaling experiments show that reciprocal Kolen--Pollack (KP) feedback can train large networks, but do not show that SDIL adds anything to KP. This experiment asks whether SDIL is useful when an ordinary top-down signal is required during inference and the same physical pathway also carries the teaching signal. BabyAI supplies this structure without an injected nuisance. The visual observation is the basal input, while the language mission is required to choose the correct object and enters each hidden population through a fixed apical projection. During imitation learning, the measured apical signal is ```text a_l = C_l m + t_l, ``` where `C_l m` is the mission field used during inference and `t_l` is the reciprocal KP teaching field. Raw shared-path learning uses `a_l`. SDIL fits a per-cell affine prediction of `C_l m` from the local hidden activity and uses `a_l - P_l(h_l)`. This is a fixed-distribution, randomly interleaved grounded-language task. It is not a continual-learning experiment. The endpoint tests signal separation and task learning, not catastrophic forgetting. ## Data and task The first task is the official `BabyAI-GoToObjS6-v1` environment from MiniGrid 3.1.0. Expert demonstrations are generated by the official `BabyAIBot`. Every recorded trajectory must end with positive reward; failed expert episodes abort data generation rather than being silently discarded. The basal observation is the standard partial `7 x 7 x 3` symbolic image plus agent direction. Each of the object, color, and state channels is one-hot encoded. The mission is a normalized bag of training-vocabulary tokens and is not included in the basal input. Validation missions use the training vocabulary, with an explicit unknown token retained for auditability. Development data contain 20,000 expert episodes for training and 2,000 disjoint expert episodes for validation. A separate set of 500 validation seeds is used for closed-loop policy rollouts. No test seeds are generated or evaluated until the method configuration and mechanism comparison are frozen. ## Model and local updates The policy is a tanh MLP with a fixed mission projection into every hidden population and a seven-action linear readout: ```text u_l = W_l h_(l-1) + b_l + C_l m h_l = tanh(u_l). ``` `C_l` is fixed after initialization. Forward weights and independently stored reciprocal weights use the same modified-KP momentum, decay, and local correlation rule used in the existing shared-feedback implementation. The reciprocal tensor is updated from the same locally available activity product; it is never copied from the forward tensor. SDIL predictor fitting receives only the current hidden activity and ordinary mission field, never the expert action, loss, output error, KP teaching field, or downstream weight. ## B0 clean selector B0 exists only to choose a model that the task and clean learning rules can train. It runs BP and clean KP, never raw shared KP or SDIL. The fixed grid is ```text hidden layers: 2, 4 hidden width: 256 learning rate: 0.01, 0.03 context gain: 1.0 epochs: 15 batch size: 256 model/data seed: 4101 ``` All candidates share demonstrations, minibatch order, initialization within a model shape, and rollout seeds. A candidate is eligible only if BP and clean KP each reach at least 80% validation rollout success and removing the mission field reduces BP rollout success by at least 20 points. Among eligible candidates, choose the highest clean-KP rollout success; ties are resolved by action accuracy, then fewer layers, then the smaller learning rate. If no candidate is eligible, B0 fails and the shared-path endpoint is not run. ## B1 mechanism endpoint The selected depth and learning rate are frozen. B1 trains for 40 epochs on model and minibatch-order seeds `4101`, `4102`, and `4103`, with identical data and minibatch orders across conditions: 1. `bp`: exact backpropagation reference with the same fixed mission pathway. 2. `clean_kp`: instruction-only reciprocal KP; this receives a separate clean teaching wire and is the local-learning upper bound. 3. `raw_shared`: the local teaching rule directly uses `C_l m + t_l`. 4. `sdil`: a per-cell neutral affine predictor is fitted on 4,096 training observations before each epoch and subtracts the predicted mission field. The predictor's extra forward observations, wall time, and arithmetic are reported. The main metrics are validation closed-loop mission success, validation expert-action accuracy, mission-lesion success, cross-entropy, predictor explained variance, and the residual mission-field RMS. Accuracy and mission success, not residual size, decide the result. B1 supports the mechanism if clean KP is successful, mission removal damages the task, raw shared KP is worse than clean KP, and SDIL recovers a substantial part of that downstream success gap consistently across seeds. No fixed five-point threshold is imposed before observing the natural effect size. Norm-matched raw feedback is added only after a positive raw--SDIL difference, because it diagnoses that difference but cannot create it. Passing B1 opens `PickupLoc` and `PutNextLocalS6N4` with the same method and selection rule, followed by depth scaling. Failure is retained as evidence that task-required shared feedback alone is insufficient to make residualization useful in this setting. ## B0 outcome All eight clean-selector runs were finite. BP and clean KP both reached `97.8%` validation rollout success and `99.74%` expert-action accuracy in every candidate. The two-layer, `0.01` learning-rate candidate was selected by the frozen tie rule: its BP mission lesion reduced rollout success from `97.8%` to `70.4%`, while it tied for the best clean-KP success and action accuracy and used the fewest layers and smaller learning rate. The selector read no raw, SDIL, or test result. This outcome opens B1 without establishing an SDIL advantage. ## B1 outcome: diagnostic only The task-structure audit found exactly one ball, box, or key in each of 256 generated `GoToObjS6` episodes. That sole object is always the target, so the mission does not select among alternatives. The original claim that language is required by this task was therefore false; mission lesion is only an input ablation in this environment. The three-seed downstream result is nevertheless useful as an implementation diagnostic. Mean rollout success was `97.8%` for BP, `97.8%` for clean KP, `83.93%` for raw shared KP, and `95.87%` for SDIL. The paired SDIL-minus-raw rollout differences were `+30.6`, `+5.6`, and `-0.4` points; expert-action differences were `+12.22`, `+0.16`, and `-1.11` points. Thus mixed traffic can damage learning and SDIL often recovers it, but the effect is not consistent across seeds and the task does not satisfy the central endogenous-context premise. B1 fails as paper evidence and is retained without a norm-matched follow-up. The next BabyAI task is `PickupLoc`, which contains eight candidate objects per audited episode. ## P0 PickupLoc clean feasibility `BabyAI-PickupLoc-v0` replaces the diagnostic task. A direct structure audit found eight ball, box, or key objects in every one of 100 generated episodes; the mission selects which object to pick up. The standard partial `7 x 7` symbolic observation is retained. Because the present policy is feedforward, P0 first asks whether this architecture can solve the task before any raw or SDIL condition is exposed. P0 generates 20,000 training demonstrations, 2,000 validation demonstrations, and 500 disjoint validation rollout seeds from the official `BabyAIBot`. It uses the same clean-only grid as B0: hidden depth `2` or `4`, width `256`, learning rate `0.01` or `0.03`, context gain `1.0`, 15 epochs, and seed `4101`. A candidate is eligible if BP rollout success is at least `70%`, clean-KP success is at least `60%`, and removing the mission field reduces BP success by at least 20 points. Selection maximizes clean-KP rollout success, then action accuracy, then chooses fewer layers and the smaller learning rate. If no candidate is eligible, raw and SDIL are not run on this architecture; the next method change must supply task memory rather than tune shared-feedback strength. If P0 passes, the chosen configuration enters the same 40-epoch, three-seed BP/clean-KP/raw/SDIL comparison as B1. ## P0 outcome All eight runs were finite, but no feedforward candidate passed. The best BP rollout success was `63.6%` (four layers, learning rate `0.03`) and the best clean-KP success was `52.2%` at the same setting, below the frozen `70%` and `60%` thresholds. Their expert-action accuracies were `84.81%` and `83.84%`. Removing the mission reduced the best BP policy to `11.6%` success, confirming that this task depends on the language signal. Raw and SDIL were not run. The next clean-only feasibility stage keeps the best P0 depth and learning rate and adds a fixed basal history of observations and previous actions. It tests history lengths four and eight for 25 epochs, using only BP and clean KP. The same `70%` BP, `60%` clean-KP, and 20-point BP mission-lesion thresholds apply; ties favor the shorter history. This changes the missing state information, not the shared-feedback signal or its strength.