1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
|
# BabyAI shared-feedback protocol
## Question
The clean scaling experiments show that reciprocal Kolen--Pollack (KP)
feedback can train large networks, but do not show that SDIL adds anything to
KP. This experiment asks whether SDIL is useful when an ordinary top-down
signal is required during inference and the same physical pathway also carries
the teaching signal.
BabyAI supplies this structure without an injected nuisance. The visual
observation is the basal input, while the language mission is required to
choose the correct object and enters each hidden population through a fixed
apical projection. During imitation learning, the measured apical signal is
```text
a_l = C_l m + t_l,
```
where `C_l m` is the mission field used during inference and `t_l` is the
reciprocal KP teaching field. Raw shared-path learning uses `a_l`. SDIL fits
a per-cell affine prediction of `C_l m` from the local hidden activity and uses
`a_l - P_l(h_l)`.
This is a fixed-distribution, randomly interleaved grounded-language task. It
is not a continual-learning experiment. The endpoint tests signal separation
and task learning, not catastrophic forgetting.
## Data and task
The first task is the official `BabyAI-GoToObjS6-v1` environment from
MiniGrid 3.1.0. Expert demonstrations are generated by the official
`BabyAIBot`. Every recorded trajectory must end with positive reward; failed
expert episodes abort data generation rather than being silently discarded.
The basal observation is the standard partial `7 x 7 x 3` symbolic image plus
agent direction. Each of the object, color, and state channels is one-hot
encoded. The mission is a normalized bag of training-vocabulary tokens and is
not included in the basal input. Validation missions use the training
vocabulary, with an explicit unknown token retained for auditability.
Development data contain 20,000 expert episodes for training and 2,000
disjoint expert episodes for validation. A separate set of 500 validation
seeds is used for closed-loop policy rollouts. No test seeds are generated or
evaluated until the method configuration and mechanism comparison are frozen.
## Model and local updates
The policy is a tanh MLP with a fixed mission projection into every hidden
population and a seven-action linear readout:
```text
u_l = W_l h_(l-1) + b_l + C_l m
h_l = tanh(u_l).
```
`C_l` is fixed after initialization. Forward weights and independently stored
reciprocal weights use the same modified-KP momentum, decay, and local
correlation rule used in the existing shared-feedback implementation. The
reciprocal tensor is updated from the same locally available activity product;
it is never copied from the forward tensor. SDIL predictor fitting receives
only the current hidden activity and ordinary mission field, never the expert
action, loss, output error, KP teaching field, or downstream weight.
## B0 clean selector
B0 exists only to choose a model that the task and clean learning rules can
train. It runs BP and clean KP, never raw shared KP or SDIL. The fixed grid is
```text
hidden layers: 2, 4
hidden width: 256
learning rate: 0.01, 0.03
context gain: 1.0
epochs: 15
batch size: 256
model/data seed: 4101
```
All candidates share demonstrations, minibatch order, initialization within a
model shape, and rollout seeds. A candidate is eligible only if BP and clean
KP each reach at least 80% validation rollout success and removing the mission
field reduces BP rollout success by at least 20 points. Among eligible
candidates, choose the highest clean-KP rollout success; ties are resolved by
action accuracy, then fewer layers, then the smaller learning rate. If no
candidate is eligible, B0 fails and the shared-path endpoint is not run.
## B1 mechanism endpoint
The selected depth and learning rate are frozen. B1 trains for 40 epochs on
model and minibatch-order seeds `4101`, `4102`, and `4103`, with identical data
and minibatch orders across conditions:
1. `bp`: exact backpropagation reference with the same fixed mission pathway.
2. `clean_kp`: instruction-only reciprocal KP; this receives a separate clean
teaching wire and is the local-learning upper bound.
3. `raw_shared`: the local teaching rule directly uses `C_l m + t_l`.
4. `sdil`: a per-cell neutral affine predictor is fitted on 4,096 training
observations before each epoch and subtracts the predicted mission field.
The predictor's extra forward observations, wall time, and arithmetic are
reported. The main metrics are validation closed-loop mission success,
validation expert-action accuracy, mission-lesion success, cross-entropy,
predictor explained variance, and the residual mission-field RMS. Accuracy
and mission success, not residual size, decide the result.
B1 supports the mechanism if clean KP is successful, mission removal damages
the task, raw shared KP is worse than clean KP, and SDIL recovers a substantial
part of that downstream success gap consistently across seeds. No fixed
five-point threshold is imposed before observing the natural effect size.
Norm-matched raw feedback is added only after a positive raw--SDIL difference,
because it diagnoses that difference but cannot create it.
Passing B1 opens `PickupLoc` and `PutNextLocalS6N4` with the same method and
selection rule, followed by depth scaling. Failure is retained as evidence
that task-required shared feedback alone is insufficient to make
residualization useful in this setting.
## B0 outcome
All eight clean-selector runs were finite. BP and clean KP both reached
`97.8%` validation rollout success and `99.74%` expert-action accuracy in every
candidate. The two-layer, `0.01` learning-rate candidate was selected by the
frozen tie rule: its BP mission lesion reduced rollout success from `97.8%` to
`70.4%`, while it tied for the best clean-KP success and action accuracy and
used the fewest layers and smaller learning rate. The selector read no raw,
SDIL, or test result. This outcome opens B1 without establishing an SDIL
advantage.
## B1 outcome: diagnostic only
The task-structure audit found exactly one ball, box, or key in each of 256
generated `GoToObjS6` episodes. That sole object is always the target, so the
mission does not select among alternatives. The original claim that language
is required by this task was therefore false; mission lesion is only an input
ablation in this environment.
The three-seed downstream result is nevertheless useful as an implementation
diagnostic. Mean rollout success was `97.8%` for BP, `97.8%` for clean KP,
`83.93%` for raw shared KP, and `95.87%` for SDIL. The paired SDIL-minus-raw
rollout differences were `+30.6`, `+5.6`, and `-0.4` points; expert-action
differences were `+12.22`, `+0.16`, and `-1.11` points. Thus mixed traffic can
damage learning and SDIL often recovers it, but the effect is not consistent
across seeds and the task does not satisfy the central endogenous-context
premise. B1 fails as paper evidence and is retained without a norm-matched
follow-up. The next BabyAI task is `PickupLoc`, which contains eight candidate
objects per audited episode.
## P0 PickupLoc clean feasibility
`BabyAI-PickupLoc-v0` replaces the diagnostic task. A direct structure audit
found eight ball, box, or key objects in every one of 100 generated episodes;
the mission selects which object to pick up. The standard partial `7 x 7`
symbolic observation is retained. Because the present policy is feedforward,
P0 first asks whether this architecture can solve the task before any raw or
SDIL condition is exposed.
P0 generates 20,000 training demonstrations, 2,000 validation demonstrations,
and 500 disjoint validation rollout seeds from the official `BabyAIBot`. It
uses the same clean-only grid as B0: hidden depth `2` or `4`, width `256`,
learning rate `0.01` or `0.03`, context gain `1.0`, 15 epochs, and seed `4101`.
A candidate is eligible if BP rollout success is at least `70%`, clean-KP
success is at least `60%`, and removing the mission field reduces BP success by
at least 20 points. Selection maximizes clean-KP rollout success, then action
accuracy, then chooses fewer layers and the smaller learning rate.
If no candidate is eligible, raw and SDIL are not run on this architecture;
the next method change must supply task memory rather than tune shared-feedback
strength. If P0 passes, the chosen configuration enters the same 40-epoch,
three-seed BP/clean-KP/raw/SDIL comparison as B1.
|
