1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
|
# BabyAI shared-feedback protocol
## Question
The clean scaling experiments show that reciprocal Kolen--Pollack (KP)
feedback can train large networks, but do not show that SDIL adds anything to
KP. This experiment asks whether SDIL is useful when an ordinary top-down
signal is required during inference and the same physical pathway also carries
the teaching signal.
BabyAI supplies this structure without an injected nuisance. The visual
observation is the basal input, while the language mission is required to
choose the correct object and enters each hidden population through a fixed
apical projection. During imitation learning, the measured apical signal is
```text
a_l = C_l m + t_l,
```
where `C_l m` is the mission field used during inference and `t_l` is the
reciprocal KP teaching field. Raw shared-path learning uses `a_l`. SDIL fits
a per-cell affine prediction of `C_l m` from the local hidden activity and uses
`a_l - P_l(h_l)`.
This is a fixed-distribution, randomly interleaved grounded-language task. It
is not a continual-learning experiment. The endpoint tests signal separation
and task learning, not catastrophic forgetting.
## Data and task
The first task is the official `BabyAI-GoToObjS6-v1` environment from
MiniGrid 3.1.0. Expert demonstrations are generated by the official
`BabyAIBot`. Every recorded trajectory must end with positive reward; failed
expert episodes abort data generation rather than being silently discarded.
The basal observation is the standard partial `7 x 7 x 3` symbolic image plus
agent direction. Each of the object, color, and state channels is one-hot
encoded. The mission is a normalized bag of training-vocabulary tokens and is
not included in the basal input. Validation missions use the training
vocabulary, with an explicit unknown token retained for auditability.
Development data contain 20,000 expert episodes for training and 2,000
disjoint expert episodes for validation. A separate set of 500 validation
seeds is used for closed-loop policy rollouts. No test seeds are generated or
evaluated until the method configuration and mechanism comparison are frozen.
## Model and local updates
The policy is a tanh MLP with a fixed mission projection into every hidden
population and a seven-action linear readout:
```text
u_l = W_l h_(l-1) + b_l + C_l m
h_l = tanh(u_l).
```
`C_l` is fixed after initialization. Forward weights and independently stored
reciprocal weights use the same modified-KP momentum, decay, and local
correlation rule used in the existing shared-feedback implementation. The
reciprocal tensor is updated from the same locally available activity product;
it is never copied from the forward tensor. SDIL predictor fitting receives
only the current hidden activity and ordinary mission field, never the expert
action, loss, output error, KP teaching field, or downstream weight.
## B0 clean selector
B0 exists only to choose a model that the task and clean learning rules can
train. It runs BP and clean KP, never raw shared KP or SDIL. The fixed grid is
```text
hidden layers: 2, 4
hidden width: 256
learning rate: 0.01, 0.03
context gain: 1.0
epochs: 15
batch size: 256
model/data seed: 4101
```
All candidates share demonstrations, minibatch order, initialization within a
model shape, and rollout seeds. A candidate is eligible only if BP and clean
KP each reach at least 80% validation rollout success and removing the mission
field reduces BP rollout success by at least 20 points. Among eligible
candidates, choose the highest clean-KP rollout success; ties are resolved by
action accuracy, then fewer layers, then the smaller learning rate. If no
candidate is eligible, B0 fails and the shared-path endpoint is not run.
## B1 mechanism endpoint
The selected depth and learning rate are frozen. B1 trains for 40 epochs on
model and minibatch-order seeds `4101`, `4102`, and `4103`, with identical data
and minibatch orders across conditions:
1. `bp`: exact backpropagation reference with the same fixed mission pathway.
2. `clean_kp`: instruction-only reciprocal KP; this receives a separate clean
teaching wire and is the local-learning upper bound.
3. `raw_shared`: the local teaching rule directly uses `C_l m + t_l`.
4. `sdil`: a per-cell neutral affine predictor is fitted on 4,096 training
observations before each epoch and subtracts the predicted mission field.
The predictor's extra forward observations, wall time, and arithmetic are
reported. The main metrics are validation closed-loop mission success,
validation expert-action accuracy, mission-lesion success, cross-entropy,
predictor explained variance, and the residual mission-field RMS. Accuracy
and mission success, not residual size, decide the result.
B1 supports the mechanism if clean KP is successful, mission removal damages
the task, raw shared KP is worse than clean KP, and SDIL recovers a substantial
part of that downstream success gap consistently across seeds. No fixed
five-point threshold is imposed before observing the natural effect size.
Norm-matched raw feedback is added only after a positive raw--SDIL difference,
because it diagnoses that difference but cannot create it.
Passing B1 opens `PickupLoc` and `PutNextLocalS6N4` with the same method and
selection rule, followed by depth scaling. Failure is retained as evidence
that task-required shared feedback alone is insufficient to make
residualization useful in this setting.
## B0 outcome
All eight clean-selector runs were finite. BP and clean KP both reached
`97.8%` validation rollout success and `99.74%` expert-action accuracy in every
candidate. The two-layer, `0.01` learning-rate candidate was selected by the
frozen tie rule: its BP mission lesion reduced rollout success from `97.8%` to
`70.4%`, while it tied for the best clean-KP success and action accuracy and
used the fewest layers and smaller learning rate. The selector read no raw,
SDIL, or test result. This outcome opens B1 without establishing an SDIL
advantage.
|
