exp: freeze BabyAI shared-feedback protocol

1 files changed, 116 insertions, 0 deletions

diff --git a/BABYAI_SHARED_FEEDBACK.md b/BABYAI_SHARED_FEEDBACK.md
new file mode 100644
index 0000000..addd58a
--- /dev/null
+++ b/BABYAI_SHARED_FEEDBACK.md
@@ -0,0 +1,116 @@
+# BabyAI shared-feedback protocol
+
+## Question
+
+The clean scaling experiments show that reciprocal Kolen--Pollack (KP)
+feedback can train large networks, but do not show that SDIL adds anything to
+KP.  This experiment asks whether SDIL is useful when an ordinary top-down
+signal is required during inference and the same physical pathway also carries
+the teaching signal.
+
+BabyAI supplies this structure without an injected nuisance.  The visual
+observation is the basal input, while the language mission is required to
+choose the correct object and enters each hidden population through a fixed
+apical projection.  During imitation learning, the measured apical signal is
+
+```text
+a_l = C_l m + t_l,
+```
+
+where `C_l m` is the mission field used during inference and `t_l` is the
+reciprocal KP teaching field.  Raw shared-path learning uses `a_l`.  SDIL fits
+a per-cell affine prediction of `C_l m` from the local hidden activity and uses
+`a_l - P_l(h_l)`.
+
+This is a fixed-distribution, randomly interleaved grounded-language task.  It
+is not a continual-learning experiment.  The endpoint tests signal separation
+and task learning, not catastrophic forgetting.
+
+## Data and task
+
+The first task is the official `BabyAI-GoToObjS6-v1` environment from
+MiniGrid 3.1.0.  Expert demonstrations are generated by the official
+`BabyAIBot`.  Every recorded trajectory must end with positive reward; failed
+expert episodes abort data generation rather than being silently discarded.
+
+The basal observation is the standard partial `7 x 7 x 3` symbolic image plus
+agent direction.  Each of the object, color, and state channels is one-hot
+encoded.  The mission is a normalized bag of training-vocabulary tokens and is
+not included in the basal input.  Validation missions use the training
+vocabulary, with an explicit unknown token retained for auditability.
+
+Development data contain 20,000 expert episodes for training and 2,000
+disjoint expert episodes for validation.  A separate set of 500 validation
+seeds is used for closed-loop policy rollouts.  No test seeds are generated or
+evaluated until the method configuration and mechanism comparison are frozen.
+
+## Model and local updates
+
+The policy is a tanh MLP with a fixed mission projection into every hidden
+population and a seven-action linear readout:
+
+```text
+u_l = W_l h_(l-1) + b_l + C_l m
+h_l = tanh(u_l).
+```
+
+`C_l` is fixed after initialization.  Forward weights and independently stored
+reciprocal weights use the same modified-KP momentum, decay, and local
+correlation rule used in the existing shared-feedback implementation.  The
+reciprocal tensor is updated from the same locally available activity product;
+it is never copied from the forward tensor.  SDIL predictor fitting receives
+only the current hidden activity and ordinary mission field, never the expert
+action, loss, output error, KP teaching field, or downstream weight.
+
+## B0 clean selector
+
+B0 exists only to choose a model that the task and clean learning rules can
+train.  It runs BP and clean KP, never raw shared KP or SDIL.  The fixed grid is
+
+```text
+hidden layers: 2, 4
+hidden width: 256
+learning rate: 0.01, 0.03
+context gain: 1.0
+epochs: 15
+batch size: 256
+model/data seed: 4101
+```
+
+All candidates share demonstrations, minibatch order, initialization within a
+model shape, and rollout seeds.  A candidate is eligible only if BP and clean
+KP each reach at least 80% validation rollout success and removing the mission
+field reduces BP rollout success by at least 20 points.  Among eligible
+candidates, choose the highest clean-KP rollout success; ties are resolved by
+action accuracy, then fewer layers, then the smaller learning rate.  If no
+candidate is eligible, B0 fails and the shared-path endpoint is not run.
+
+## B1 mechanism endpoint
+
+The selected depth and learning rate are frozen.  B1 trains for 40 epochs on
+three model seeds with identical data and minibatch orders across conditions:
+
+1. `bp`: exact backpropagation reference with the same fixed mission pathway.
+2. `clean_kp`: instruction-only reciprocal KP; this receives a separate clean
+   teaching wire and is the local-learning upper bound.
+3. `raw_shared`: the local teaching rule directly uses `C_l m + t_l`.
+4. `sdil`: a per-cell neutral affine predictor is fitted on 4,096 training
+   observations before each epoch and subtracts the predicted mission field.
+
+The predictor's extra forward observations, wall time, and arithmetic are
+reported.  The main metrics are validation closed-loop mission success,
+validation expert-action accuracy, mission-lesion success, cross-entropy,
+predictor explained variance, and the residual mission-field RMS.  Accuracy
+and mission success, not residual size, decide the result.
+
+B1 supports the mechanism if clean KP is successful, mission removal damages
+the task, raw shared KP is worse than clean KP, and SDIL recovers a substantial
+part of that downstream success gap consistently across seeds.  No fixed
+five-point threshold is imposed before observing the natural effect size.
+Norm-matched raw feedback is added only after a positive raw--SDIL difference,
+because it diagnoses that difference but cannot create it.
+
+Passing B1 opens `PickupLoc` and `PutNextLocalS6N4` with the same method and
+selection rule, followed by depth scaling.  Failure is retained as evidence
+that task-required shared feedback alone is insufficient to make
+residualization useful in this setting.