diff options
Diffstat (limited to 'BABYAI_SHARED_FEEDBACK.md')
| -rw-r--r-- | BABYAI_SHARED_FEEDBACK.md | 26 |
1 files changed, 26 insertions, 0 deletions
diff --git a/BABYAI_SHARED_FEEDBACK.md b/BABYAI_SHARED_FEEDBACK.md index 009709f..aa24b84 100644 --- a/BABYAI_SHARED_FEEDBACK.md +++ b/BABYAI_SHARED_FEEDBACK.md @@ -194,3 +194,29 @@ chooses four steps. The P2 mechanism endpoint uses four hidden layers, width 256, learning rate `0.03`, four-step history, 40 epochs, and model/minibatch seeds `4101`, `4102`, and `4103`. No raw or SDIL result was run or read during selection. + +## P2 outcome: diagonal predictor fails + +The task and raw-failure premises both hold, but the current SDIL predictor +does not. Across seeds `4101--4103`, rollout success was + +| method | seed 4101 | seed 4102 | seed 4103 | +| --- | ---: | ---: | ---: | +| BP | 80.8 | 83.6 | 73.6 | +| clean KP | 67.2 | 77.0 | 68.4 | +| raw shared KP | 0.0 | 23.2 | 6.8 | +| diagonal per-cell SDIL | 0.0 | 0.0 | 0.0 | + +Raw mission traffic therefore damages reciprocal KP by `53.8--70.2` points, +but SDIL is never better than raw. Its neutral affine predictor explains only +about `30--31%` of per-cell mission variance on average, and the worst layer +retains about `60--62%` of the original context RMS. All runs remain finite, +so this is a downstream learning failure rather than numerical termination. + +P2 closes the diagonal predictor on this task. The originally stated SDIL +equation used a population map `P_l h_l`, whereas this implementation used only +`p_li h_li + b_li`. A population-predictor rescue may proceed only through a +neutral prediction screen that reads no action, teaching, raw, SDIL, or test +outcome. If it cannot explain at least 80% of held-out mission-field variance +and leave at most 25% context RMS in every layer, no further PickupLoc endpoint +is run. |