summaryrefslogtreecommitdiff
path: root/BABYAI_SHARED_FEEDBACK.md
diff options
context:
space:
mode:
authorYurenHao0426 <Blackhao0426@gmail.com>2026-08-10 10:25:00 -0500
committerYurenHao0426 <Blackhao0426@gmail.com>2026-08-10 10:25:00 -0500
commit5ab644db6823491ffadb793162be36f4aa5edd04 (patch)
treeba70c4ea359b5a7735f7970a5d23d28e4b0e3779 /BABYAI_SHARED_FEEDBACK.md
parente3011f144a018f6415b3c2a636c0646e412b3ced (diff)
results: retain GoToObj as diagnostic only
Diffstat (limited to 'BABYAI_SHARED_FEEDBACK.md')
-rw-r--r--BABYAI_SHARED_FEEDBACK.md19
1 files changed, 19 insertions, 0 deletions
diff --git a/BABYAI_SHARED_FEEDBACK.md b/BABYAI_SHARED_FEEDBACK.md
index d00af72..74188ea 100644
--- a/BABYAI_SHARED_FEEDBACK.md
+++ b/BABYAI_SHARED_FEEDBACK.md
@@ -126,3 +126,22 @@ frozen tie rule: its BP mission lesion reduced rollout success from `97.8%` to
used the fewest layers and smaller learning rate. The selector read no raw,
SDIL, or test result. This outcome opens B1 without establishing an SDIL
advantage.
+
+## B1 outcome: diagnostic only
+
+The task-structure audit found exactly one ball, box, or key in each of 256
+generated `GoToObjS6` episodes. That sole object is always the target, so the
+mission does not select among alternatives. The original claim that language
+is required by this task was therefore false; mission lesion is only an input
+ablation in this environment.
+
+The three-seed downstream result is nevertheless useful as an implementation
+diagnostic. Mean rollout success was `97.8%` for BP, `97.8%` for clean KP,
+`83.93%` for raw shared KP, and `95.87%` for SDIL. The paired SDIL-minus-raw
+rollout differences were `+30.6`, `+5.6`, and `-0.4` points; expert-action
+differences were `+12.22`, `+0.16`, and `-1.11` points. Thus mixed traffic can
+damage learning and SDIL often recovers it, but the effect is not consistent
+across seeds and the task does not satisfy the central endogenous-context
+premise. B1 fails as paper evidence and is retained without a norm-matched
+follow-up. The next BabyAI task is `PickupLoc`, which contains eight candidate
+objects per audited episode.