diff options
Diffstat (limited to 'NOTE.md')
| -rw-r--r-- | NOTE.md | 37 |
1 files changed, 36 insertions, 1 deletions
@@ -5,7 +5,7 @@ - **pilot**: Controlled iteration (commits 0b9ebb2, 7baf7ae) - **frozen**: Code at commit 0b9ebb2 for all reported results -## Status: PHASE 7A SNAPSHOT TIME SWEEP — EARLY SNAPSHOTS SHOW POSITIVE TRANSFER +## Status: PHASE 8 SCHEDULE TIMING — ONLINE CO-LEARNING IS THE REMAINING BOTTLENECK --- @@ -449,3 +449,38 @@ Credit bridge should be used from epoch 0. ### Experiment IDs (Phase 7) - `snapshot_time/`: Phase 7A snapshot time sweep with BP checkpoints + +--- + +## Phase 8: Schedule Timing Hypothesis Test + +**Setup**: CIFAR-10, L=4, d=256, 100 epochs, seed=42 + +| Schedule | acc@5 | acc@20 | final | +|----------|-------|--------|-------| +| DFA_only | **0.297** | **0.308** | **0.312** | +| Vec_only_from_0 | 0.135 | 0.151 | 0.154 | +| Vec_T5_then_DFA | 0.135 | 0.213 | 0.266 | +| DFA_T20_then_Vec | 0.297 | 0.308 | 0.129 | + +**Phase 7A's timing hypothesis does NOT transfer to online training.** + +Vec from epoch 0 gets stuck at 15% (near chance). The online Vec estimator +starts from random initialization and cannot learn useful credit fast enough +when the forward net is also random (cold-start paradox). + +DFA alone remains the best non-BP method (31.2%). + +### The cold-start paradox: +Vec credit is most useful on early features, but Vec can only learn useful credit +from features with structure. DFA provides structure slowly, but by the time Vec +is ready, the early window is closed. + +### Project conclusion at this point: +- Vec estimator WORKS (synthetic + frozen CIFAR) +- Local surrogate CAN exploit it (same-batch, Phase 6.5A) +- Early snapshots show generalization (Phase 7A, offline-trained Vec) +- But online co-learning of estimator + forward net is unsolved (cold-start) + +### Experiment IDs (Phase 8) +- `schedule_timing/`: Phase 8 schedule comparison |