diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-03-24 18:03:55 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-03-24 18:03:55 -0500 |
| commit | 5550e2cac45758e579810ae36bf716a0b819cebc (patch) | |
| tree | 28f263e4030d6d5144af5badcebd533b27f4da78 /NOTE.md | |
| parent | 3d17cbad98f320905c52509c7f18691eab8bf2a0 (diff) | |
Add Phase 5: vector field audit, frozen CIFAR transfer, online pilot
Phase 5A: Audit passes — shuffle control collapses, gains are real
Phase 5B: Transfer SUCCESS — vec_M4 beats scalar CB by +0.25 Gamma, +0.31 rho on frozen CIFAR
Phase 5C: Online FAILURE — vec does worse than scalar CB online despite better frozen credit
Core finding: bottleneck is in local surrogate / co-adaptation, not estimator quality
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Diffstat (limited to 'NOTE.md')
| -rw-r--r-- | NOTE.md | 69 |
1 files changed, 68 insertions, 1 deletions
@@ -5,7 +5,7 @@ - **pilot**: Controlled iteration (commits 0b9ebb2, 7baf7ae) - **frozen**: Code at commit 0b9ebb2 for all reported results -## Status: PHASE 4 DIAGNOSTIC DISSECTION COMPLETE +## Status: PHASE 5 VECTOR FIELD AUDIT + TRANSFER COMPLETE --- @@ -262,3 +262,70 @@ The scalar V curvature problem is real and avoidable. 2. Key concern: M=4 perturbation directions may not suffice for d=256/512 dimensions (signal/noise ~ sqrt(M/d) ~ sqrt(4/256) ~ 0.12) 3. May need M=32+ for CIFAR, which is expensive but tractable + +--- + +## Phase 5: Vector Field Audit + Real-Task Transfer + +### Phase 5A: Synthetic Audit (4 sanity checks) + +**Setup**: alpha=1.0, L={4,8}, d=128, 3 seeds, 80 epochs + +**Smoke test result (L=4, seed=42):** + +| Method | Gamma | rho | nudge | +|--------|-------|-----|-------| +| scalar_cb | 0.224 | 0.210 | -0.007 | +| vec_eT_M4 | **0.847** | **0.951** | -0.026 | +| vec_shuffleCtrl | 0.051 | 0.068 | -0.001 | +| vec_noTerm | 0.955 | 0.971 | -0.027 | +| vec_onesided | 0.832 | 0.943 | -0.024 | + +**Check B (shuffle)**: PASS — shuffled control collapses (5/6 near zero across full audit) +**Check C (noTerm)**: Terminal NOT needed — perturbation target alone gives 0.95+ Gamma +**Check D (onesided)**: PASS — one-sided ≈ central difference + +**Full 3-seed audit**: All 6 configs pass (delta_Gamma >= 0.49, delta_rho >= 0.55) + +### Phase 5B: Frozen CIFAR Vector Transfer + +**Setup**: CIFAR-10, frozen BP ref (L=4, d=256, 61.7%), 100 epochs estimators + +| Method | Gamma | rho | nudge | +|--------|-------|-----|-------| +| DFA | 0.005 | 0.005 | -0.000006 | +| ScalarCB_eT | 0.115 | 0.120 | -0.000370 | +| StateBridge_eT | 0.287 | 0.264 | -0.000957 | +| **Vec_eT_M4** | **0.364** | **0.426** | **-0.001406** | + +**TRANSFER SUCCESS**: Vec beats scalar CB by +0.25 Gamma, +0.31 rho. +Vec also beats state bridge on rho (0.43 vs 0.26). +M=4 is sufficient (M=8, M=16 give same results). + +### Phase 5C: Online CIFAR Vector Pilot + +**Setup**: CIFAR-10, L=4, d=256, 100 epochs, seed=42 + +| Config | Acc | Gamma | rho | S1 | S2 | +|--------|-----|-------|-----|----|----| +| DFA | 0.312 | 0.101 | -0.005 | 0 | 0 | +| vec wr=0.2 tw=1.0 | 0.243 | 0.001 | 0.000 | -0.100 | +0.005 | +| (for comparison: scalar CB wr=0.2 tgw=1.0 from Phase 4) | 0.283 | 0.179 | 0.009 | +0.079 | +0.014 | + +**ONLINE FAILURE**: Vector field does WORSE than scalar CB online, despite being +much better on frozen features. No config achieves S1>0. + +### Core Finding of Phases 4-5 + +**The bottleneck is NOT the credit estimator.** Improving credit quality from +scalar CB (Gamma=0.12) to vector field (Gamma=0.36) on frozen features does NOT +help online training. The bottleneck is in the **local surrogate update**: +<F_l(h_l), a_l> cannot effectively exploit directional credit information, +and co-adaptation between forward net and credit estimator rapidly degrades +the signal. + +### Experiment IDs (Phase 5) +- `vector_audit/`: Phase 5A smoke test +- `vector_audit_full/`: Phase 5A full 3-seed audit +- `frozen_cifar_vec/`: Phase 5B frozen CIFAR vector transfer +- `online_vec_pilot/`: Phase 5C online CIFAR vector pilot |
