diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-03-26 08:37:39 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-03-26 08:37:39 -0500 |
| commit | ef4aed70130e2212b4ed1cb7212e2ea6c7c7adb2 (patch) | |
| tree | ad9f128753350ec4f430f77baa018189e4a9d4be /NOTE.md | |
| parent | 05ccd23154d1e9d090178b9d4d5f2c821711e784 (diff) | |
Add Phase 10A: no prefit threshold — even random Vec blend beats DFA by +1.3%
E_prefit=0 (random Vec) + blend(0.75): 32.4% vs DFA 31.1% (+1.3%)
E_prefit=15: 32.3% (+1.2%)
E_prefit=60: 32.5% (+1.4%)
Frozen Gamma/rho near zero at all prefit levels. The Phase 9A success was NOT
from Vec learning useful credit — it was from the blend mechanism itself providing
regularization/diversification over pure DFA.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Diffstat (limited to 'NOTE.md')
| -rw-r--r-- | NOTE.md | 24 |
1 files changed, 23 insertions, 1 deletions
@@ -5,7 +5,7 @@ - **pilot**: Controlled iteration (commits 0b9ebb2, 7baf7ae) - **frozen**: Code at commit 0b9ebb2 for all reported results -## Status: PHASE 9 COMPLETE — OFFLINE PREFIT + BLEND IS THE KEY +## Status: PHASE 10A — NO PREFIT THRESHOLD, BLEND ITSELF IS THE ACTIVE INGREDIENT --- @@ -550,3 +550,25 @@ The +1.5% gain from 9A's blend(0.75) at t0=5 is the project's best online result - `checkpointed_handoff/`: Phase 9A checkpointed handoff with branches - `periodic_refit/`: Phase 9B periodic refit - `topdown_curriculum/`: Phase 9C top-down curriculum + +--- + +## Phase 10A: Prefit Threshold Curve + +**Setup**: t0=5, blend_075, E_prefit in {0, 15, 60}, seed=42 + +| E_prefit | Gamma_frozen | rho_frozen | final acc | diff vs DFA | +|----------|-------------|-----------|-----------|-------------| +| 0 (random Vec) | -0.005 | 0.014 | **0.324** | **+1.3%** | +| 15 | 0.002 | 0.011 | 0.323 | +1.2% | +| 60 | -0.001 | -0.009 | 0.325 | +1.4% | +| continue_DFA | — | — | 0.311 | baseline | + +**Case C: NO prefit threshold exists.** Even random Vec (E=0) with blend(0.75) beats DFA. + +**Critical reinterpretation of Phase 9A**: the +1.5% gain was NOT from Vec learning good credit. +Frozen Gamma/rho are near zero at all prefit levels. The benefit comes from the blend mechanism +itself — blending DFA with any additional signal provides regularization/diversification. + +### Experiment IDs (Phase 10) +- `prefit_threshold/`: Phase 10A prefit threshold curve |
