diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-03-26 16:27:53 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-03-26 16:27:53 -0500 |
| commit | 610e1169e19378cccd2d9b92a588c24dca7f3df7 (patch) | |
| tree | 532f8dc2fda6c68ab1409b20d7431b76d8d6f378 /NOTE.md | |
| parent | ef4aed70130e2212b4ed1cb7212e2ea6c7c7adb2 (diff) | |
Add Phase 10A.5: blend gain is implicit regularization, not learned credit
Dissection of 6 branches from same DFA checkpoint:
- blend_random_frozen: 12.6% (CATASTROPHIC — frozen noise destroys training)
- blend_random_trainable: 32.2% (+1.2% — trainable network helps)
- blend_shuffled_trainable: 32.5% (+1.4% — even wrong targets work!)
- blend_gaussian_noise: 30.8% (neutral)
- scaled_DFA_norm_match: 31.0% (neutral)
The gain comes from implicit regularization through a co-optimized auxiliary
network, NOT from learned credit quality. Phase 9A's +1.5% was an optimization
dynamics effect, not evidence of useful credit assignment.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Diffstat (limited to 'NOTE.md')
| -rw-r--r-- | NOTE.md | 20 |
1 files changed, 19 insertions, 1 deletions
@@ -5,7 +5,7 @@ - **pilot**: Controlled iteration (commits 0b9ebb2, 7baf7ae) - **frozen**: Code at commit 0b9ebb2 for all reported results -## Status: PHASE 10A — NO PREFIT THRESHOLD, BLEND ITSELF IS THE ACTIVE INGREDIENT +## Status: PHASE 10A.5 — BLEND GAIN IS IMPLICIT REGULARIZATION, NOT LEARNED CREDIT --- @@ -570,5 +570,23 @@ The +1.5% gain from 9A's blend(0.75) at t0=5 is the project's best online result Frozen Gamma/rho are near zero at all prefit levels. The benefit comes from the blend mechanism itself — blending DFA with any additional signal provides regularization/diversification. +### Phase 10A.5: Blend Mechanism Dissection + +| Branch | final acc | diff vs DFA | +|--------|-----------|-------------| +| continue_DFA | 0.311 | baseline | +| blend_random_**frozen** | **0.126** | **-18.5%** (catastrophic) | +| blend_random_**trainable** | 0.322 | +1.2% | +| blend_shuffled_trainable | 0.325 | +1.4% | +| blend_gaussian_noise | 0.308 | -0.3% | +| scaled_DFA_norm_match | 0.310 | -0.0% | + +**Mechanism identified**: The gain is from **implicit regularization through a trainable +auxiliary network**, NOT from learned credit. Frozen random Vec crashes (12.6%). +Trainable Vec helps even with shuffled targets. Gaussian noise and norm scaling don't help. + +Phase 9A's +1.5% was not evidence of useful credit — it was an optimization dynamics effect. + ### Experiment IDs (Phase 10) - `prefit_threshold/`: Phase 10A prefit threshold curve +- `blend_dissection/`: Phase 10A.5 blend mechanism dissection |
