diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-03-27 18:07:58 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-03-27 18:07:58 -0500 |
| commit | 2a230acd5ee3fa6605892d524badf281ba7e9cfd (patch) | |
| tree | 9b3eadb60966b895a0349cbb457e6dab2004af47 /NOTE.md | |
| parent | 4d6e689fe6bfffef6db7a4650aec210cd3eeed5c (diff) | |
Add Phase 10A.8C: 3-seed replication — scaffold gains are marginal
3-seed results (mean±std):
- DFA: 0.306±0.006
- perlayer_vector α=0.75: 0.304±0.006 (-0.2%, not significant)
- random_trainable α=0.75: 0.313±0.007 (+0.7%, marginal, error bars overlap)
Single-seed gains (+1.1% perlayer, +0.8% vec) do not robustly replicate.
The scaffold mechanism provides at best a marginal, statistically uncertain benefit.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Diffstat (limited to 'NOTE.md')
| -rw-r--r-- | NOTE.md | 10 |
1 files changed, 10 insertions, 0 deletions
@@ -647,6 +647,16 @@ But 28.5% < DFA 31.2% → continuous trainability adds ~2.7% additional value. Both methods optimal at α=0.75. perlayer_vector (+1.1%) ≈ random_trainable (+0.8%). Per-layer vector is the minimal sufficient scaffold. +### Phase 10A.8C: 3-Seed Replication + +| Method | final acc (3 seeds) | diff vs DFA | +|--------|---------------------|-------------| +| DFA | 0.306±0.006 | baseline | +| perlayer_vector α=0.75 | 0.304±0.006 | -0.2% (not significant) | +| random_trainable α=0.75 | 0.313±0.007 | +0.7% (marginal) | + +Single-seed gains do not robustly replicate. Error bars overlap. + ### Experiment IDs (Phase 10) - `prefit_threshold/`: Phase 10A prefit threshold curve - `blend_dissection/`: Phase 10A.5 blend mechanism dissection |
