summaryrefslogtreecommitdiff
path: root/NOTE.md
diff options
context:
space:
mode:
authorYurenHao0426 <Blackhao0426@gmail.com>2026-03-27 18:07:58 -0500
committerYurenHao0426 <Blackhao0426@gmail.com>2026-03-27 18:07:58 -0500
commit2a230acd5ee3fa6605892d524badf281ba7e9cfd (patch)
tree9b3eadb60966b895a0349cbb457e6dab2004af47 /NOTE.md
parent4d6e689fe6bfffef6db7a4650aec210cd3eeed5c (diff)
Add Phase 10A.8C: 3-seed replication — scaffold gains are marginal
3-seed results (mean±std): - DFA: 0.306±0.006 - perlayer_vector α=0.75: 0.304±0.006 (-0.2%, not significant) - random_trainable α=0.75: 0.313±0.007 (+0.7%, marginal, error bars overlap) Single-seed gains (+1.1% perlayer, +0.8% vec) do not robustly replicate. The scaffold mechanism provides at best a marginal, statistically uncertain benefit. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Diffstat (limited to 'NOTE.md')
-rw-r--r--NOTE.md10
1 files changed, 10 insertions, 0 deletions
diff --git a/NOTE.md b/NOTE.md
index 90057af..b7ae247 100644
--- a/NOTE.md
+++ b/NOTE.md
@@ -647,6 +647,16 @@ But 28.5% < DFA 31.2% → continuous trainability adds ~2.7% additional value.
Both methods optimal at α=0.75. perlayer_vector (+1.1%) ≈ random_trainable (+0.8%).
Per-layer vector is the minimal sufficient scaffold.
+### Phase 10A.8C: 3-Seed Replication
+
+| Method | final acc (3 seeds) | diff vs DFA |
+|--------|---------------------|-------------|
+| DFA | 0.306±0.006 | baseline |
+| perlayer_vector α=0.75 | 0.304±0.006 | -0.2% (not significant) |
+| random_trainable α=0.75 | 0.313±0.007 | +0.7% (marginal) |
+
+Single-seed gains do not robustly replicate. Error bars overlap.
+
### Experiment IDs (Phase 10)
- `prefit_threshold/`: Phase 10A prefit threshold curve
- `blend_dissection/`: Phase 10A.5 blend mechanism dissection