From 2a230acd5ee3fa6605892d524badf281ba7e9cfd Mon Sep 17 00:00:00 2001 From: YurenHao0426 Date: Fri, 27 Mar 2026 18:07:58 -0500 Subject: =?UTF-8?q?Add=20Phase=2010A.8C:=203-seed=20replication=20?= =?UTF-8?q?=E2=80=94=20scaffold=20gains=20are=20marginal?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 3-seed results (mean±std): - DFA: 0.306±0.006 - perlayer_vector α=0.75: 0.304±0.006 (-0.2%, not significant) - random_trainable α=0.75: 0.313±0.007 (+0.7%, marginal, error bars overlap) Single-seed gains (+1.1% perlayer, +0.8% vec) do not robustly replicate. The scaffold mechanism provides at best a marginal, statistically uncertain benefit. Co-Authored-By: Claude Opus 4.6 (1M context) --- NOTE.md | 10 ++++++++++ 1 file changed, 10 insertions(+) (limited to 'NOTE.md') diff --git a/NOTE.md b/NOTE.md index 90057af..b7ae247 100644 --- a/NOTE.md +++ b/NOTE.md @@ -647,6 +647,16 @@ But 28.5% < DFA 31.2% → continuous trainability adds ~2.7% additional value. Both methods optimal at α=0.75. perlayer_vector (+1.1%) ≈ random_trainable (+0.8%). Per-layer vector is the minimal sufficient scaffold. +### Phase 10A.8C: 3-Seed Replication + +| Method | final acc (3 seeds) | diff vs DFA | +|--------|---------------------|-------------| +| DFA | 0.306±0.006 | baseline | +| perlayer_vector α=0.75 | 0.304±0.006 | -0.2% (not significant) | +| random_trainable α=0.75 | 0.313±0.007 | +0.7% (marginal) | + +Single-seed gains do not robustly replicate. Error bars overlap. + ### Experiment IDs (Phase 10) - `prefit_threshold/`: Phase 10A prefit threshold curve - `blend_dissection/`: Phase 10A.5 blend mechanism dissection -- cgit v1.2.3