From 84fa70be406248e660488ddc4c56ca5ea0034189 Mon Sep 17 00:00:00 2001 From: YurenHao0426 Date: Tue, 9 Jun 2026 15:30:20 -0500 Subject: Add teacher-task test-gap sweep and figure MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit SGD teacher sweep (widths 8-96, 24 FA runs/width): train gap soft and monotone (0.249 -> 0.0069) while the test gap changes sign — FA generalizes better than BP at intermediate width (-0.120 +/- 0.013 at w=32) and settles slightly positive once both fit. Scopes the paper's optimization-gap claims and preempts the generalization question. Co-Authored-By: Claude Fable 5 --- notes/36_evidence_ledger.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) (limited to 'notes/36_evidence_ledger.md') diff --git a/notes/36_evidence_ledger.md b/notes/36_evidence_ledger.md index 14af7df..ed952bf 100644 --- a/notes/36_evidence_ledger.md +++ b/notes/36_evidence_ledger.md @@ -146,9 +146,12 @@ what they are; (31) and (25) deserve explicit mention in the paper. - MNIST e0 + estimator: `outputs/real_data_validation_mnist/` (script `scripts/real_data_validation.py`, self-test verifies kernels vs autograd at 1e-15). -- Teacher-task test gap (SGD, widths 8..96): `outputs/teacher_test_gap_sgd_T8000/` - — addresses "what about generalization": train-gap story plus a descriptive - test-MSE companion. Random-label sweeps measure optimization gap by design. +- Teacher-task test gap (note 39, SGD, widths 8..96, 24 FA runs/width): + train gap soft and monotone (0.249 -> 0.0069) while the **test gap changes + sign** — +0.177 (w=8), **-0.120 +/- 0.013 (w=32, FA generalizes better)**, + +0.045 (w=96). The optimization gap is the invariant cost; its test-side + consequence is task-dependent — exactly why the theory is built on the + former. `outputs/teacher_test_gap_sgd_T8000/teacher_train_vs_test_gap.png`. ## 6. Figure plan (sources verified on disk) -- cgit v1.2.3