summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorYurenHao0426 <Blackhao0426@gmail.com>2026-07-22 03:18:54 -0500
committerYurenHao0426 <Blackhao0426@gmail.com>2026-07-22 03:18:54 -0500
commit5789cf1358542a3216761e578d5d87e021dec35c (patch)
tree98cce6049a1b87609923e89bf60376b475068184
parent9c9da14974da85c4f06853a7da882601af6963c5 (diff)
results: mark useful-depth BP screen passed
-rw-r--r--RESULTS.md22
-rw-r--r--ROADMAP.md5
2 files changed, 27 insertions, 0 deletions
diff --git a/RESULTS.md b/RESULTS.md
index 14c9f2b..d98e89d 100644
--- a/RESULTS.md
+++ b/RESULTS.md
@@ -276,6 +276,28 @@ DFA's teaching signal to early/deep layers collapses toward random (~0.05–0.19
SDIL stays ~0.85 and is depth-invariant. This is the "does credit reach early layers" metric that
DFA fails and SDIL passes — the core advantage.
+## C2 BP useful-depth screen (validation only)
+
+The frozen candidate uses a two-level Telgarsky tent-map task, width-8 ReLU residual networks,
+and three independently generated task/data seeds. Student initialization is fixed at seed 0 for
+this shape screen. All 15 runs came from clean revision `76688af`, used stratified validation
+subsets drawn only from generated training data, and never evaluated the independent test sets.
+
+| depth | validation acc (%) | task-seed endpoints (%) | final-third lesion drops (points) |
+|------:|-------------------:|:------------------------|:---------------------------------|
+| 1 | 63.933 ± 10.272 | 75.25, 55.20, 61.35 | — |
+| 2 | 60.883 ± 7.243 | 57.85, 69.15, 55.65 | +14.25, +24.75, +15.10 |
+| 3 | 97.250 ± 1.097 | 96.30, 97.00, 98.45 | +58.80, +58.75, +60.15 |
+| 4 | 96.883 ± 1.457 | 97.35, 98.05, 95.25 | +40.85, +35.05, +34.85 |
+| 6 | 90.733 ± 7.743 | 96.35, 93.95, 81.90 | +40.05, +40.35, -2.10 |
+
+For the preselected d1-to-d4 comparison, paired BP gains are `+22.10`, `+42.85`, and `+33.90`
+points (mean `+32.95`). The corresponding final-third lesion drops average `36.92` points and are
+positive for every task seed. **The BP useful-depth sub-gate passes.** The nonmonotonic curve is
+also informative: d2 can optimize worse than d1, and d6 is less stable than d3/d4. The claim is
+therefore about a verified useful-depth pair, not monotonic improvement from adding arbitrary
+blocks.
+
## Paper-faithful non-backprop baselines
`PETITE` in the initial request is treated as **PEPITA**. Implementations and hyperparameters were
diff --git a/ROADMAP.md b/ROADMAP.md
index 308cce8..8b73deb 100644
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@ -105,6 +105,11 @@ replace d4 after results are observed. The BP screen passes only if mean d1-to-d
5 points, every task seed gains, mean d4 final-third-lesion drop is at least 2 points, and every
task seed is hurt by the lesion. Only final validation metrics are observed.
+**BP useful-depth status: passed on 2026-07-22.** Across task seeds 0--2, d1-to-d4 gains were
+`+22.10`, `+42.85`, and `+33.90` points (mean `+32.95`). Bypassing the final third of d4 blocks
+reduced accuracy by `40.85`, `35.05`, and `34.85` points (mean `36.92`). The contextual d6 curve
+was less stable, so the preselected d4 endpoint remains fixed.
+
If BP passes, the local-rule validation panel freezes d1/d4 for BP, FA, DFA, and no-traffic SDIL
with C3's `K=1/every=4`, crossing the same three task seeds with student seeds 0--4. Its test panel
may run only if the original C2 gate holds on validation and the d4 SDIL lesion loses at least