diff options
| -rw-r--r-- | RESULTS.md | 26 | ||||
| -rw-r--r-- | ROADMAP.md | 4 |
2 files changed, 29 insertions, 1 deletions
@@ -158,6 +158,32 @@ slower than forward learning: once teaching and traffic are separated by neutral tracking is beneficial, while a long warmup is not consistently useful. This is a frozen validation choice for the multi-seed C1 experiment, not evidence that C1 has passed. +## Causal-calibration query screen (validation only) + +This seed-0 development screen used CIFAR-10 d20/w64 for five epochs and a 5,000-example +training-only validation split (hash `8328b206...515b`). All eight runs came from clean revision +`1a22a99`; the test set was never evaluated. Gain retention is +`(candidate - DFA) / (K16/e4 - DFA)` at the final validation endpoint. + +| method | directions K | every | val (%) | batch loss queries | query reduction | calibration work reduction | gain retention | +|:-------|-------------:|------:|--------:|-------------------:|----------------:|---------------------------:|---------------:| +| DFA | — | — | 36.66 | 0 | — | — | — | +| learned NP | 16 | 4 | 37.46 | 14,080 | 1.0x | 1.0x | 1.000 | +| **learned NP** | **1** | **4** | **38.08** | **880** | **16.0x** | **11.0x** | **1.775** | +| learned NP | 2 | 8 | 37.32 | 880 | 16.0x | 13.2x | 0.825 | +| learned NP | 4 | 16 | 35.98 | 880 | 16.0x | 14.7x | -0.850 | +| learned NP | 1 | 8 | 36.70 | 440 | 32.0x | 22.0x | 0.050 | +| learned NP | 1 | 16 | 36.52 | 220 | 64.0x | 44.0x | -0.175 | +| learned NP | 2 | 32 | 36.78 | 220 | 64.0x | 52.8x | 0.150 | + +`K=1/every=4` is frozen for confirmation: it exceeds the C3 10x-query/90%-retention threshold in +validation and is the only candidate that improves on the reference endpoint. The ablation also +separates directions from update frequency: spending the same 880 queries as `K=2/every=8` or +`K=4/every=16` is worse, so frequent noisy regression targets are more useful than precise sparse +targets. GTX-1080 wall time remains approximately 15–23 seconds across the grid despite the 11x +calibration-work change, demonstrating why the Pareto audit needs hardware-independent work in +addition to wall time. C3 is not declared passed until frozen multi-seed confirmation. + ## Legacy credit assignment / depth utility Per-layer alignment cos(r_l, −∇_{h_l}L), early-third average, CIFAR no-BN residual: | depth | DFA early-align | SDIL early-align | @@ -65,7 +65,9 @@ must hold in a hardware-independent cost coordinate as well as GTX-1080 wall tim The validation screen is frozen to CIFAR-10 d20/w64, five epochs, seed 0, and compares the current `K=16/every=4` reference with `K/every` pairs `1/4`, `2/8`, `4/16`, `1/8`, `1/16`, and `2/32`, plus DFA. Selection uses final validation accuracy and logical batch loss queries; the test split is not -evaluated. The chosen low-query protocol then receives a multi-seed frozen confirmation. +evaluated. `K=1/every=4` was selected: it used 16x fewer batch loss queries and retained 177.5% of +the reference gain over DFA. This selection is now frozen; the protocol receives a five-seed test +confirmation with no further tuning before C3 can be declared passed. ### C4. The comparison set contains the closest alternatives |