diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 04:51:49 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 04:51:49 -0500 |
| commit | d344f7fa4b8a6b907d364d1900348ca8e4b709ce (patch) | |
| tree | 13fcf933e308c918c8047aa343787a662044e962 /RESULTS.md | |
| parent | e8d698e9da1e2441d1d137eb8f8ab6cd46b4282c (diff) | |
results: confirm C2 feedback amortization bottleneck
Diffstat (limited to 'RESULTS.md')
| -rw-r--r-- | RESULTS.md | 32 |
1 files changed, 32 insertions, 0 deletions
@@ -354,6 +354,36 @@ nonfinite loss and chance accuracy. **The independent recovery gate fails and C2 prohibited.** A richer vectorizer can improve mean credit without curing coupled A/W basin instability; this is now a documented limitation rather than an oral-level scaling result. +## C2 direct node-perturbation causal diagnosis (validation only) + +The failed context-vectorizer panel leaves two distinct explanations: the local eligibility/update +rule may be unable to solve the compositional task, or the learned apical vectorizer may fail to +amortize a causal direction that would work if it were available. To separate them, direct node +perturbation replaces every hidden teaching vector by an unamortized antithetic layerwise estimate +on every update. A task-seed-0 development screen selected `eta=0.03` and K16; the independent +panel then reused the exact untouched validation splits for task seeds 3--5 and crossed d1/d4 with +five model seeds. All 30 runs came from clean revision `e8d698e`; test sets remain unevaluated. + +| method | depth | validation acc (%) | d1-to-d4 gain | d4 lesion drop | +|:-------|------:|-------------------:|--------------:|---------------:| +| BP | 1 | 68.633 ± 5.351 | — | — | +| BP | 4 | 97.187 ± 1.532 | +28.553 | +34.170 | +| context-SDIL | 1 | 63.977 ± 8.636 | — | — | +| context-SDIL | 4 | 79.120 ± 22.765 | +15.143 | +27.150 | +| direct NP | 1 | 65.703 ± 8.800 | — | — | +| direct NP | 4 | 96.150 ± 1.719 | +30.447 | +25.743 | + +Direct NP recovers `106.6%` of BP's mean depth gain. All 15 paired runs improve with depth, 14/15 +are harmed by the frozen final-block lesion, all final losses are finite, and its d4 endpoint beats +context-SDIL by `17.030` points and the stronger FA/DFA endpoint by `15.387` points. **The frozen +causal diagnosis passes: the local eligibility rule and a forward-only causal direction are +sufficient here; learning a reliable amortized apical vectorizer is the present bottleneck.** + +This is a diagnosis and upper-cost baseline, not the proposed scalable method. At d4 it uses a mean +`82,560,000` per-example scalar loss evaluations and `43,757,037` forward-equivalent examples, +`68.4x` ordinary training work, with `108.2` seconds mean wall time. The next recovery must retain +the vectorizer and disclose how much of this query burden it avoids. + ## Paper-faithful non-backprop baselines `PETITE` in the initial request is treated as **PEPITA**. Implementations and hyperparameters were @@ -419,6 +449,8 @@ Five-depth scaling: `experiments/figure_scaling_sweep.sh`; canonical local basel `experiments/ep_original_sweep.sh`; exact d1 comparison: `experiments/ep_archmatch_sweep.sh`; canonical d2 comparison: `experiments/ep_depth2_sweep.sh`; near-parameter comparison: `experiments/ep_matched_sweep.sh`. +C2 direct causal diagnosis: `experiments/c2_nodepert_validation.sh` followed by +`python experiments/analyze_c2_nodepert_validation.py`. Audited aggregate tables: `experiments/analyze_verified.py`. The claim-locked finalizer is `experiments/finalize_claims.sh`; it rejects missing/dirty inputs, regenerates both main figures, their captions, source-hash manifest, and `results/audited_tables.md`, then runs all smoke checks. |
