summaryrefslogtreecommitdiff
path: root/RESULTS.md
diff options
context:
space:
mode:
authorYurenHao0426 <Blackhao0426@gmail.com>2026-07-22 04:51:49 -0500
committerYurenHao0426 <Blackhao0426@gmail.com>2026-07-22 04:51:49 -0500
commitd344f7fa4b8a6b907d364d1900348ca8e4b709ce (patch)
tree13fcf933e308c918c8047aa343787a662044e962 /RESULTS.md
parente8d698e9da1e2441d1d137eb8f8ab6cd46b4282c (diff)
results: confirm C2 feedback amortization bottleneck
Diffstat (limited to 'RESULTS.md')
-rw-r--r--RESULTS.md32
1 files changed, 32 insertions, 0 deletions
diff --git a/RESULTS.md b/RESULTS.md
index 0661b42..2affc24 100644
--- a/RESULTS.md
+++ b/RESULTS.md
@@ -354,6 +354,36 @@ nonfinite loss and chance accuracy. **The independent recovery gate fails and C2
prohibited.** A richer vectorizer can improve mean credit without curing coupled A/W basin
instability; this is now a documented limitation rather than an oral-level scaling result.
+## C2 direct node-perturbation causal diagnosis (validation only)
+
+The failed context-vectorizer panel leaves two distinct explanations: the local eligibility/update
+rule may be unable to solve the compositional task, or the learned apical vectorizer may fail to
+amortize a causal direction that would work if it were available. To separate them, direct node
+perturbation replaces every hidden teaching vector by an unamortized antithetic layerwise estimate
+on every update. A task-seed-0 development screen selected `eta=0.03` and K16; the independent
+panel then reused the exact untouched validation splits for task seeds 3--5 and crossed d1/d4 with
+five model seeds. All 30 runs came from clean revision `e8d698e`; test sets remain unevaluated.
+
+| method | depth | validation acc (%) | d1-to-d4 gain | d4 lesion drop |
+|:-------|------:|-------------------:|--------------:|---------------:|
+| BP | 1 | 68.633 ± 5.351 | — | — |
+| BP | 4 | 97.187 ± 1.532 | +28.553 | +34.170 |
+| context-SDIL | 1 | 63.977 ± 8.636 | — | — |
+| context-SDIL | 4 | 79.120 ± 22.765 | +15.143 | +27.150 |
+| direct NP | 1 | 65.703 ± 8.800 | — | — |
+| direct NP | 4 | 96.150 ± 1.719 | +30.447 | +25.743 |
+
+Direct NP recovers `106.6%` of BP's mean depth gain. All 15 paired runs improve with depth, 14/15
+are harmed by the frozen final-block lesion, all final losses are finite, and its d4 endpoint beats
+context-SDIL by `17.030` points and the stronger FA/DFA endpoint by `15.387` points. **The frozen
+causal diagnosis passes: the local eligibility rule and a forward-only causal direction are
+sufficient here; learning a reliable amortized apical vectorizer is the present bottleneck.**
+
+This is a diagnosis and upper-cost baseline, not the proposed scalable method. At d4 it uses a mean
+`82,560,000` per-example scalar loss evaluations and `43,757,037` forward-equivalent examples,
+`68.4x` ordinary training work, with `108.2` seconds mean wall time. The next recovery must retain
+the vectorizer and disclose how much of this query burden it avoids.
+
## Paper-faithful non-backprop baselines
`PETITE` in the initial request is treated as **PEPITA**. Implementations and hyperparameters were
@@ -419,6 +449,8 @@ Five-depth scaling: `experiments/figure_scaling_sweep.sh`; canonical local basel
`experiments/ep_original_sweep.sh`; exact d1 comparison: `experiments/ep_archmatch_sweep.sh`;
canonical d2 comparison: `experiments/ep_depth2_sweep.sh`; near-parameter comparison:
`experiments/ep_matched_sweep.sh`.
+C2 direct causal diagnosis: `experiments/c2_nodepert_validation.sh` followed by
+`python experiments/analyze_c2_nodepert_validation.py`.
Audited aggregate tables: `experiments/analyze_verified.py`. The claim-locked finalizer is
`experiments/finalize_claims.sh`; it rejects missing/dirty inputs, regenerates both main figures,
their captions, source-hash manifest, and `results/audited_tables.md`, then runs all smoke checks.