summaryrefslogtreecommitdiff
path: root/results
AgeCommit message (Collapse)Author
2026-04-08paper v2.32: BP+penalty multi-seeded (was single-seed s42)YurenHao0426
The §5 ¶3 BP+penalty value (0.530, +18.1 pp margin) was single-seed s42. Ran s123 and s456 to multi-seed it, matching the BP-no-pen 3-seed control. 3-seed BP+pen 30ep results (lam=0.01, AdamW lr=1e-3 wd=0.01, cosine, batch 128): s42: 0.5303, +18.13 pp vs frozen s123: 0.5262, +17.72 pp s456: 0.5397, +19.07 pp 3-seed mean: 0.5321 ± 0.0057, +18.31 pp Updates: - §5 ¶3: BP+pen "0.530 (single seed)" → "0.532 ± 0.006" (3-seed) - §5 ¶3: BP penalty cost -5.5 pp → -5.3 pp - §5 ¶3: BP+pen margin +18.1 → +18.3 pp - §5 ¶3: BP-to-DFA gap 17.0 → 17.2 pp - §4 ¶4: BP+pen +18.1 → +18.3 pp comparison - Figure 3 panel C bar values: BP with_pen 0.530 → 0.532 - Figure 3 panel C title: BP-pen-cost -5.5pp → -5.3pp The +18.3 pp 3-seed mean is essentially the same as the s42 single-seed +18.13 pp, so the headline conclusion (BP+pen far above frozen baseline, huge gap vs DFA+pen) is unchanged. This commit removes the last single-seed value labeled as a key control. New auditable file: results/bp_with_penalty_3seed_summary.json Page layout preserved: 9 pages main, refs p10, 0 overfull boxes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08Commit dfa_pen_short lam=1e-4 s123/s456 JSONs (auditable source for §5 ¶2)YurenHao0426
The §5 ¶2 lambda sweep claim "at λ=1e-4, three-seed mean ‖h_L‖≈2.2e4 and ‖g_L‖≈7.0e-7" depends on these three files: results/dfa_pen_short/dfa_pen_lam0.0001_s42.json (already committed) results/dfa_pen_short/dfa_pen_lam0.0001_s123.json (this commit) results/dfa_pen_short/dfa_pen_lam0.0001_s456.json (this commit) The s123 and s456 files were untracked. Committing them as part of the auditable source set for the §5 ¶2 lambda-sweep claim. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08Save threshold_sensitivity_output.txt for §6 ¶1 + Appendix E auditYurenHao0426
Re-ran protocol/examples/threshold_sensitivity.py and captured the full output. This is the auditable source for: §6 ¶1 "63× calibration gap" between healthy max-per-block growth (EP s123 = 11.0) and degenerate min (CB s123 = 694) §6 ¶1 "24,338× gap" between healthy min ‖g_L‖ (EP s123 = 1.02e-4) and degenerate max (CB s123 = 4.18e-9) Appendix E threshold robustness: at default (a)>50× and (b)<1e-7, all DFA/SB/CB seeds fire and all BP/EP seeds pass The threshold robustness band is wide: (a) verdicts only flip when threshold goes above ~700× (CB s123 boundary) or below ~6× (EP s123 boundary); (b) verdicts only flip when threshold passes above ~1e-4 (EP s123 boundary) or below ~5e-9 (CB s123 boundary). The chosen default thresholds (50× and 1e-7) sit in the middle of these wide bands, confirming the qualitative-stability claim in Appendix E. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08Save null_calibration_penalized_dfa.json for §6 ¶2 auditYurenHao0426
The §6 ¶2 fresh-B null control claim "deep cos +0.002 ± 0.022 (n=20 draws), per-layer stds 0.013-0.023" was verified against a fresh re-run of experiments/null_calibration_penalized_cos.py: training-Bs deep cos: +0.1627 (matches Appendix L row) fresh-Bs deep cos: +0.0022 ± 0.0220 (per-layer std avg, n=20) per-layer stds: [0.0125, 0.0221, 0.0162, 0.0229, 0.0228] (l0-l4) The "0.013-0.023" range matches the per-layer std range exactly. The "± 0.022" is the average per-layer std across deep layers (l1-l4). Saved as the auditable source. The script (experiments/null_calibration_ penalized_cos.py) can re-derive these values from the saved checkpoint in results/dfa_pen_short/dfa_pen_lam0.01_s42.pt. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08paper v2.31.8: Appendix I EP random-target ‖h_L‖ values from saved JSONYurenHao0426
Appendix I claimed EP random-target ‖h_L‖ "≈586 at 5 ep" and "≈2,085 at 100 ep" without a saved-JSON source. Re-measured on the saved checkpoints with consistent methodology (model.eval(), n=2048 test median), giving 557 (5 ep) and 2151 (100 ep). The ~5% discrepancy is likely model.train() vs model.eval() LN-batch-stats; the new values are reproducible. Saved results/ep_random_h_L_summary.json as the source of truth. The "26× smaller than DFA's 14,510 at 3 ep" comparison still holds (was "25×"; updated to "26×" with the new EP values). The fixed-feedback vs energy-based separation conclusion is unchanged. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08paper v2.31: matched 30-epoch BP/DFA controls (was unsourced 0.609/0.308)YurenHao0426
The §5 ¶3 BP-no-penalty value of 0.609 ± 0.004 and DFA-no-penalty value of 0.308 ± 0.014 turned out to be unsourced — they were carried over from a hardcoded comment in experiments/bp_with_penalty_control.py ("BP-trainable (3-seed mean): 0.609") that nobody had actually measured with a matched 30-epoch run. Ran the missing matched controls under the same recipe as BP+pen (lam=0, 30 epochs, AdamW 1e-3, wd 0.01, cosine schedule, batch 128, 3 seeds 42/123/456): BP no-pen 30ep: per-seed 0.5851, 0.5845, 0.5863 → 0.585 ± 0.001 (paper said 0.609 ± 0.004, off by 0.024) DFA no-pen 30ep: per-seed 0.3070, 0.2985, 0.2966 → 0.301 ± 0.005 (paper said 0.308 ± 0.014) Also re-grounded DFA+penalty 30ep using the dfa_pen_short 3-seed run (0.3593, 0.3610, 0.3604 → 0.360 ± 0.001), which is what the deep-cosine +0.155 figure was computed on. The paper had 0.363 ± 0.001 — that came from the 100-epoch run, not the 30-epoch run, so it was an apples-to- oranges comparison with BP+pen 30-ep. Paper changes (§5 ¶3): BP penalty cost: -8 pp → -5.5 pp DFA pen rescue: +5.5 → +5.9 pp DFA+pen margin vs frozen: +1.4 → +1.1 pp BP-to-DFA gap: 17 → 17.0 pp (unchanged) BP-to-SB gap: 7.7 → 7.7 pp (unchanged) BP-to-DFA gap is still the lower-bound credit-quality cost claim; 17 pp gap is unchanged in magnitude. Also updated: - §5 ¶1 prose: 0.363 → 0.360, 0.308 → 0.301 - §4 ¶4 prose: DFA+pen 0.363 → 0.360 - Appendix J Table 9 caption: 0.363 → 0.360, +9.0 → +9.3 pp gap to SB - Appendix L paragraph: +5.5 → +5.9 pp DFA penalty rescue - Figure 3 panel C bar values + title pen-cost annotation - New results/matched_30ep_control_summary.json as auditable record Page layout preserved: 9 main pages + refs p10, 18 total, 0 overfull. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08paper v2.30: fix layer-0 cosine numbers + add per-seed appendix MYurenHao0426
Found a numerical error in §4 ¶3: the layer-0 vanilla DFA cosines were listed as +0.42, +0.45, +0.39 across seeds 42/123/456 but the actual re-measurement on the saved early-epoch checkpoints gives +0.421, +0.436, +0.418 (the s456 value was off by 0.03). The deep-mean numbers in Table 2 (-0.008 ± 0.013) were already correct. Changes: - §4 ¶3: layer-0 trio updated to +0.42, +0.44, +0.42 across seeds and cite now points to a new per-seed appendix. - New Appendix M (Layer-0 Dominance): 6-row table of per-seed per-layer cosines on vanilla DFA early checkpoints (3 seeds × ep 1, 2), with per-layer ||g||. Documents the layer-0 dominance pattern that drives the headline aggregate Γ on these checkpoints. - results/vanilla_dfa_early_ckpts/per_layer_cos_3seed.json: machine- readable dump of all 6 measurements for future audit. - §7 compressed (~30 words trimmed across the closing paragraph) and Figure 3 width 0.92 → 0.82 to keep main content at exactly 9 pages after the appendix addition. Verified: 9 pages main + refs on p10, 18 total, 0 overfull boxes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08λ sweep on penalty strength: lam ∈ {1e-4, 1e-2, 1e-1} cos + rho resultsYurenHao0426
Round 19's #5 recommendation. Major new finding for the paper: | lam | acc | ||h_L|| | ||g_2|| | deep cos | deep rho | |-------|------:|--------:|--------:|---------:|---------:| | 0 | 0.308 | 4e8 | 5e-10 | -0.008 | -0.003 | | 1e-4 | 0.359 | 2.4e4 | 6.3e-7 | -0.022 | -0.004 | | 1e-2 | 0.363 | 4e4 | 1e-6 | +0.155 | +0.080 | | 1e-1 | 0.349 | 1.2e4 | 1.6e-6 | +0.131 | +0.067 | KEY: at lam=1e-4 the residual stream is contained AND ||g|| is healthy (mode 1 ALLEVIATED), but deep cos and rho are still essentially zero (mode 2 NOT alleviated). This is independent dissociation of the two modes via penalty strength: at weak penalty you get mode 1 fix WITHOUT mode 2 fix. Both metrics (cos, rho) agree at every lambda. Penalty strength has a non-monotonic effect on mode 2 alleviation: - lam=1e-4: too weak, mode 2 not alleviated (cos ~0) - lam=1e-2: sweet spot, cos +0.16, rho +0.08 - lam=1e-1: slightly over-constrained, cos +0.13, rho +0.07 This is the 7th independent validation of the two-mode separation, and the strongest one because it shows mode 1 alleviation WITHOUT mode 2 alleviation — the modes do not even respond to the same intervention strength.
2026-04-08Multi-seed vanilla DFA early-epoch cos: lock-in for round 19 disambiguationYurenHao0426
Round 20's minimal lock-in experiment: 3 seeds × {ep 1, ep 2} vanilla DFA cosine. Closes the 'single-seed fluke' objection. Vanilla DFA early-epoch deep cosines (l1-l4): | seed | ep | ||g|| | deep mean | |---|---|---|---| | 42 | 1 | 6.7e-7 | -0.025 | | 42 | 2 | 1.5e-7 | -0.038 | | 123 | 1 | 6.5e-7 | +0.002 | | 123 | 2 | 1.4e-7 | -0.006 | | 456 | 1 | 3.9e-7 | +0.000 | | 456 | 2 | 8.5e-8 | -0.009 | 3-seed mean at ep 1 (most meaningful regime): -0.008 ± 0.013 3-seed mean at ep 2: -0.018 ± 0.018 ALL 24 measurements (3 seeds × 2 ep × 4 deep layers) are in [-0.04, +0.02]. Compare to penalized DFA 3-seed mean +0.155 ± 0.025. The penalty CREATING deep alignment finding is now seed-robust. Three seeds × two early epochs all show vanilla deep cos essentially zero even when ||g|| is in the meaningful regime. This is the round 20 lock-in. Framing is locked.
2026-04-08BP+penalty control result: mode 2 (intrinsic credit quality) confirmed REALYurenHao0426
BP + lam=1e-2 ||f||^2 penalty trained for 30 epochs (s42): ep 30 final: test_acc 0.5303 margin vs DFA-shallow 0.349: +18.13 pp The 2x2 accuracy grid: no penalty with penalty BP 0.609 0.530 DFA 0.308 0.363 Penalty effect on BP: -8 pp (capacity regularization cost) Penalty effect on DFA: +5.5 pp (rescue from active harm) Mode 2 (intrinsic credit quality) is confirmed REAL by this control: even after the penalty's capacity cost, BP achieves +18 pp depth utilization. DFA under the same penalty achieves only +1.4 pp. The difference (~17 pp) cannot be attributed to capacity loss — it is genuine credit-quality cost of random feedback vs true backprop gradient. This validates the round 19 'two distinct failure modes' framing: mode 2 is not a penalty-induced regularization artifact.
2026-04-083-seed multi-seed verification of penalized DFA deep cos = +0.17YurenHao0426
| seed | l0 | l1 | l2 | l3 | l4 | layer-mean | |---|---:|---:|---:|---:|---:|---:| | 42 | +0.316 | +0.169 | +0.151 | +0.165 | +0.166 | +0.193 | | 123 | +0.333 | +0.093 | +0.155 | +0.178 | +0.177 | +0.187 | | 456 | +0.339 | +0.131 | +0.123 | +0.150 | +0.150 | +0.179 | 3-seed mean deep cos (l1-l4): ~0.155 ± 0.025 3-seed layer-mean: +0.186 ± 0.007 The +0.17 finding is rock-solid, combined with: - null calibration: training-Bs +0.16 vs fresh-Bs +0.002 - hypothesis B confirmed: vanilla early ep deep cos ~0 - 3-seed reproducibility (this commit) This is the §4 evidence for the paper's 'penalty creates partial deep alignment, partially alleviating mode 2'.
2026-04-08DISAMBIGUATION: vanilla DFA early-epoch checkpoints + cos measurementYurenHao0426
Round 19's #3 critical experiment. Trained vanilla DFA s42 for 5 epochs, saved checkpoint at each, then measured per-layer cos(e_T B^T, BP grad). Key trajectory of ||g_l|| during vanilla DFA training: ep 0: ~1e-3 (random init, healthy) ep 1: ~1.4e-6 (3 OOM drop, STILL above 1e-7 floor) ep 2: ~3e-7 (above floor) ep 3: ~1.3e-7 (above floor, barely) ep 4: ~7e-8 (BELOW floor) ep 5: ~4e-8 (well below floor) So ep 1, 2, 3 vanilla checkpoints are in the MEANINGFUL ||g|| regime. Cos measurement on those: ep 1: l0=+0.42, l1=+0.005, l2=-0.028, l3=-0.039, l4=-0.038 ep 2: l0=+0.44, l1=-0.002, l2=-0.040, l3=-0.055, l4=-0.054 ep 3: l0=+0.43, l1=+0.007, l2=-0.039, l3=-0.054, l4=-0.054 DEEP-LAYER COSINES ARE ESSENTIALLY ZERO AT EVERY VANILLA EPOCH, even when ||g|| is in the meaningful regime (ep 1: ||g||=6.7e-7). Compare to penalized DFA s42 at 30 ep: deep cos = +0.17. Hypothesis B confirmed: the penalty CREATED the deep-layer alignment. It is a training outcome of the regularization, not a measurement-regime revelation. Paper implications: there are two distinct failure modes after all, but they are not 'scale + direction'. They are: (1) Measurement degeneracy via terminal LN gradient cancellation (caught by diagnostic (b)) (2) Low intrinsic credit quality of random feedback even in the meaningful regime (caught by direct cos measurement) The penalty partially alleviates BOTH (residual stream contained AND deep alignment improved from ~0 to +0.17), but neither fully.
2026-04-08MAJOR: penalized DFA deep-layer cosine is +0.17, NOT zeroYurenHao0426
Direct deep-block credit measurement on penalized DFA s42 checkpoint (lam=1e-2, 30 epochs, just trained): per-layer cos(e_T B^T, BP grad) — TRAINING Bs, no eps clamp: l0: +0.316 (±0.188) ||g||=9.18e-7 ||a||=4.53 l1: +0.169 (±0.087) ||g||=8.87e-7 ||a||=4.57 l2: +0.151 (±0.084) ||g||=8.77e-7 ||a||=4.50 l3: +0.165 (±0.099) ||g||=8.73e-7 ||a||=4.64 l4: +0.166 (±0.098) ||g||=8.69e-7 ||a||=4.64 layer-mean: +0.193 Compare to vanilla DFA (existing measurement, scale-broken regime): l0: +0.42 l1-4: ~0 (essentially zero) CRITICAL INTERPRETATION: The penalty doesn't just fix scale, it ALSO restores deep-layer direction quality from ~0 to ~0.17. This contradicts the prior 'two failure modes' framing where I assumed direction would remain broken even after scale fix. The honest story is: - vanilla DFA: scale catastrophic, BP grad at floor, cosine measurement DEGENERATE (cos ~0 is noise dominance, not 'no alignment') - penalized DFA: scale fixed, BP grad healthy, cosine measurement INTERPRETABLE — and the value is +0.17 on deep layers (partially aligned, much less than BP's self-cosine of 1.0) - the +0.17 alignment explains why penalized DFA gets 0.36 (60% of BP's 0.61) — partial credit gives partial training, not zero training The 'second failure mode' claim is wrong. There's ONE unified failure mode (scale + measurement degeneracy), and the penalty rescues BOTH. The remaining gap to BP is 'partial credit quality', not a separate failure mode.
2026-04-08Add penalty λ sweep figure: shows λ-dependence of (d) verdictYurenHao0426
3-panel figure: vanilla DFA + penalty at λ=1e-3 (green) + penalty at λ=1e-2 (blue): (a) ‖h_L‖: vanilla 4e8, both penalties ~4e4 (similar) (b) ‖g_2‖: vanilla 5e-10, penalties 7e-7 to 1e-6 (above floor) (c) acc: vanilla 0.31, λ=1e-2 0.36, λ=1e-3 0.37; horizontal lines at DFA-shallow 0.349 and 2pp threshold 0.371 Visual: at λ=1e-3 the test acc curve crosses ABOVE the 2pp threshold line; at λ=1e-2 it stays below. This is the (d) lambda-dependence finding from the round 18 follow-up.
2026-04-07Add CNN third-architecture audit: BN, no terminal LNYurenHao0426
5 methods × 3 seeds on the SmallCNN (3 conv + BN + 1 FC + head, no terminal LN) using existing checkpoints in results/cnn_baseline/. Key findings: BP CNN: 0.866 acc, max/block 1.3, trustworthy State Bridge CNN: 0.633 acc, max/block 2.4, trustworthy EP CNN: 0.512 acc, max/block 12, trustworthy DFA CNN: 0.566 acc, max/block 237, walked back via (a) Credit Bridge CNN: 0.325 acc, max/block 96, walked back via (a) CRITICAL: diagnostic (b) ||g_L|| floor NEVER fires on CNN for any method. The deepest BP grad is at ~1e-5 to 6e-1, all well above the 1e-7 floor. This is the cleanest confirmation that terminal LayerNorm is the structural cause of the catastrophic gradient collapse in (b). Without out_ln, the BP grad does NOT collapse to the floor, even on DFA. The scale pathology (a) still appears on DFA and CB, but the gradient collapse pathology (b) is specific to terminal-LN architectures. DFA CNN's accuracy (56.6%) is much higher than DFA ResMLP (30.8%) or DFA ViT (23.7%) — partially because the scale pathology is less catastrophic without the LN-driven gradient cancellation amplifying it. This is the cross-architecture mechanism story made concrete.
2026-04-07Add §4 penalty rescue figure: visual two-failure-modes storyYurenHao0426
3-panel side-by-side showing per-epoch trajectories of vanilla DFA vs DFA + lambda*||f||^2 penalty: (a) ||h_L||: vanilla 4e8 vs penalty 4e4 (4 OOM rescue) (b) ||g_L||: vanilla 5e-10 vs penalty ~1e-6 (4 OOM rescue) (d) test acc: vanilla 0.31 vs penalty 0.36 vs frozen baseline 0.349 vs BP 0.61 The visual story: (a) and (b) show the penalty pulling the diagnostics back into the healthy regime, but (d) shows the rescue translates to only +1 pp above the DFA-shallow baseline and 24 pp below BP-trainable. The two failure modes (scale + direction) are visually separable: scale is fixed, direction is not. Together with figure_audit_5method.png and figure_cross_arch_temporal_s42.png, this is the third paper-ready figure for §3-§4.
2026-04-07Add §2/§3 hero figure: 5-method audit horizontal bar chartYurenHao0426
4-panel layout (one per diagnostic), 5 methods sorted bottom-to-top by ascending accuracy, color-coded healthy (BP/EP, blue) vs degenerate (DFA/SB/CB, red), with threshold lines drawn: (a) max per-block growth (log scale, threshold 50x) (b) ||g_L|| (log scale, floor 1e-7) (c) cross-batch stability (linear, ceiling 0.30) (d) headline acc (linear, frozen baseline 0.349) The visual layout makes it immediately obvious that: - (a) and (b) cleanly split healthy from degenerate (4-7 OOM gap) - (c) is bimodal and doesn't cleanly split — confirms it's a sub-mode discriminator, not a primary detector - (d) shows BP above the frozen baseline by ~25 pp while DFA/CB/SB are at or below it
2026-04-07Add d=512 ResMLP audit table (3 seeds): cross-width validationYurenHao0426
Same protocol applied to the 4-block d=512 ResMLP variant (vs the d=256 default). 4 methods × 3 seeds = 12 conditions: BP @ d=512: trustworthy on all 3 seeds (acc 0.60-0.61) DFA @ d=512: walked back on all 3 seeds via (a)+(b) State Bridge @ d=512: walked back on all 3 seeds via (a)+(b), with drift sub-mode on s123 (stability 0.879) Credit Bridge @ d=512: walked back on all 3 seeds via (a)+(b) Width effect: max-per-block growth is HIGHER at d=512 (6e3-7e4) than at d=256 (~1e3). Larger width amplifies the explosion. The protocol verdicts are robust to this — same binary outcome, more extreme quantitative numbers. This is the cross-width validation: the protocol's findings are not d=256-specific. The §3 audit results generalize across the width dimension.
2026-04-07Add §3 cross-architecture temporal evolution figureYurenHao0426
3-column 3-row plot: rows: ||h_L||, ||g_L||, test accuracy cols: ResMLP (with LN) | ViT-Mini (cls + LN) | StudentNet (no LN) BP and DFA trajectories overlaid. Floor threshold drawn on the ||g_L|| row. Visualizes the cross-architecture causal control: with-LN architectures both show ||g_L|| collapse below 1e-7 (DFA hits the floor within 5 epochs); without-LN architecture shows ||g_L|| stays in the healthy regime even though ||h_L|| still grows (catastrophic vs mild).
2026-04-07Cross-architecture temporal validation: 3 archs x 3 seeds x 2 methodsYurenHao0426
ResMLP (4-block d=256, with out_ln, CIFAR-10): s42: DFA (a) ep 8, (b) ep 4, acc 0.308 s123: DFA (a) ep 11, (b) ep 4, acc 0.320 s456: DFA (a) ep 8, (b) ep 3, acc 0.300 ViT-Mini (4-block d=128, cls token + terminal LN, CIFAR-10): s42: DFA (a) ep 1, (b) ep 3, acc 0.256 s123: DFA (a) ep 1, (b) ep 2, acc 0.202 s456: DFA (a) ep 1, (b) ep 3, acc 0.253 StudentNet (4-block d=128, NO terminal LN, synthetic alpha=1.0): s42: DFA (a) ep 18, (b) NEVER, acc 0.332 s123: DFA (a) ep 14, (b) NEVER, acc 0.314 s456: DFA (a) ep 25, (b) NEVER, acc 0.336 BP: never fires on any seed x any architecture (9/9 sanity passes). Key cross-architecture finding: diagnostic (b) is specifically the LN- driven failure mode. Without out_ln, the BP grad never crosses the 1e-7 floor, even though (a) still fires (the residual stream still grows, just without the LN-cancellation pathology that drives the BP grad to the floor). This is the causal architectural control: (b) specifically tests 'is terminal-LN gradient cancellation active?' and (a) tests 'is the residual stream growing without bound?'. They are linked but separable. This is the §3 cross-architecture validation evidence.
2026-04-07Protocol diagnostic (a): use max per-block growth, not max/min ratioYurenHao0426
Old metric: max(||h||) / max(||h_0||, eps). False-positives on ViT-style architectures because the cls token at layer 0 (right after patch_embed) has anomalously small magnitude (~0.3-1.5), inflating the ratio even on healthy BP-trained ViTs. New metric: max_l(||h_{l+1}|| / ||h_l||) — the largest single-block residual amplification. Architecture-invariant. Calibration: - BP-trained, late training: <5x per block - BP ViT, early epochs (cls token resolving): 13-25x max - DFA-trained ResMLP/ViT: 100-4000x per block Threshold raised from 10 to 50 to sit cleanly between healthy-early- training (max 25) and failure-regime (min 100). Re-verifications: - smoke test (BP/DFA/EP): all 3 verdicts unchanged - random init (3 seeds): trustworthy on all 3 - 5-method audit table single-seed: identical verdicts - decision-utility ablation: identical (still 0/5 by S1, 3/5 by S_full) - temporal evolution 3-seed: (b) now fires first at ep 3-4, (a) at ep 8-11. Both well before training ends. The 'protocol fires ~92 epochs early' story still holds. - ViT temporal evolution: BP no longer false-fires; DFA fires (a) ep 1, (b) ep 3 — protocol works on the second architecture.
2026-04-07Temporal evolution 3-seed: protocol fires at DFA epoch 3-4 on all seedsYurenHao0426
s42: (a)+(b) fire at epoch 4, DFA final acc 0.3076 s123: (a)+(b) fire at epoch 4, DFA final acc 0.3203 s456: (a)+(b) fire at epoch 3, DFA final acc 0.2998 BP never fires on any seed (final acc 0.61-0.63). The 'protocol catches it 96 epochs early' finding is fully reproducible across seeds.
2026-04-07Add temporal diagnostic evolution: protocol fires at epoch 4 of DFAYurenHao0426
Replays per-epoch logged data from results/snapshot_evolution_v2/ through the protocol thresholds. Result: diagnostics (a) ||h_l|| explosion AND (b) ||g_L|| at floor BOTH first fire at epoch 4 of DFA training. At that point, DFA test acc is 0.308 — its final value at epoch 100 is also 0.308. The protocol could have walked back the headline 96 epochs before training finished. DFA's gamma hovers at 0.087-0.107 for all 100 epochs. A reviewer looking at acc+gamma would conclude 'DFA is hovering at 31% acc with ~0.10 alignment, both reasonable'. Wrong on both counts. BP never fires any diagnostic at any epoch. Stays bounded at ||h_L||~200, ||g_L||~3-5e-5, accuracy climbs to 0.61. This is the temporal validation of decision utility: the protocol catches the pathology AS IT HAPPENS, not just retrospectively.
2026-04-07Audit table extension to 3 seeds (s42/s123/s456)YurenHao0426
3 seeds × 5 methods × 4 diagnostics = 60 measurements. Key reproducibility findings: - BP: trustworthy on all 3 seeds (acc 0.61-0.62, h_L ~200, g_L ~3-4e-4) - EP: trustworthy on all 3 seeds (acc 0.29-0.36, h_L 3-8e3, g_L ~1e-4) - DFA, SB, CB: walked back on all 3 seeds × all 3 of (a)/(b)/(d) Diagnostic (c) is bimodal across seeds — confirms the prior memory finding: - DFA s42=0.047 (noise), s123=0.436 (drift), s456=-0.005 (noise) - SB s42=0.992 (drift), s123=0.561 (drift), s456=0.035 (noise) - CB s42=0.352 (drift), s123=0.250 (~edge), s456=0.518 (drift) (c) catches different methods on different seeds. (a)/(b)/(d) catch all 3 failing methods on all 3 seeds — robust binary detection.
2026-04-07Add protocol decision-utility ablation tableYurenHao0426
Builds on the 5-method audit JSON. For each method, evaluates 7 reporting strategies (S0=acc only, S1=+Γ field standard, S2-S5=+single diagnostic, S_full=full protocol), and emits the verdict each strategy would have reached. Result: 3 of 5 methods (DFA/SB/CB) are walked back by S_full but NOT by S1. Each of (a)scale, (b)floor, (d)frozen is independently sufficient for binary detection of those 3 failures. Diagnostic (c)stability adds sub-mode discrimination (drift vs noise) but not new positive detections. This is the §3 protocol decision-utility evidence.
2026-04-07Add audit table example: protocol applied to BP/DFA/SB/CB/EPYurenHao0426
5-method audit table on 4-block d=256 ResMLP CIFAR-10 seed 42: - BP: trustworthy (acc 0.615, h_L=2e2, g_L=4e-4, stab 0.099) - DFA: walked back via (a)+(b)+(d) — h_L=4e8, g_L=4e-9, undercuts frozen - State Bridge: walked back via all 4 diagnostics — stability 0.992 is the cleanest possible drift-dominated case - Credit Bridge: walked back via all 4 — stability 0.352, also drift mode - EP: trustworthy (acc 0.359, h_L=3e3, g_L=2e-4, stab -0.036) — paper's internal control case This is the §2 audit evidence for the main-track paper. Confirms that standard headline acc + Γ silently fails on 3 of 5 methods on this architecture, while the 4-diagnostic protocol catches all three.
2026-04-03Add 5 extra seeds to synthetic cross-state distance (now 10 seeds for all ↵YurenHao0426
methods) BP/DFA/SB/CB: added seeds 2048,3000,4000,5000,6000 (L=4 only, all 3 alphas) Total: 1290 rows (was 990) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03Recompute all CNN diagnostics with fixed BP gradient flowYurenHao0426
CNN CIFAR-10 (5 seeds, fixed Gamma): BP: acc=86.8%, Gamma=0.970, rho=0.603 DFA: acc=56.7%, Gamma=0.896, rho=0.061 EP: acc=50.6%, Gamma=0.484, rho=0.450 SB: acc=63.3%, Gamma=1.000 (BP self-cos, feedback nets not saved), rho=0.601 CB: acc=31.8%, Gamma=1.000 (BP self-cos), rho=0.226 DFA Gamma=0.896 is notably high — CNN DFA credit aligns well with BP gradients. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03Complete EP data: 10-seed synthetic + 6-seed CIFAR persample + cross-stateYurenHao0426
EP synthetic: 30 JSONs + 30 checkpoints (10 seeds × 3α) EP CIFAR persample: 6 seeds × 4 layers × 256 samples = 6144 rows added Synth cross-state: 150 EP rows added (990 total) cifar_persample_all.csv: 30720 rows (was 24576, +6144 EP) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03EP synthetic 10 seeds complete: 30 JSONs + 30 checkpoints + cross-state distanceYurenHao0426
Updated synth_cross_state_distance.csv with 150 EP rows (990 total). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03Add EP cross-state distance for CIFAR + verify CNN summaryYurenHao0426
EP CIFAR d_BP: L0=2.0×, L4=26.7× (much closer to BP than DFA=162×/2.5M×) EP synthetic: no checkpoints saved (ep_synthetic.py didn't save .pt) CNN summary: 20 rows confirmed correct Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03Add EP synthetic per-seed CSV + synthetic cross-state distanceYurenHao0426
EP synthetic: 15 rows (3α × 5 seeds) Synth cross-state: 840 rows (3α × 2L × 4 methods × 5 seeds × (L+1) layers) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03Add EP to BP gradient sparsity analysisYurenHao0426
EP CIFAR d=256: s(1e-6)=100%, mean_norm=1.41e-04 EP produces networks where ALL samples have non-zero BP gradients, unlike DFA (0.4%), SB (21%), CB (3%). EP is closer to BP (98.7%). Updated clean_sparsity_summary.csv (980 rows, now includes EP). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03Add cross-method hidden state distance vs BPYurenHao0426
Non-BP methods produce radically different representations: DFA L0: 162×, L4: 2.5M× relative to BP hidden norms SB L0: 3.2×, L4: 1.1M× CB L0: 59×, L4: 1.4M× (BP vs itself = 0) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03Update all EP results with sign-corrected creditYurenHao0426
EP Synthetic (fixed): Gamma=+0.13~0.20, rho=+0.25 EP CIFAR d=256: Gamma=+0.007, rho=+0.051 EP CIFAR d=512: Gamma=+0.000, rho=-0.002 EP CNN: Gamma=+0.248, rho=+0.492 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02Recompute EP CIFAR d=256 diagnostics with sign fixYurenHao0426
EP d=256 (5 seeds): acc=31.9%, Gamma=+0.007 (was -0.13), rho=+0.051 (was -0.037) Sign correction: -(h_nudge - h_free)/β aligns EP credit with BP gradient direction. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02Add EP synthetic results: 15 JSONs (3α × 5 seeds)YurenHao0426
EP synthetic: acc high (92-96%) but Gamma negative (-0.13 to -0.20), rho=-0.25 EP credit direction may be inverted or diagnostics have issue. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02Add CNN SB+CB results (5 seeds each), update summary CSVYurenHao0426
CNN CIFAR-10 (5 seeds): BP: 86.8%±0.3%, Gamma=0.238, rho=0.250 DFA: 56.7%±2.0%, Gamma=0.216, rho=0.017 SB: 63.3%±0.5%, Gamma=0.045, rho=0.298 CB: 31.8%±6.2%, Gamma=0.013, rho=0.033 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02Fix and recompute GELU ablation Gamma from checkpointsYurenHao0426
ReLU MLP (L=4 d=256): BP: acc=61.1%, Gamma=1.000, rho=0.998 DFA: acc=30.7%, Gamma=0.104, rho=-0.001 SB: acc=15.5%, Gamma=0.300, rho=0.159 CB: acc=28.7%, Gamma=0.298, rho=0.007 Note: SB/CB Gamma uses BP gradient as proxy (feedback nets not checkpointed). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02Add summary CSVs for EP, GELU ablation, CNNYurenHao0426
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02Add EP baseline (5 seeds), GELU ablation (20 runs), CNN BP+DFA (10 runs)YurenHao0426
EP (L=4 d=256): acc≈30%, Gamma≈0, rho≈0 — EP credit signal weak on feedforward MLP GELU ablation (ReLU variant): 4 methods × 5 seeds complete CNN BP+DFA: 5 seeds each, BP + DFA on SmallCNN Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01Add d=512 support sparsity: 20 JSONs + summary CSVYurenHao0426
BP: s(1e-6)=92.7%, norm=2.70e-04, r_inf=0.159, PR=0.300 DFA: s(1e-6)=0.1%, norm=5.31e-08 SB: s(1e-6)=20.3%, norm=2.33e-06 CB: s(1e-6)=1.2%, norm=9.88e-08 Same pattern as d=256, confirming width-independence of the sparsity gap. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01Add missing bp_s456.json for CIFAR d=512 (rerun after SIGTERM)YurenHao0426
bp s=456: acc=0.5999, rho=0.9881, nse=0.4764 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01Add CIFAR L=4 d=512 confirmatory: 4 methods × 5 seeds with checkpointsYurenHao0426
BP: 60.6%±0.3%, rho=0.989 DFA: 30.8%±0.5%, rho=0.003 State Bridge: 21.2%±3.7%, rho=0.119 Credit Bridge: 30.1%±0.5%, rho=0.002 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01Add P3 protocol panel: method ranking across 5 protocol slicesYurenHao0426
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01Add per-sample gradient stats: 24576 rows (256 samples × 4 layers × 4 ↵YurenHao0426
methods × 6 seeds) Columns: method, seed, layer, sample_id, grad_norm, log10_grad_norm, r_inf, pr, hoyer, topk1, topk5 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01Add clean_sparsity_summary.csv: 960 rows aggregated from 168 JSONsYurenHao0426
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01Add clean sparsity results: 168 JSONs from independent processes on GPU 1YurenHao0426
CIFAR: 24 JSONs (4 methods × 6 seeds), BP s(1e-6)=98% confirmed Synthetic: 144 JSONs (4 methods × 6 seeds × 3 alphas × 2 depths) All data reliable — each method+seed in separate Python process. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01Add clean gradient check: independent Python process per method, GPU 1YurenHao0426
Clean results (each method in fresh Python process): BP: mean_norm=2.58e-04, s(1e-6)=98% — CONFIRMED DFA: layer 0 = 2.86e-07 (1.2%), layers 1-3 ≈ 2.4e-09 (0%) SB: layer 0 = 6.13e-06 (86%), layers 1-3 ≈ 1e-09 (0%) CB: layer 0 = 6.33e-07 (18%), layers 1-3 ≈ 5e-10 (0%) Method A (autograd.grad) and Method B (retain_grad) give identical results. Previous 1e-12 results were caused by Python process state pollution in combined scripts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01Add element-wise gradient concentration analysis (CPU, from checkpoints)YurenHao0426
BP gradients are relatively uniform: top1%=7.1%, PR=0.327, eff_dim=0.632 DFA gradients extremely concentrated: top1%=40.6%, PR=0.089, eff_dim=0.272 SB/CB intermediate: top1%=17-21%, PR=0.14-0.17 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
generated by cgit v1.2.3 (git 2.25.1) at 2026-06-15 04:59:40 +0000