| Age | Commit message (Collapse) | Author |
|---|
|
(matches bibitem)
|
|
table), not 30ep 0.308±0.014
|
|
|
|
per-round narration
Codex round 37 verdict: 'Page budget is the bottleneck, not mechanism uncertainty.'
Mode 1 is mechanism-complete after rounds 32-36. Strict compression rule:
'one claim sentence per falsified alternative, one for the positive mechanism,
everything numeric goes to appendix tables.'
§3 rewrite (4 -> 3 paragraphs):
1. Phenomenon class — 6-line geometric argument inlined as one sentence;
LN Jacobian derivation for (b); empirical anchors for vanilla DFA.
2. Falsification chain — 4 alternative attributions each in one sentence:
not residual-skip-driven (App H), not task-signal-driven (App I),
not DFA-specific (App I), not shared by EP.
3. Positive necessity for (b) — same-backbone no_outln control with full
numbers; cross-architecture support; temporal early-fire result.
Result: main content 9 -> 8 pages (1 page of slack restored).
Total 15 -> 14 pages.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
|
Two changes from round 36:
1. §3 paragraph 3: replace 'observational association' with full causal claim
based on existing April 7 no-out_ln data (3 seeds, ResMLP-d256+terminal-LN
removed, residual skip kept): ||h_L||=1.21e7 (Mode 1 (a) still fires) but
||g_L||=7.4e-4 (HEALTHY, ~10000x above floor — (b) eliminated). Final acc
0.327±0.013 indistinguishable from vanilla DFA's 0.308±0.014. Wording
upgraded to 'terminal LayerNorm is necessary for Mode 1(b) in the audited
residual ResMLP and ViT-Mini setting'.
2. §3 paragraph after random-target ablation: add EP under random targets
smoke result (||h_L||=586 at ep 5 vs DFA's 14510 at ep 3, 25x gap).
Random-target assay now cleanly separates fixed-feedback methods (explode)
from EP (bounded). Cross-method negative control complete.
- experiments/ep_baseline.py: add --random_targets flag + train_ep parameter
- v2.5 paper compiles to 15 pages, main content 1-9 (right at E&D limit)
Combined picture (rounds 32-36):
- Mode 1 (a) localized to 'fixed-feedback local-credit objectives without
scale control on architectures absorbing scale at output'. Falsified:
residual skip (round 33), task signal (round 34), DFA-specific (round 35).
EP is the working negative control (round 36).
- Mode 1 (b) localized to terminal LayerNorm via the 1/||h|| Jacobian.
Causally established by April 7 no_outln 3-seed data.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
|
targets)
H2 100ep: ||h_L||=1.06e8, ||g_L||=1.09e-10 (below (b) floor)
Random-target DFA 100ep: ||h_L||=1.67e8, ||g_L||=8e-12 (worse than vanilla)
Both fully confirm the smoke-test trends at converged training horizons.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
|
- experiments/cifar_resmlp.py: add --methods filter and --random_targets flag;
extend compute_diagnostics to log hidden_norms_per_layer and bp_grad_norms_per_layer
- paper/main.tex §3 ¶1: broaden random-target finding to all 3 fixed-feedback methods
(DFA: ||h_L||=14510, SB: ||h_L||=6225, CB: ||h_L||=19974 at ep 3, all at chance acc)
- paper/main.tex Appendix J: extended with cross-method smoke-test table
This generalizes the §3 mechanism story from 'DFA-specific' to 'all 3 audited
fixed-feedback local-credit methods'. Combined with rounds 32-34, the proximate
cause of Mode 1 (a) is now well-localized:
- Not requires residual skip (round 33 H2 walkback)
- Not requires task signal (round 34 random targets, DFA)
- Not DFA-specific (round 35 random targets, SB+CB)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
|
Codex round 34 picked OPTION A (i.i.d. random class targets per minibatch) over the
analytic-only OPTION D as the most discriminating test of 'is (a) intrinsic to DFA
update geometry or task-driven?'. Smoke test result is unambiguous:
ep 0: ||h_L||=8.9 ||g_L||=9.8e-4
ep 1: ||h_L||=1616 ||g_L||=5.1e-6
ep 2: ||h_L||=9768 ||g_L||=8.5e-7
ep 3: ||h_L||=14510 ||g_L||=5.6e-7 (test acc still at chance ~0.07)
Three orders of magnitude growth in ||h_L|| in 3 epochs, three orders of magnitude
collapse in ||g_L|| in the same 3 epochs, with NO task signal whatsoever — DFA's
local-loss geometry is the proximate driver, not data adaptation.
- experiments/snapshot_evolution_residual_explosion.py: add --random_targets and
--skip_bp flags
- paper/main.tex §3 ¶1: replace 'no explicit scale constraint' framing with codex
round 34's 6-line geometric argument and the random-target empirical falsifier
- paper/main.tex Appendix J: full smoke-test table + interpretation
- v2.3: 14 pages total, main content still 8 pages
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
|
hypothesis
- models/residual_mlp.py: add residual_add and w2_std flags (default unchanged)
- experiments/snapshot_evolution_residual_explosion.py: add --no_residual_add and --w2_std CLI flags
- paper/main.tex §3 ¶3: add 1-sentence reference to no-residual control showing Mode 1 still fires
- paper/main.tex Appendix I: full smoke-test table + interpretation
- v2.2 main content stays at 8 pages (within 9-page E&D budget); 13 pages total
Smoke test (3 ep, w2_std=0.5, seed 42):
- DFA no-residual: ||h_L|| 4.69 -> 22050, ||g|| 1.6e-7 (Mode 1 (a) fires; (b) at floor)
- BP no-residual: acc only 0.16 at ep 3 (architecture is partially degenerate)
- Conclusion: residual skip is NOT necessary for Mode 1; the proximate trigger is more general
- Codex round 33 verdict: WALK BACK H2; demote 100ep run to confirmatory
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
|
L=2,4,6,8,12 data
The cifar_depth_scan_s42 results were already on disk but not surfaced in
the paper. Across L in {2,4,6,8,12} on the d=512 ResMLP, DFA's layer-0
cosine stays in [+0.39,+0.40] and its mean deep cosine stays within
[-0.005,+0.000], while BP retains a deep cosine of +0.94 even at L=12.
This rules out the 'too deep to receive useful credit' explanation:
making the network shallower does not reach the deep blocks any better.
- §5 paragraph 4: one-sentence depth-invariance summary citing the new appendix
- New Appendix H: Depth-Sweep Layerwise Profiles, with full table
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
|
content-complete
Six paragraphs total via codex round 31:
- §6 §6.1: measurement-validity-first ordering with 63x and 24338x calibration gaps
- §6 §6.2: minimal four-check protocol; decision-utility 0/5 vs 3/5 walk-back
- §6 §6.3: conservative asymmetry (BP/EP preserved, DFA/SB/CB walked back)
- §7 §7.1: scope claim — evaluation failure not algorithmic impossibility
- §7 §7.2: limits — CIFAR-10 only, observational LN interpretation, lower-bound BP+penalty control
- §7 §7.3: lesson — decompose evaluation question, position vs Jordan/O'Bray/Paleka
Compiles to 12 pages (main content 1-8, refs+appendices 8-12), within E&D 9-page main budget.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
|
|
|
|
|
|
|
Codex round 27 produced 4 substantive paragraphs for §2, replacing thin
placeholders. Each paragraph follows round 23's prescription:
P1: canonical setting (4-block d=256, AdamW, 100 ep, 3 seeds) +
table/figure references
P2: under field-standard reporting, all 5 methods look fine
P3: EP internal comparison — same trustworthy measurement regime BUT
EP depth contribution is also marginally negative (-3.3 pp vs
frozen baseline). Honest about EP being trustworthy-measurement
but neutral-depth-contribution (per round 27 prompt's caveat).
P4: frozen-baseline comparison gives the walk-back: BP +26.6 pp, DFA
-4.3 pp, SB -14.4 pp, CB -6.0 pp. Diagnostic split lines up with
acc split.
Compiles cleanly. Next: §3 Failure Mode 1 prose via round 28.
|
|
Codex round 26 produced 3 substantive paragraphs for §1, replacing the
3 thin placeholder sentences. Each paragraph follows round 23's
prescription:
P1: claim sentence + numerical evidence (DFA 0.306 < frozen 0.349;
layer-0 +0.42 vs deep ~0; ||g_L|| ~ 5e-10 < eps clamp 1e-8) +
closing 'measurement regime must be valid'
P2: 5-method audit shows the two modes; intervention dissociation
(lambda=1e-4 alleviates Mode 1 not Mode 2; vanilla ep 1 has
meaningful ||g|| but deep cos still ~0) + closing
P3: methodological contribution framing + cite Paleka, O'Bray,
Jordan + closing roadmap
Compiles cleanly. PDF still has §2-§7 with topic sentences only (TODO
next via per-section codex rounds).
|
|
Tables filled with real values:
Table 1: 5-method audit (3-seed mean ± std for acc, headline Γ, verdict)
Table 2: 4-condition mode 2 validation (cos and ρ values from existing
checkpoint measurements)
Table 3: protocol thresholds (50×, 1e-7, 0.30, 2pp)
Figures generated from existing data:
fig2_decision_utility.pdf: 5×7 verdict heatmap from
results/protocol_audit/ablation_decision_utility.json
fig4_penalty_rescue.pdf: 3-panel — trajectory + cos/ρ bars + 2×2 acc
from snapshot_evolution_v2 + dfa_residual_penalty + bp_with_penalty
fig5_cross_arch_summary.pdf: 5×4 BP/DFA verdict matrix across
architectures
Compiles to 8 pages with all tables/figures rendered. §1-§7 main body
still has only paragraph topic sentences (TODO: per-section prose
filling via codex). Figure numbering is wrong (codex put figures in
section order not numerical order — need fixing).
|
|
Round 24's skeleton had 3 deviations from round 23 redo:
- Made §3 'Diagnostic Protocol' instead of 'Failure Mode 1'
- Collapsed Mode 1 + Mode 2 into one §4
- Added §6 'Reference Implementation' (was supposed to be dropped)
Round 25 fixed all three. New §3-§7 match round 23 redo exactly:
§3 Failure Mode 1: Measurement Degeneracy
§4 Failure Mode 2: Low Intrinsic Credit-Direction Quality
§5 Intervention and Cross-Architecture Evidence
§6 Recommended FA Evaluation Protocol
§7 Discussion, Limits, Conclusion
Also added:
- In-line bibliography with 12 \bibitem entries (Paleka, O'Bray, Jordan
+ FA literature) — citations resolve correctly now
- Appendices A-G with actual prose content (not just headers)
- 7-pitfall catalog with descriptions
- Walk-back chain methodology paragraph
- 7-validation summary table
Compiles to 9 pages with figures 1+3 inline (existing PNGs) and figures
2/4/5 as placeholder text PDFs (TODO: regenerate). Tables 1/2/3 still
have TODO placeholders for numerical values.
Next: fill in tables 1-3 with existing JSON data, generate figures 2/4/5
from existing data, then consult codex per-section for prose filling.
|
|
User rejected the v1 draft as '流水账实验报告' (sequential experiment
report). Round 22 + 23 redid the outline with E&D-genre prescription.
Saving v1 as v1_rejected.tex for reference. New main.tex will be
written from round 24 LaTeX skeleton (codex offered to provide it),
section by section, with codex check on each section's prose.
|
|
Compiled with tectonic (the only LaTeX engine on this server). Two
fixes needed:
1. Pass [numbers,compress] to natbib via PassOptionsToPackage so the
numerical bibliography style works
2. Use bibstyle 'abbrvnat' instead of 'plain' (compatible with natbib)
Result: 10-page PDF, ~7.5 content pages (well under 9-page E&D limit),
references on pages 8-9, appendices A-D on pages 9-10.
PDF uploaded to broker as 1843506b_main.pdf for user review.
|
