summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2026-03-24Add CIFAR deltaL test (failed) and pivot design memoYurenHao0426
- CIFAR deltaL: s=grad_hL CE (dim=512) -> acc=17.2%, Gamma≈0 Confirms scalar value field has dimensionality bottleneck on CIFAR - Pivot memo: direct vector credit field a_phi(h,t,s) -> R^d Trained with perturbation-based target, avoids curvature problem Still satisfies no hidden BP anchor constraint Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24Add exploration visualization: CIFAR depth scan, boundary ablation, synth vs ↵YurenHao0426
CIFAR gap Three new plots: - cifar_depth_scan.png: acc/Gamma/rho vs depth for all methods - boundary_ablation.png: s_type, tgw, warmup ratio sweeps - synth_vs_cifar.png: dimensionality gap comparison (d=128 vs d=512) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24Add Phase 3 boundary-condition ablation results and combined memoYurenHao0426
Key findings: - deltaL (output-layer gradient) gives best Gamma (0.562 vs 0.452 for eT) - Concatenating h_L to s destroys credit quality (value net cheats) - Terminal gradient matching is monotonically beneficial - Best config: deltaL + tgw=1.0 + wr=0.05 -> Gamma=0.768, rho=0.691 - CIFAR depth scan shows no Goldilocks regime (dimensionality bottleneck) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23Add Phase 1 synthetic ladder results and memoYurenHao0426
Key finding: credit bridge advantage scales with nonlinearity. At alpha=1.0 (full tanh), CB > SB > DFA on both Gamma and rho at all depths. The crossover where CB surpasses SB happens around alpha=0.7-1.0. Full 4x4x3 grid complete with 3 seeds each. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23Add Phase 2 explore experiments: synthetic nonlinearity ladder + CIFAR depth ↵YurenHao0426
scan - synth_nonlinearity_ladder.py: teacher-student with phi_alpha(z) = (1-a)z + a*tanh(z) Sweeps alpha x depth to find where state bridge / credit bridge fail - cifar_depth_scan.py: CIFAR-10 with L={2,4,6,8,12}, d={256,512} Finds Goldilocks regime for credit bridge vs DFA - plot_synth_ladder.py: phase diagram visualization Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23Add sweep results confirming terminal gradient matching is essentialYurenHao0426
12-config sweep: no hyperparameter combination recovers useful credit gradients without terminal gradient matching (best cos ~0.3 early, decays to ~0).
2026-03-23Add final report, plots, experiment guide, and complete NOTE.mdYurenHao0426
All experiments complete: - Toy LQ: credit bridge matches state bridge (~0.94 costate cosine) - CIFAR-10: credit bridge (29.6%) comparable to DFA (30.0%), both beat state bridge (18.5%) - State bridge confirms core hypothesis: perfect state prediction != useful credit - Terminal gradient matching is essential for credit bridge
2026-03-23Add experiment notes and .gitignoreYurenHao0426
Track experiment phases (debug/pilot/frozen), key findings, and design decisions.
2026-03-23Sync state bridge: use normalized MSE target in both toy and CIFARYurenHao0426
Reason: toy used raw MSE, CIFAR used normalized. They must be the same method for consistent reporting. Normalized MSE is more robust to varying h_L magnitudes.
2026-03-23Initial implementation: all models, methods, toy and CIFAR experimentsYurenHao0426
Debug phase. Toy LQ experiments (3 seeds) complete with terminal gradient matching. Credit bridge matches state bridge on linear system (~0.94 cosine). CIFAR experiments in progress.