| Age | Commit message (Collapse) | Author |
|
Key findings:
- deltaL (output-layer gradient) gives best Gamma (0.562 vs 0.452 for eT)
- Concatenating h_L to s destroys credit quality (value net cheats)
- Terminal gradient matching is monotonically beneficial
- Best config: deltaL + tgw=1.0 + wr=0.05 -> Gamma=0.768, rho=0.691
- CIFAR depth scan shows no Goldilocks regime (dimensionality bottleneck)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
|
Key finding: credit bridge advantage scales with nonlinearity.
At alpha=1.0 (full tanh), CB > SB > DFA on both Gamma and rho at all depths.
The crossover where CB surpasses SB happens around alpha=0.7-1.0.
Full 4x4x3 grid complete with 3 seeds each.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
|
All experiments complete:
- Toy LQ: credit bridge matches state bridge (~0.94 costate cosine)
- CIFAR-10: credit bridge (29.6%) comparable to DFA (30.0%), both beat state bridge (18.5%)
- State bridge confirms core hypothesis: perfect state prediction != useful credit
- Terminal gradient matching is essential for credit bridge
|
|
Track experiment phases (debug/pilot/frozen), key findings, and design decisions.
|
|
Debug phase. Toy LQ experiments (3 seeds) complete with terminal gradient matching.
Credit bridge matches state bridge on linear system (~0.94 cosine).
CIFAR experiments in progress.
|