diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-04-02 14:53:44 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-04-02 14:53:44 -0500 |
| commit | 6e280e59d492203ea7f7765a65949a6c256bf73a (patch) | |
| tree | cde981720f6f977757de2dbeb570893ef7b8beb5 /results/gelu_ablation/relu_dfa_s789.json | |
| parent | c8407db32d3a159613ce5f0a567843ed4c970a27 (diff) | |
Fix and recompute GELU ablation Gamma from checkpoints
ReLU MLP (L=4 d=256):
BP: acc=61.1%, Gamma=1.000, rho=0.998
DFA: acc=30.7%, Gamma=0.104, rho=-0.001
SB: acc=15.5%, Gamma=0.300, rho=0.159
CB: acc=28.7%, Gamma=0.298, rho=0.007
Note: SB/CB Gamma uses BP gradient as proxy (feedback nets not checkpointed).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Diffstat (limited to 'results/gelu_ablation/relu_dfa_s789.json')
| -rw-r--r-- | results/gelu_ablation/relu_dfa_s789.json | 5 |
1 files changed, 2 insertions, 3 deletions
diff --git a/results/gelu_ablation/relu_dfa_s789.json b/results/gelu_ablation/relu_dfa_s789.json index bc16a09..9469fc0 100644 --- a/results/gelu_ablation/relu_dfa_s789.json +++ b/results/gelu_ablation/relu_dfa_s789.json @@ -3,8 +3,7 @@ "method": "dfa", "seed": 789, "acc": 0.3065, - "StateErr": null, - "Gamma": 1.0, - "rho": 0.012797923758625984, + "Gamma": 0.09626383980503306, + "rho": 0.007170299533754587, "naive_StateErr": 1.1228328943252563 }
\ No newline at end of file |