<feed xmlns='http://www.w3.org/2005/Atom'>
<title>faeval.git/experiments/measure_direction_quality_existing_ckpt.py, branch master</title>
<subtitle>Unnamed repository; edit this file 'description' to name the repository.
</subtitle>
<link rel='alternate' type='text/html' href='https://git.blackhao.com/faeval.git/'/>
<entry>
<title>Add fast direction-quality measurement on existing DFA checkpoints</title>
<updated>2026-04-08T04:21:32+00:00</updated>
<author>
<name>YurenHao0426</name>
<email>Blackhao0426@gmail.com</email>
</author>
<published>2026-04-08T04:21:32+00:00</published>
<link rel='alternate' type='text/html' href='https://git.blackhao.com/faeval.git/commit/?id=8f67bdeebac543961871b9896a62cd07b7a5be26'/>
<id>8f67bdeebac543961871b9896a62cd07b7a5be26</id>
<content type='text'>
3-seed result on the existing dfa_s{42,123,456}.pt checkpoints from
results/confirmatory/checkpoints_A2/, computing per-layer cosine of
DFA's local credit signal e_T@B_l^T vs the true BP gradient at h_l.

Key findings:
  per-layer cos (3-seed mean):
    l0: +0.42  (high — embedding alignment)
    l1: +0.006 (essentially zero)
    l2: -0.015 (essentially zero)
    l3: -0.004 (essentially zero)
    l4: -0.004 (essentially zero)
  layer-mean across all 5: +0.07-0.10

The deep blocks (l1-l4) have essentially zero alignment with BP grad in
the vanilla scale-failure regime. Layer 0 dominates the headline.

The script reconstructs the training-time random Bs by replaying the RNG
sequence (torch.manual_seed + ResidualMLP construction + randn draws),
since the existing checkpoints don't save Bs. For the still-running
direction-quality experiment which DOES save Bs, the script auto-detects
the dict format and uses the saved Bs directly.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
3-seed result on the existing dfa_s{42,123,456}.pt checkpoints from
results/confirmatory/checkpoints_A2/, computing per-layer cosine of
DFA's local credit signal e_T@B_l^T vs the true BP gradient at h_l.

Key findings:
  per-layer cos (3-seed mean):
    l0: +0.42  (high — embedding alignment)
    l1: +0.006 (essentially zero)
    l2: -0.015 (essentially zero)
    l3: -0.004 (essentially zero)
    l4: -0.004 (essentially zero)
  layer-mean across all 5: +0.07-0.10

The deep blocks (l1-l4) have essentially zero alignment with BP grad in
the vanilla scale-failure regime. Layer 0 dominates the headline.

The script reconstructs the training-time random Bs by replaying the RNG
sequence (torch.manual_seed + ResidualMLP construction + randn draws),
since the existing checkpoints don't save Bs. For the still-running
direction-quality experiment which DOES save Bs, the script auto-detects
the dict format and uses the saved Bs directly.
</pre>
</div>
</content>
</entry>
</feed>
