<feed xmlns='http://www.w3.org/2005/Atom'>
<title>faeval.git/protocol/CHECKLIST.md, branch master</title>
<subtitle>Unnamed repository; edit this file 'description' to name the repository.
</subtitle>
<link rel='alternate' type='text/html' href='https://git.blackhao.com/faeval.git/'/>
<entry>
<title>CHECKLIST pitfall #6: layer-0 dominance is ResMLP-specific, not universal</title>
<updated>2026-04-08T04:58:04+00:00</updated>
<author>
<name>YurenHao0426</name>
<email>Blackhao0426@gmail.com</email>
</author>
<published>2026-04-08T04:58:04+00:00</published>
<link rel='alternate' type='text/html' href='https://git.blackhao.com/faeval.git/commit/?id=ab1b783c7a4f3d586d082ba142d7c046453a310c'/>
<id>ab1b783c7a4f3d586d082ba142d7c046453a310c</id>
<content type='text'>
Verified by extracting per-layer gamma_dfa from existing ViT-Mini snapshot
JSON (3 seeds, final epoch). On ViT all 4 layers have per-layer cosine
near zero (~0.001 with eps clamp); no layer dominates. Compare to ResMLP
where layer 0 has +0.42 and layers 1-4 are essentially zero.

The pitfall is real on ResMLP but the specific 'layer 0 dominates' framing
doesn't generalize to ViT. Reframed as 'aggregation hides per-layer
structure'; lesson is to always report per-layer values regardless of
which architecture-specific pattern you might be hiding.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Verified by extracting per-layer gamma_dfa from existing ViT-Mini snapshot
JSON (3 seeds, final epoch). On ViT all 4 layers have per-layer cosine
near zero (~0.001 with eps clamp); no layer dominates. Compare to ResMLP
where layer 0 has +0.42 and layers 1-4 are essentially zero.

The pitfall is real on ResMLP but the specific 'layer 0 dominates' framing
doesn't generalize to ViT. Reframed as 'aggregation hides per-layer
structure'; lesson is to always report per-layer values regardless of
which architecture-specific pattern you might be hiding.
</pre>
</div>
</content>
</entry>
<entry>
<title>CHECKLIST: add pitfall #6.5 — silent self-cosine fallback when aux nets not saved</title>
<updated>2026-04-08T04:48:09+00:00</updated>
<author>
<name>YurenHao0426</name>
<email>Blackhao0426@gmail.com</email>
</author>
<published>2026-04-08T04:48:09+00:00</published>
<link rel='alternate' type='text/html' href='https://git.blackhao.com/faeval.git/commit/?id=ec20a776e0c3e026236942fe99f3840a39e211fd'/>
<id>ec20a776e0c3e026236942fe99f3840a39e211fd</id>
<content type='text'>
Discovered in our own cnn_baseline.py: when the random feedback Bs (for
DFA) or bridge predictor (for SB/CB) are not persisted alongside the
model checkpoint, post-hoc Gamma computation cannot reconstruct the
local credit signal. Instead of erroring, the script falls back to
cos(BP_grad, BP_grad) = 1.0 and records that as Gamma. Reader who
doesn't notice the small 'Gamma_note' field interprets 1.0 as perfect
alignment.

Recommendation: always save aux nets alongside checkpoints; if they're
missing, report Gamma as N/A, not 1.0.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Discovered in our own cnn_baseline.py: when the random feedback Bs (for
DFA) or bridge predictor (for SB/CB) are not persisted alongside the
model checkpoint, post-hoc Gamma computation cannot reconstruct the
local credit signal. Instead of erroring, the script falls back to
cos(BP_grad, BP_grad) = 1.0 and records that as Gamma. Reader who
doesn't notice the small 'Gamma_note' field interprets 1.0 as perfect
alignment.

Recommendation: always save aux nets alongside checkpoints; if they're
missing, report Gamma as N/A, not 1.0.
</pre>
</div>
</content>
</entry>
<entry>
<title>Add FA diagnostic protocol reference implementation</title>
<updated>2026-04-08T03:20:48+00:00</updated>
<author>
<name>YurenHao0426</name>
<email>Blackhao0426@gmail.com</email>
</author>
<published>2026-04-08T03:20:48+00:00</published>
<link rel='alternate' type='text/html' href='https://git.blackhao.com/faeval.git/commit/?id=7b64702ad970c16171142665365e16a8e1737190'/>
<id>7b64702ad970c16171142665365e16a8e1737190</id>
<content type='text'>
Codex round 15 #1 priority for the E&amp;D-track paper:
  - protocol/protocol.py: 4 diagnostics (residual norms, BP grad norms,
    cross-batch direction stability, and a frozen-baseline comparator)
  - protocol/report.py: DiagnosticReport with per-diagnostic verdicts and
    pretty-printer
  - protocol/smoke_test.py: validates BP/DFA/EP checkpoints produce the
    expected verdicts (BP/EP trustworthy; DFA walked back via residual
    explosion + BP grad at floor)
  - protocol/README.md: usage, audit cases, threshold rationale
  - protocol/CHECKLIST.md: 6 evaluation pipeline pitfalls (norm(-1),
    cosine_similarity eps clamp, fp16 underflow, Bs reproducibility,
    aggregation, layer-0 dominance)
  - protocol/REPORTING_TEMPLATE.md: per-method fillable form for FA papers
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Codex round 15 #1 priority for the E&amp;D-track paper:
  - protocol/protocol.py: 4 diagnostics (residual norms, BP grad norms,
    cross-batch direction stability, and a frozen-baseline comparator)
  - protocol/report.py: DiagnosticReport with per-diagnostic verdicts and
    pretty-printer
  - protocol/smoke_test.py: validates BP/DFA/EP checkpoints produce the
    expected verdicts (BP/EP trustworthy; DFA walked back via residual
    explosion + BP grad at floor)
  - protocol/README.md: usage, audit cases, threshold rationale
  - protocol/CHECKLIST.md: 6 evaluation pipeline pitfalls (norm(-1),
    cosine_similarity eps clamp, fp16 underflow, Bs reproducibility,
    aggregation, layer-0 dominance)
  - protocol/REPORTING_TEMPLATE.md: per-method fillable form for FA papers
</pre>
</div>
</content>
</entry>
</feed>
