|
FA does NOT drop below frozen 0.349 at deeper L on d=256:
L=8: FA mean 0.394, min 0.386 (gap +3.7pp)
L=12: FA mean 0.391, min 0.368 (gap +1.9pp)
FA accuracy is essentially depth-invariant (~0.39) even though FA
deep cosine drops from +0.13 (L=8) to +0.09 (L=12). DFA is always
below frozen (~0.27-0.30).
Conclusion: on CIFAR-10 with d=256 ResMLP, FA is too good at L≥4
to fail the frozen baseline. The only qualifying setting at deeper L
would require ~20+ seeds to find a rare 2σ outlier. The d=512 L=2
setting (seeds 1,2,5) remains the cleanest qualifying case.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
