diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-06-14 04:06:32 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-06-14 04:06:32 -0500 |
| commit | aa73718eb6427d7da3b9cb416275802d90c4b2ed (patch) | |
| tree | b68b0a664fb650744ef934a1c22abd740a7b62a6 /results/vit_frozen_blocks_s42.log | |
| parent | 827c658fa9a750f3c6ebdb87703762f10f69f6ff (diff) | |
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Diffstat (limited to 'results/vit_frozen_blocks_s42.log')
| -rw-r--r-- | results/vit_frozen_blocks_s42.log | 34 |
1 files changed, 34 insertions, 0 deletions
diff --git a/results/vit_frozen_blocks_s42.log b/results/vit_frozen_blocks_s42.log new file mode 100644 index 0000000..c2a20f6 --- /dev/null +++ b/results/vit_frozen_blocks_s42.log @@ -0,0 +1,34 @@ +Device: cuda:0 + +=== BP frozen-blocks baseline (4 random-init transformer blocks, frozen) === +BP-frozen-blocks: 16266/809354 params trainable + BP-frozen ep 1: test_acc=0.3762 + BP-frozen ep 5: test_acc=0.4724 + BP-frozen ep 10: test_acc=0.4961 + BP-frozen ep 15: test_acc=0.5189 + BP-frozen ep 20: test_acc=0.5252 + BP-frozen ep 25: test_acc=0.5366 + BP-frozen ep 30: test_acc=0.5402 +FINAL BP-frozen-blocks acc: 0.5402 + +=== DFA frozen-blocks baseline === +DFA-frozen-blocks: 16266/809354 params trainable + DFA-frozen ep 1: test_acc=0.2529 + DFA-frozen ep 5: test_acc=0.2477 + DFA-frozen ep 10: test_acc=0.2530 + DFA-frozen ep 15: test_acc=0.2566 + DFA-frozen ep 20: test_acc=0.2530 + DFA-frozen ep 25: test_acc=0.2545 + DFA-frozen ep 30: test_acc=0.2554 +FINAL DFA-frozen-blocks acc: 0.2554 + +=== Summary === +BP-frozen-blocks: 0.5402 (chance=0.10) +DFA-frozen-blocks: 0.2554 +Compare to ViT-Mini 4-block trainable (3-seed avg): BP=0.792, DFA=0.237 +Compare to ViT-Mini 0-block (shallow baseline): BP=0.10, DFA=0.10 + +Interpretation: + If DFA-frozen-blocks ≈ 0.237: blocks are passengers, DFA is just learning patch_embed+head + If DFA-frozen-blocks << 0.237: trainable blocks ARE doing learned work + If DFA-frozen-blocks ~ 0.10: untrained blocks add no useful mixing (less informative) |