blob: 32b632532e93148beccf31d45fa192e85e8974ff (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
|
# Phase 5B Memo: Frozen CIFAR Vector Credit Transfer
**Date**: 2026-03-24
**Config**: CIFAR-10, frozen BP reference (L=4, d=256, 61.7% acc), 100 epochs estimator training
## Question
Can the direct vector credit field recover better credit than scalar CB on frozen real-task representations?
## Results
| Method | mean Gamma | mean rho | mean nudge |
|--------|-----------|---------|-----------|
| DFA | 0.005 | 0.005 | -0.000006 |
| ScalarCB_eT | 0.115 | 0.120 | -0.000370 |
| StateBridge_eT | 0.287 | 0.264 | -0.000957 |
| **Vec_eT_M4** | **0.364** | **0.426** | **-0.001406** |
| Vec_eT_M8 | 0.364 | 0.396 | -0.001379 |
| Vec_eT_M16 | 0.368 | 0.422 | -0.001393 |
## Key Findings
1. **Transfer SUCCESS**: Vector field outperforms scalar CB by +0.25 Gamma and +0.31 rho (both >> 0.05 threshold).
2. **Vector field surpasses state bridge on rho** (0.43 vs 0.26), the most important no-BP-needed metric. On Gamma, vector field (0.36) is slightly above state bridge (0.29).
3. **M=4 is sufficient** on d=256 frozen CIFAR. No improvement from M=8 or M=16. The perturbation target provides enough signal even with 4 directions in 256 dimensions.
4. **Layer gradient**: Vector field credit quality increases with depth (layer 3: Gamma=0.61, rho=0.68). This is consistent with the terminal matching loss being strongest at the last layer.
## Verdict
**TRANSFER SUCCESS.** Proceed to Phase 5C (online shallow CIFAR).
Best config for Phase 5C: vec_eT_M4 (cheapest, equally good).
|
